Post Code-Freeze Review, Bugsfixes and Documentation

- Fix traefik Portal Button - Touchup Security Policies - Update Support Policies - Re-Add Deluge as incubator App - Add Canary to Security Policies - Add all-contributors basic config
2021-04-09 13:34:29 +02:00
parent 56211222bf
commit cde18639c7
568 changed files with 2295 additions and 1614 deletions
@@ -0,0 +1,23 @@
+# Migration Guides
+
+Sometimes we can not guarantee automatic updating between new versions of our Apps.
+These pages include some common scenario's and give some guidance how to manually move between different versions and Trains.
+
+##### Between Trains
+
+An app will always be just in one train. If an App moves to a different train, you will not be able to update automatically
+Our basic guidence for these cases is:
+
+
+- Backup all your files, config and databases.
+- Make notes on how you set up your app previously
+- Delete the old App
+- Install the new app using your previously made setup notes
+- Stop the new App
+- Put back all your old files.
+- Re start the new App
+
+##### Between Versions
+
+When new versions introduce breaking changes (which would often be major version changes) they might(!) require the user to reinstall.
+However: We often already provide migration scrips and dealth with these issues manually.
@@ -0,0 +1,43 @@
+# Adding TrueCharts to SCALE
+
+Adding the TrueCharts Community App Catalog is relatively straight forwards.
+
+##### Requirements
+
+- Make sure your storage-pool is created and working
+- Make Make sure you selected your storage-pool when you first opened the "Apps" interface, if not please refert to the TrueNAS SCALE documentation how to setup your Apps using the apps settings menu.
+- Make sure you have a working internet connection and can reach github and truecharts.org from the host system.
+
+##### Adding TrueCharts
+
+- Go to "Apps" in the left hand menu
+- Select the "Manage Catalogs" tab
+- Click "Add Catalog" and enter the required information:
+
+Name: `truecharts`
+
+Repository: `https://github.com/truecharts/apps`
+
+Preferred Trains: `stable` (and optionally: `incubator`)
+
+Branch: `master`
+
+##### Difference between Stable and Incubator
+
+TrueCharts has 2 trains: `Staging` and `Incubator`. Both trains contain Apps that should work fine. However they have a slightly different meaning:
+
+`Staging` Contains Apps that have an active maintainer that uses and maintains these Apps. This person guarantees that the Apps should function without issues. They also are reviewed more strictly before being accepted into `Stable`, for both security and stability.
+
+`Incubator` Contains Apps that either do not have an active maintainer or are not fully reviewed to comply to all our technical guidelines. They should, however, still function without issues.
+
+##### Adding TrueCharts on TrueNAS SCALE 21.02ALPHA
+
+Because we are very close to release of TrueNAS SCALE 21.04ALPHA and we have made considerably bugfixes and rewrites of code already,  TrueNAS SCALE 21.02ALPHA is considered to be depricated by the TrueCharts team.
+However: Your old installed apps should still be available in the "installed applications" section.
+
+In case you really need to add the old "legacy" version of TrueCharts, you can install a special legacy branch named `legacy_21.02ALPHA`
+
+
+##### Notes
+
+- If this doesn't work right away, try clicking "Refresh Catalogs".
@@ -0,0 +1,84 @@
+# How to: Create Certificates
+
+At TrueCharts we support HTTPS deployments of every app using our Traefik Reverse Proxy. We support both self-signed, custom and lets-encrypt certificates, using the TrueNAS SCALE building certificate manager. Available under "Credentials"
+
+### Self Signed certificates
+
+Self signed certificates are relatively straight forward and handled by Traefik itself. You just select "self signed" when adding a reverse-proxy to your App and Traefik does the rest!
+
+Please be aware that these certificates are not really secure, but are "good enough" for testing.
+
+### Lets-Encrypt Certificates
+
+With the current version of TrueNAS SCALE, it's possible to automatically generate certificates for your domain(s) using letsencrypt. However, this process is not very clear, hence we added a short how-to guide on getting up-and-running with TrueNAS SCALE and Letsencrypt.
+After you managed to complete this how-to, you should be able to select "iX Certificate" as certificate option and your personal certificate in the other drop-down box!
+
+##### Requirements
+
+To use iX Certificates with letsencrypt there are a few requirements:
+- Preferably use a DNS server that doesn't have any caching (no local DNS server) for your TrueNAS system.
+- Have an email address entered for your TrueNAS SCALE `root` user. (this email will also be used for letsencrypt reminder!)
+- Own a domain name
+- Use either Cloudflare or AWS Route53 for your domain. (In case you wonder: Using Cloudflare as DNS provider is free)
+- Have an active internet connection so TrueNAS SCALE can contact Cloudflare or AWS to verify your domain ownership
+
+##### How-To
+
+- Click `Credentials` in the Left side menu and go the `Certificates` page.
+
+
+TODO: Insert Screenshot
+
+
+- Notice `ACME DNS-Authenticators`, select `Add` besides `ACME DNS-Authenticators` to open the menu for adding your DNS provider for domain verification.
+
+
+TODO: Insert Screenshot
+
+
+- Enter the required information and click `save`.
+For Cloudflare you need either a global API-Key or a limited-scope API token. Please refer to cloudflare and/or AWS on how to get the required credentials.
+
+
+TODO: Insert Screenshot
+
+
+- Notice `Certificate Signing Requests`, select `Add` besides `Certificate Signing Requests` to open the menu for adding the domain information you want a certificate for.
+
+TODO: Insert Screenshot
+
+
+- Enter all information required in the wizard and save it.
+If you are not sure, the defaults are alsmost always "alright", because most of what you enter here is completely ignored by Letsencrypt.
+`Common Name` in this case means `Primary domain name`, whereas `Subject Alternate Names` means `Extra domain names`.
+
+
+TODO: Insert Screenshot
+
+
+- Notice your new `Certificate Signing Request` showing up in the box below `Certificate Signing Requests`. Also notice the small `wrench` icon to the right of your `Certificate Signing Request`
+
+
+- Click the small `wrench` icon, this will open the `Create ACME Certificate` menu. In this menu we can actually request either a real (Production) certificate or a testing (staging) certificate from Letsencrypt.
+For clearity, it's advicable to use the same Authenticator for all domain names. However: It's okey to generate both a testing and a staging certificate for the same domain.
+
+- After saving and awaiting the generation proces, you should end up with another `Certificate Signing Request` and a new `Certificate` under `Certificates`, this new `Certificate Signing Request` is used to renew your `Certificate` in the future and should not be deleted!
+
+### Import existing certificates
+
+TrueNAS SCALE also allows you to manually import certificates, this is rather straight forward:
+Copy-Paste the keys into their respective boxes and hit `save`
+
+
+### Notes
+
+There are a few known bugs, issues and/or oddities currently in regards to Certificates
+
+##### Traefik not accepting/using certificates
+
+Sometimes you might notice Traefik ignores your certificate. This is most likely due to the domain on your certificate, being different from the domain you entered into the reverse proxy host box.
+Traefik requires your certificate to match the domain used for Ingress. This is an upstream design decision and something we can easily and safely disable.
+
+##### New certificates not showing without refresh
+
+Currently recently added certificates will not show in the App UI, without hitting the button to refresh the App list.
@@ -0,0 +1,48 @@
+# Clustering
+
+One of the eventual goals of SCALE is to run hyper-converged clusters based on ZFS, Gluster and Kubernetes.
+While this is awesome, we like to highlight 3 ways of using Clustering with TrueCharts Apps in the future.
+
+Currently no clustering is supported with TrueCharts yet, but we already make precautions during our design phase to implement clustering smoothly in the future.
+
+
+
+##### Single-Pod Flexible
+
+These Apps (can) only run a single instance of a pod at a time, so no "high available" setup available. However these Apps can still dynamically moved over to different nodes if a node fails, this should make sure the downtime stays relatively small.
+These apps can also be "spread" over all nodes by kubernetes. This also means it's worth cutting big Single-Pod Apps into multiple smaller deployements, For example: n we prefer to deploy small database servers with every App (as those can more dynamically be spread over multiple nodes) over one Big single-pod Flexbile App.
+
+Some examples of Apps that can not run with more than one pod at a time, even if we wanted to, are:
+
+- Sonarr
+- Lidarr
+- Radarr
+- Plex
+
+
+
+##### Single-Pod non-flexible
+
+These Apps can not be run with more than 1 pod at a time and on the other hand are bound to a node.
+This means: Host-Down? Pod-Down!
+
+Almost always this is caused by Apps being bound to a specific hardware setup.
+
+*Examples:*
+
+- zwavejs2mqtt
+- HomeAssistant (depending on added hardware)
+- Handbrake (depending on added hardware)
+
+
+
+##### High Availability Apps
+
+These Apps are designed to be the most resiliant of all, they can handle node failures and pod failures without any issue, because they always run multiple Pods at once that fill in once pods or nodes start failing.
+
+Because these Apps are also the most complicated of all, we try to limit them to key-area's of the TrueCharts ecosystem that can not be cut into smaller pieces (Single-Pod Flexible) easily. Like: Ingress and Monitoring.
+
+*Examples:*
+
+
+- Traefik
@@ -0,0 +1,24 @@
+# General Configuration
+
+There are some setting that are always present on every App from TrueCharts.
+
+##### Explaination sheet:
+
+- **timezone**
+
+_Select your timezone for correct time inside the containers_
+
+
+- **PUID**
+
+_Select the user id of the user running your application. Defaults to 568 aka the `apps` user_
+
+
+- **PGID**
+
+_Select the group of the user running your application. Defaults to 568 aka the `apps` group_
+
+
+- **UMASK**
+
+_The mask here gets substracted from the default CHOWN setting of your App, it's an advanced setting and we advice not to change this unless you already know what UMASK does_.
@@ -0,0 +1,22 @@
+# Linking Apps Together
+
+We often need to connect individual apps together, for example: Ombi and Plex. This means we first need to know how to reach those Apps.
+
+##### Linking Apps Internally
+
+The backend for TrueNAS SCALE Apps is Kubernetes. Linking apps together in kubernetes is done slightly different than in other systems, as you can't point directly to other Containers using their IP-Address.
+
+Instead we need to use their internal(!) domain name. Please be ware: this name is only available between Apps and can not be reached from the host/node or your own PC.
+The format for internal domain name for the main service is as follows, please replace `$APPNAME` with the name you gave your App when installing.
+
+`$APPNAME.ix-$APPNAME.svc.cluster.local`
+
+However, if you need to reach a different service (which is not often the case!), you need a slightly different format, where `$SVCNAME` is the name of the service you want to reach:
+
+`$SVCNAME.ix-$APPNAME.svc.cluster.local`
+
+*For example:*
+
+To reach an app named "plex", we use the following internal domain name:
+
+`plex.ix-plex.svc.cluster.local`
@@ -0,0 +1,30 @@
+# Networking
+
+TrueCharts contain a number of networking options, some super-easy, others quite-advanced. In this document we will try to give a general overview what the general configuration options are and what are their downside and upsides.
+
+### General Choices
+
+##### Host Networking
+
+This option is considered to be an advanced option and is rarely needed. It connects the network-stack of the host to the App.
+
+The reason this is not needed in most Apps, is because we already have great options to deal with most forms of network traffic and every maintainer has the goal to make sure you don't need this setting.
+
+Please refer to the documentation of individual Apps or the Support-Discussions section on github, if you think you might need this setting in your specific usecase
+
+##### ClusterIP
+
+The "Basic" network mode, it create a special load-balancer called a "service" thats only available on the internal network between the Apps.
+
+Don't think you can't connect to it though, because our [Reverse proxy](https://wiki.truecharts.org/general/reverse-proxy/) can forward most traffic for you!
+
+##### NodePort
+
+The name already makes clear what this one does: It connect to a port on your node (the PC hosting your App).
+
+Its a special ClusterIP that forwards all traffic from a certain port on your host-system aka "node", directly to the service. However, it's also still a ClusterIP, so it's very well possible to use both the Reverse proxy and the NodePort, just not at the same port.
+
+There are, however, multiple downsides to using nodeports:
+
+- You can only pick ports above 9000
+- You can not connect two Apps to the same port
@@ -0,0 +1,34 @@
+# Reverse Proxy
+
+Within TrueCharts our aim is to make it as easy as possible to secure your Apps. To support this we supply a seperate Traefik "Reverse Proxy" app, which has been preconfigured to provide secure and fast connections.
+
+To use Traefik as a Reverse Proxy, all you have to do is enable "Reverse Proxy" in the App of your choice and fill out a little form.
+
+### Types of Reverse Proxys
+
+We currently offer the following types of pre-configured reverse proxies:
+
+- HTTP
+
+- HTTP using CRD (Advanced)
+
+- TCP
+
+- UDP
+
+Besides HTTP, all these options, require Traefik to be installed before you enable Reverse Proxy on your App. In many cases, the maintainer of your app has hidden specific settings, like the type of Reverse proxies available, to suit your App.
+
+
+### Adding Certificates
+
+To add certificates to Apps, we use the TrueNAS SCALE certificate storage. This means you first need to add Certificates to TrueNAS SCALE, after which you can select them when Installing or Editing your App.
+
+### Notes
+
+There are a few highlights to take into account when adding a reverse proxy to an App:
+
+##### Adding hosts is required
+
+By default the hosts list is empty, this is due to upstream design choices and is a issue that is yet to be solved upstream.
+
+However: adding hosts (preferably just one) is required for ANY app to function with a reverse proxy enabled. Apps might not install and throw errors if you do not add any hosts.
@@ -0,0 +1,41 @@
+# Storage
+
+TrueCharts tries to stay in sync with the Official IX-Systems App Catalog when it comes to storage. Which is currently pretty much a "work in progres".
+
+### Storage types
+
+Storage is currently seperated into two types:
+1. ix_volumes
+2. hostPath
+
+While, from a k8s point of view, both of these are technically hostPath volumes, for you, as a user, there is a significant difference.
+
+##### ix_volumes
+
+ix_volumes, are the default storage option for every TrueCharts App. They always get created and used unless "hostPath" is checked.
+ix_volumes are fully managed by TrueNAS SCALE, they get created and destroyed on demand when creating, updating or editing an App.
+But, most importantly, they can be reverted if an upgrade goes wrong. Which makes them an great to use for storing config files.
+
+They are, normally, stored in the following directory:
+`/mnt/poolname/ix-applications/releases/releasename/volumes/ix_volumes/`
+
+##### hostPath
+
+Hostpath is quite simple: It mounts a directory from your TrueNAS SCALE system, directly to a directory inside the App you're installing.
+There is, in contrast to the ix_volumes, no "special magic sauce" when adding hostPath storage.
+
+### Permissions
+
+Permission settings are rather important and are often something that causes issues for users.
+For that reason TrueCharts introduced a feature to automatically set permissions individually for each of your storage options.
+
+Setting permissions automatically means we `chown` the folder and all folder within it, to a user and group of your choice.
+However, we only do so when installing or updating an app.
+
+Please be aware that automatically setting ownership/permissions, does mean it overrides your current CHOWN and CHMOD settings. This could break things and yes, it will destroy your system if used carelessly. It's also not wise to enable the automatic permissions on mounted shares from an external system.
+
+These permission get based on the user and group you enter in the App configuration dialog and default to `568` (the SCALE default Apps user).
+
+### Custom Storage
+
+Besides the earlier mentioned storage options, we also provide the option to mount as many host folders as you want. This is limited to hostPath mounts only.