From 5fd65bb3d51a739e5c80317040c762ac6fc06e41 Mon Sep 17 00:00:00 2001 From: Kjeld Schouten Date: Sun, 9 Feb 2025 11:24:43 +0100 Subject: [PATCH] feat(common): enable middlewares when referenced on ingress and add chart+common middlewares (#31535) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit **Description** Currently users have to enable the ingressMiddleware AND reference said middleware under their ingress. This adds additional work and is confusion when, for example, we include our own middleware. This PR checks if a disabled middleware, is referenced anywhere on an enabled ingress with traefik-integration enabled. It does so by excluding any external middlewares (with namespace explicitly defined) If it finds any that are not enabled, it will still enable/create them regardless. This PR also increases the places ingress-middleware can be referenced on the ingress: 1. common-middlewares (.Values.global.traefik.commonMiddlewares) Special bare-bones middlewares that are applied to all ingresses by default, unless very explicitly overridden 2. chart-middlewares ($ingress.integrations.traefik.chartMiddlewares) Special middlewares that are defined by the chart creator and should always be loaded regardless of user middlewares, for example: nextcloud 3. user-middlewares what we had already **โš™๏ธ Type of change** - [x] โš™๏ธ Feature/App addition - [ ] ๐Ÿช› Bugfix - [ ] โš ๏ธ Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] ๐Ÿ”ƒ Refactor of current code **๐Ÿงช How Has This Been Tested?** **๐Ÿ“ƒ Notes:** **โœ”๏ธ Checklist:** - [ ] โš–๏ธ My code follows the style guidelines of this project - [ ] ๐Ÿ‘€ I have performed a self-review of my own code - [ ] #๏ธโƒฃ I have commented my code, particularly in hard-to-understand areas - [ ] ๐Ÿ“„ I have made corresponding changes to the documentation - [ ] โš ๏ธ My changes generate no new warnings - [ ] ๐Ÿงช I have added tests to this description that prove my fix is effective or that my feature works - [ ] โฌ†๏ธ I increased versions for any altered app according to semantic versioning - [ ] I made sure the title starts with `feat(chart-name):`, `fix(chart-name):` or `chore(chart-name):` **โž• App addition** If this PR is an app addition please make sure you have done the following. - [ ] ๐Ÿ–ผ๏ธ I have added an icon in the Chart's root directory called `icon.png` --- _Please don't blindly check all the boxes. Read them and only check those that apply. Those checkboxes are there for the reviewer to see what is this all about and the status of this PR with a quick glance._ --------- Signed-off-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com> Co-authored-by: Stavros Kois Co-authored-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com> --- charts/library/common-test/Chart.yaml | 2 +- .../tests/ingress/traefik_test.yaml | 16 ++ .../tests/traefik/enabled_test.yaml | 166 ++++++++++++++++++ charts/library/common-test/values.yaml | 2 + charts/library/common/Chart.yaml | 2 +- .../lib/ingress/integrations/_traefik.tpl | 15 +- .../templates/spawner/traefik/_middleware.tpl | 75 +++++++- charts/library/common/values.yaml | 2 + website/src/content/docs/common/global.md | 25 +++ .../content/docs/common/ingress/traefik.md | 28 +++ 10 files changed, 326 insertions(+), 7 deletions(-) create mode 100644 charts/library/common-test/tests/traefik/enabled_test.yaml diff --git a/charts/library/common-test/Chart.yaml b/charts/library/common-test/Chart.yaml index a5e228cc56a..195b2cc4970 100644 --- a/charts/library/common-test/Chart.yaml +++ b/charts/library/common-test/Chart.yaml @@ -7,7 +7,7 @@ apiVersion: v2 appVersion: unknown dependencies: - name: common - version: ~26.0.0 + version: ~26.1.0 repository: file://../common/ condition: "" alias: "" diff --git a/charts/library/common-test/tests/ingress/traefik_test.yaml b/charts/library/common-test/tests/ingress/traefik_test.yaml index b5a8eaa99ea..9967e416baa 100644 --- a/charts/library/common-test/tests/ingress/traefik_test.yaml +++ b/charts/library/common-test/tests/ingress/traefik_test.yaml @@ -270,6 +270,22 @@ tests: - failedTemplate: errorMessage: Ingress - Expected [integrations.traefik.middlewares] to be a [slice], but got [string] + - it: should fail with chartMiddlewares not a slice + set: + service: *service + ingress: + my-ingress1: + enabled: true + primary: true + integrations: + traefik: + enabled: true + chartMiddlewares: "not a slice" + hosts: *hosts + asserts: + - failedTemplate: + errorMessage: Ingress - Expected [integrations.traefik.chartMiddlewares] to be a [slice], but got [string] + - it: should fail with duplicate middlewares set: service: *service diff --git a/charts/library/common-test/tests/traefik/enabled_test.yaml b/charts/library/common-test/tests/traefik/enabled_test.yaml new file mode 100644 index 00000000000..3a84d35e8e5 --- /dev/null +++ b/charts/library/common-test/tests/traefik/enabled_test.yaml @@ -0,0 +1,166 @@ +suite: middlewares force enable test +templates: + - common.yaml +chart: + appVersion: &appVer v9.9.9 +release: + name: test-release-name + namespace: test-release-namespace +tests: + - it: should enable middleware when listed under global.traefik.commonMiddlewares + set: + global: + traefik: + commonMiddlewares: + - name: some-middleware + ingressMiddlewares: + traefik: + some-unused-middleware: + enabled: false + type: compress + some-middleware: + enabled: false + type: buffering + data: + foo: bar + service: + my-service: + enabled: true + primary: true + ports: + my-port: + enabled: true + primary: true + port: 80 + ingress: + my-ingress: + enabled: true + primary: true + hosts: + - host: chart-example.local + asserts: + - hasDocuments: + count: 3 + - documentIndex: &middlewareDoc 1 + isKind: + of: Middleware + - documentIndex: *middlewareDoc + equal: + path: metadata.name + value: test-release-name-common-test-some-middleware + - documentIndex: *middlewareDoc + equal: + path: metadata.namespace + value: test-release-namespace + + - it: should not enable middleware when listed under global.traefik.commonMiddlewares if there is no ingress + set: + global: + traefik: + commonMiddlewares: + - name: some-middleware + ingressMiddlewares: + traefik: + some-unused-middleware: + enabled: false + type: compress + some-middleware: + enabled: false + type: buffering + data: + foo: bar + asserts: + - hasDocuments: + count: 0 + + - it: should enable middleware when listed under ingress.traefik.chartMiddlewares + set: + ingressMiddlewares: + traefik: + some-unused-middleware: + enabled: false + type: compress + some-chart-middleware: + enabled: false + type: buffering + data: + foo: bar + service: + my-service: + enabled: true + primary: true + ports: + my-port: + enabled: true + primary: true + port: 80 + ingress: + my-ingress: + enabled: true + primary: true + integrations: + traefik: + chartMiddlewares: + - name: some-chart-middleware + hosts: + - host: chart-example.local + asserts: + - hasDocuments: + count: 3 + - documentIndex: &middlewareDoc 1 + isKind: + of: Middleware + - documentIndex: *middlewareDoc + equal: + path: metadata.name + value: test-release-name-common-test-some-chart-middleware + - documentIndex: *middlewareDoc + equal: + path: metadata.namespace + value: test-release-namespace + + - it: should enable middleware when listed under ingress.traefik.middlewares + set: + ingressMiddlewares: + traefik: + some-unused-middleware: + enabled: false + type: compress + some-user-middleware: + enabled: false + type: buffering + data: + foo: bar + service: + my-service: + enabled: true + primary: true + ports: + my-port: + enabled: true + primary: true + port: 80 + ingress: + my-ingress: + enabled: true + primary: true + integrations: + traefik: + middlewares: + - name: some-user-middleware + hosts: + - host: chart-example.local + asserts: + - hasDocuments: + count: 3 + - documentIndex: &middlewareDoc 1 + isKind: + of: Middleware + - documentIndex: *middlewareDoc + equal: + path: metadata.name + value: test-release-name-common-test-some-user-middleware + - documentIndex: *middlewareDoc + equal: + path: metadata.namespace + value: test-release-namespace diff --git a/charts/library/common-test/values.yaml b/charts/library/common-test/values.yaml index 8761910871d..63ae5ece97d 100644 --- a/charts/library/common-test/values.yaml +++ b/charts/library/common-test/values.yaml @@ -4,6 +4,8 @@ global: createTCNamespace: false + traefik: + commonMiddlewares: [] workload: main: diff --git a/charts/library/common/Chart.yaml b/charts/library/common/Chart.yaml index 60f2e3494dd..42fe1986c25 100644 --- a/charts/library/common/Chart.yaml +++ b/charts/library/common/Chart.yaml @@ -48,4 +48,4 @@ sources: - https://hub.docker.com/_/ - https://hub.docker.com/r/mikefarah/yq type: library -version: 26.0.3 +version: 26.1.0 diff --git a/charts/library/common/templates/lib/ingress/integrations/_traefik.tpl b/charts/library/common/templates/lib/ingress/integrations/_traefik.tpl index 6f14bc1248c..e71b4e42dd2 100644 --- a/charts/library/common/templates/lib/ingress/integrations/_traefik.tpl +++ b/charts/library/common/templates/lib/ingress/integrations/_traefik.tpl @@ -22,7 +22,15 @@ {{- $entrypoints := $traefik.entrypoints | default (list "websecure") -}} {{- $middlewares := list -}} - {{/* Add the user middlewares */}} + {{/* Add the user, common and chart middlewares */}} + {{- if $rootCtx.Values.global.traefik.commonMiddlewares -}} + {{- $middlewares = concat $middlewares $rootCtx.Values.global.traefik.commonMiddlewares -}} + {{- end -}} + + {{- if $traefik.chartMiddlewares -}} + {{- $middlewares = concat $middlewares $traefik.chartMiddlewares -}} + {{- end -}} + {{- if $traefik.middlewares -}} {{- $middlewares = concat $middlewares $traefik.middlewares -}} {{- end -}} @@ -96,5 +104,10 @@ {{- end -}} {{- end -}} + {{- if $traefik.chartMiddlewares -}} + {{- if not (kindIs "slice" $traefik.chartMiddlewares) -}} + {{- fail (printf "Ingress - Expected [integrations.traefik.chartMiddlewares] to be a [slice], but got [%s]" (kindOf $traefik.chartMiddlewares)) -}} + {{- end -}} + {{- end -}} {{- end -}} diff --git a/charts/library/common/templates/spawner/traefik/_middleware.tpl b/charts/library/common/templates/spawner/traefik/_middleware.tpl index 4bb46d519d3..af8669c30f5 100644 --- a/charts/library/common/templates/spawner/traefik/_middleware.tpl +++ b/charts/library/common/templates/spawner/traefik/_middleware.tpl @@ -12,15 +12,82 @@ {{- $_ := set $.Values.ingressMiddlewares "traefik" dict -}} {{- end -}} + {{- $filteredMiddlewares := dict -}} + {{- $hasIngressEnabled := false -}} + {{/* Go over all ingresses and get their defined middlewares */}} + {{- range $ingName, $ing := $.Values.ingress -}} + {{- $enabledIng := (include "tc.v1.common.lib.util.enabled" (dict + "rootCtx" $ "objectData" $ing + "name" $ingName "caller" "Ingress" + "key" "ingress")) -}} + + {{/* Skip disabled ingresses or ingresses without traefik integration */}} + {{- if ne $enabledIng "true" -}}{{- continue -}}{{- end -}} + {{- if not $ing.integrations -}} + {{- $_ := set $ing "integrations" dict -}} + {{- end -}} + {{- if not $ing.integrations.traefik -}} + {{- $_ := set $ing.integrations "traefik" dict -}} + {{- end -}} + {{- $traefik := $ing.integrations.traefik -}} + {{- $enabledTraefikIntegration := "true" -}} + {{- if and (hasKey $traefik "enabled") (kindIs "bool" $traefik.enabled) -}} + {{- $enabledTraefikIntegration = $traefik.enabled | toString -}} + {{- end -}} + {{- if ne $enabledTraefikIntegration "true" }}{{- continue -}}{{- end -}} + + {{- $hasIngressEnabled = true -}} + + {{/* User middlewares */}} + {{- if and $traefik.middlewares (not (kindIs "slice" $traefik.middlewares)) -}}{{- continue -}}{{- end -}} + {{- range $mw := $traefik.middlewares -}} + {{- if $mw.namespace -}}{{- continue -}}{{- end -}} + {{- $_ := set $filteredMiddlewares $mw.name "user-mw" -}} + {{- end -}} + + {{/* Chart middlewares */}} + {{- if and $traefik.chartMiddlewares (not (kindIs "slice" $traefik.chartMiddlewares)) -}}{{- continue -}}{{- end -}} + {{- range $mw := $traefik.chartMiddlewares -}} + {{- if $mw.namespace -}}{{- continue -}}{{- end -}} + {{- $_ := set $filteredMiddlewares $mw.name "chart-mw" -}} + {{- end -}} + + {{- end -}} + + {{- if $hasIngressEnabled -}} + {{/* Global Middlewares */}} + {{- range $mw := $.Values.global.traefik.commonMiddlewares -}} + {{- if $mw.namespace -}}{{- continue -}}{{- end -}} + {{- $_ := set $filteredMiddlewares $mw.name "global-mw" -}} + {{- end -}} + {{- end -}} + {{- range $name, $middleware := $.Values.ingressMiddlewares.traefik -}} {{- $enabled := (include "tc.v1.common.lib.util.enabled" (dict - "rootCtx" $ "objectData" $middleware - "name" $name "caller" "Middleware" - "key" "middlewares")) -}} + "rootCtx" $ "objectData" $middleware + "name" $name "caller" "Middleware" + "key" "middlewares")) + -}} + + {{- if ne $enabled "true" -}} + {{- $indexedMid := get $filteredMiddlewares $name -}} + {{- if not $indexedMid -}}{{- continue -}}{{- end -}} + + {{/* + If current middleware manifest is in the middlewares listed under one of the above sections + Forcefully enable it/render it. + */}} + {{- $enabled = "true" -}} + + {{- if eq $indexedMid "user-mw" -}} + {{- include "add.warning" (dict "rootCtx" $ "warn" (printf + "WARNING: Because middleware [%s] was used in an ingress under traefik integration, it was forcefully enabled." + )) -}} + {{- end -}} + {{- end -}} {{- if eq $enabled "true" -}} - {{/* Create a copy of the middleware */}} {{- $objectData := (mustDeepCopy $middleware) -}} diff --git a/charts/library/common/values.yaml b/charts/library/common/values.yaml index 4bfda98cdf4..eab5abe7708 100644 --- a/charts/library/common/values.yaml +++ b/charts/library/common/values.yaml @@ -63,6 +63,8 @@ global: traefik: # -- Adds traefik annotations to services (when needed) addServiceAnnotations: true + commonMiddlewares: + - name: tc-basic-secure-headers # -- Minimum nodePort value minNodePort: 9000 # -- Enable to stop most pods and containers including cnpg diff --git a/website/src/content/docs/common/global.md b/website/src/content/docs/common/global.md index 68425cd4038..71fd582d2cc 100644 --- a/website/src/content/docs/common/global.md +++ b/website/src/content/docs/common/global.md @@ -217,6 +217,29 @@ global: --- +### `traefik.commonMiddlewares` + +Define middlewares that will be applied to all ingresses + +| | | +| ---------- | ----------------------------------- | +| Key | `global.traefik.commonMiddlewares` | +| Type | `list` | +| Required | โŒ | +| Helm `tpl` | โŒ | +| Default | `[{name: tc-basic-secure-headers}]` | + +Example + +```yaml +global: + traefik: + commonMiddlewares: + - name: tc-basic-secure-headers +``` + +--- + ## Full Examples ```yaml @@ -232,4 +255,6 @@ global: addServiceAnnotations: true traefik: addServiceAnnotations: true + commonMiddlewares: + - name: tc-basic-secure-headers ``` diff --git a/website/src/content/docs/common/ingress/traefik.md b/website/src/content/docs/common/ingress/traefik.md index 58629efde13..37772262291 100644 --- a/website/src/content/docs/common/ingress/traefik.md +++ b/website/src/content/docs/common/ingress/traefik.md @@ -201,6 +201,32 @@ ingress: --- +## `chartMiddlewares` + +Same as [middlewares](#middlewares) but meant to be used by the chart developer +to define some custom middleware specific to this ingress. + +| | | +| ---------- | ----------------------------------------------------- | +| Key | `ingress.$name.integrations.traefik.chartMiddlewares` | +| Type | `list` of `map` | +| Required | โŒ | +| Helm `tpl` | โŒ | +| Default | `[]` | + +Example + +```yaml +ingress: + ingress-name: + integrations: + traefik: + chartMiddlewares: + - name: my-middleware +``` + +--- + ## Full Examples ```yaml @@ -216,4 +242,6 @@ ingress: - name: my-middleware namespace: "" expandObjectName: false + chartMiddlewares: + - name: my-middleware ```