Files
software-infrastructure/docs/ci-cd/adding-ci.md
T
claudeandClaude Opus 5.5 78ae47bf11 G8: bump ericomeehan.gitea for the Actions cache; document it
Picks up DevOps/ansible-role-gitea@96e5b92, which turns on the runner's
built-in cache server with a persistent volume. runner.md describes the
cache and how to troubleshoot or clear it, and corrects the DinD CPU limit
to the 750m now set for the current runner node. adding-ci.md shows how a
workflow uses actions/cache, pinned by SHA.

Refs #56

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 09:24:30 -05:00

238 lines
10 KiB
Markdown

# Adding CI to a repo (G2)
Refs: DevOps/software-infrastructure#50.
Reusable workflows live in this repo under `.gitea/workflows/reusable-*.yaml`.
A repo adds a small caller file that points at one of them. Every lint job runs
in the shared CI tools image (`gitea.eom.dev/devops/ci-tools`, G1 #49), so
nothing is installed at run time and versions are the same everywhere.
> **Status (2026-11-14):** written and tested locally, not yet run on the
> runner. The tools image isn't published yet (G1), and the runner is still
> being stabilised (S8 #41). Don't add callers to repos until both are done.
> G5 (#53) and G6 (#54) do the rollout.
## The workflows
| Workflow | For | Runs |
|---|---|---|
| `reusable-ansible-role-lint.yaml` | `ansible-role-*` repos | `ansible-lint`, which also applies yamllint's rules |
| `reusable-helm-lint.yaml` | Helm chart repos | `helm lint`, then `helm template` checked by `kubeconform` |
| `reusable-dockerfile-lint.yaml` | Repos with a Dockerfile | `hadolint` |
| `reusable-secret-scan.yaml` | Every repo | `gitleaks`, see [secret-scanning.md](secret-scanning.md) |
## Caller templates
Put one file per check in the repo's `.gitea/workflows/`. Pin the reusable
workflow to `@main` for now. Once these have releases (P8), pin a tag.
Every caller should have a `concurrency` group and, where it helps, path
filters, so a burst of pushes doesn't queue many runs on the shared runner.
### Ansible role
```yaml
name: lint
on:
push:
branches: [main]
pull_request:
concurrency:
group: lint-${{ github.ref }}
cancel-in-progress: true
jobs:
ansible:
uses: DevOps/software-infrastructure/.gitea/workflows/reusable-ansible-role-lint.yaml@main
```
| Input | Default | Notes |
|---|---|---|
| `tools-image` | `gitea.eom.dev/devops/ci-tools:1.0.0` | |
| `role-name` | `ericomeehan.` + repo name without `ansible-role-` | The name the playbooks use, e.g. `ericomeehan.gitea`. |
| `ansible-lint-args` | empty | Extra flags, e.g. `--profile production`. |
How it works and why:
- ansible-lint reads a role's name from its directory, so the job lints a copy
named like the playbooks name it. The repo's own name also resolves, because
galaxy's skeleton `tests/test.yml` uses it, and that playbook should be
syntax-checked rather than baselined (below).
- `role-name` is a warning, not a failure. Our roles keep galaxy's skeleton
`meta/main.yml` and are called `ericomeehan.<name>`, which galaxy's naming
rule rejects.
- The repo's `.ansible-lint`, `.yamllint` and `.ansible-lint-ignore` are used
if present.
**Baseline for existing problems.** Most roles have many existing findings
(ansible-role-gitea has 48, mostly `name[casing]` and `fqcn`). To pass today
and fail only on new problems, commit a baseline:
```bash
docker run --rm -v "$PWD:/src" -w /tmp gitea.eom.dev/devops/ci-tools:1.0.0 bash -c '
mkdir -p /tmp/roles && cp -a /src /tmp/roles/ericomeehan.NAME &&
ln -s /tmp/roles/ericomeehan.NAME /tmp/roles/ansible-role-NAME &&
ln -s /tmp/roles/ericomeehan.NAME /tmp/roles/NAME &&
cd /tmp/roles/ericomeehan.NAME &&
ANSIBLE_ROLES_PATH=/tmp/roles ansible-lint --warn-list role-name --exclude .gitea --generate-ignore;
cp .ansible-lint-ignore /src/'
```
Replace `NAME`. Baselined findings still print, as warnings. Fixing them later
means deleting lines from `.ansible-lint-ignore`.
The baseline is per file and rule, not per line. A new finding of a rule that
is already baselined in the same file (say a second `name[casing]` in
`tasks/main.yml`) only warns. New rules, and any finding in a new file, fail.
Shrink the baseline as files are cleaned up, so it covers less over time.
`.gitea/` is excluded: workflow files are checked by actionlint, not
ansible-lint.
**Don't baseline `syntax-check[*]`.** `--generate-ignore` will write it, but
an ignored syntax check means that playbook is never parsed. Skeleton
`tests/test.yml` files sometimes name a role that doesn't exist (a copy-paste
from another role); fix the name to the repo's name instead. G6 did this for
tor-controller and mediawiki. dex's `playbooks/deploy-eom.yml` is still
baselined this way (see #53).
**Rolled out, manual-only until S8 and G1:**
| Batch | Roles |
|---|---|
| G5 (#53) | gitea, www, dex, eom, discourse, mcp-servers, listmonk, prometheus |
| G6 (#54) batch 1 | debian, libvirt-guest, ingress-nginx, grafana, nextcloud, minio, tor-controller, mediawiki |
Not rolled out yet: see #54 for the remaining list and the roles that need a
decision first.
### Helm chart
```yaml
name: lint
on:
push:
branches: [main]
pull_request:
concurrency:
group: lint-${{ github.ref }}
cancel-in-progress: true
jobs:
helm:
uses: DevOps/software-infrastructure/.gitea/workflows/reusable-helm-lint.yaml@main
with:
chart-path: .
```
| Input | Default | Notes |
|---|---|---|
| `tools-image` | `gitea.eom.dev/devops/ci-tools:1.0.0` | |
| `chart-path` | `.` | Directory with `Chart.yaml`. |
| `kube-version` | `1.31.2` | The cluster's version on 2026-11-14. Bump it when the cluster is upgraded. |
| `values-files` | empty | Values files relative to the chart, space-separated. Each one is linted and rendered on its own. |
| `render-defaults` | `true` | Also check the chart's defaults alone. Set `false` when templates `required` values the defaults leave empty. |
| `strict` | `false` | `helm lint --strict`. |
Chart dependencies are built if `Chart.yaml` has any. kubeconform skips kinds
it has no schema for (CRDs) rather than failing on them. Schemas are fetched
from GitHub at run time. The job fails if a values set renders no resources,
because kubeconform alone would pass an empty render.
**Charts that need values to render.** Put CI-only placeholder values in the
chart's `ci/` directory (the chart-testing convention), for example
`ci/minimal-values.yaml`, set `render-defaults: false`, and list the files in
`values-files`. Use obvious placeholders (`example.com`,
`ci-placeholder-not-a-secret`), never real settings.
**Rolled out (G7, #55), manual-only until S8 and G1:**
| Repo | Values checked |
|---|---|
| DevOps/helm-jamulus | defaults |
| DevOps/helm-mailman3 | `ci/minimal-values.yaml` (SQLite, postfix sidecar), `ci/postgresql-values.yaml` (the deployed shape) |
| DevOps/helm-pixelfed | upstream's `test-values/postgresql-plain.yaml`. Prepared on branch `ci/helm-lint`, not `main`: adding `.gitea/workflows/` there stops Gitea running upstream's `.github/workflows`, which is Eric's call (#46). |
### Dockerfile
```yaml
name: lint
on:
push:
branches: [main]
paths: ["**/Dockerfile*", "**/*.Dockerfile", ".hadolint.yaml"]
pull_request:
paths: ["**/Dockerfile*", "**/*.Dockerfile", ".hadolint.yaml"]
concurrency:
group: dockerfile-lint-${{ github.ref }}
cancel-in-progress: true
jobs:
hadolint:
uses: DevOps/software-infrastructure/.gitea/workflows/reusable-dockerfile-lint.yaml@main
```
| Input | Default | Notes |
|---|---|---|
| `tools-image` | `gitea.eom.dev/devops/ci-tools:1.0.0` | |
| `dockerfiles` | every `Dockerfile*` and `*.Dockerfile` | Space-separated paths. The job fails if it finds none. |
| `failure-threshold` | `warning` | `error`, `warning`, `info` or `style`. Use a `.hadolint.yaml` for per-rule ignores. |
## Caching dependencies (G8)
The runner serves the Actions cache, so `actions/cache` works as on GitHub.
Use it for downloads that are slow and change rarely (npm, pip, Go modules,
Ansible collections). Don't cache build outputs that are cheap to rebuild.
```yaml
- name: restore npm cache
uses: actions/cache@caa296126883cff596d87d8935842f9db880ef25 # v5.1.0
with:
path: ~/.npm
key: npm-${{ hashFiles('package-lock.json') }}
restore-keys: npm-
```
- Pin by SHA, like every other action here (see
[actions-inventory.md](actions-inventory.md)). v4.3.0
(`0057852bfaa89a56745cba8c7296529d2fc39830`) also works.
- Put a hash of the lock file in the key, so a dependency change makes a new
entry instead of restoring a stale one.
- Entries are per repository. Unused entries expire after 7 days, and every
entry after 30. The first run on a new key is always a miss.
- Never cache anything that holds a secret. Anyone who can run a workflow in
the repo can restore its cache.
- If the cache is down, the step warns and the job carries on without it.
## How these were tested (2026-11-14)
Each workflow's `run:` steps were taken from the YAML as written and run inside
a local build of the tools image (1.0.0), against:
| Workflow | Case | Result |
|---|---|---|
| ansible-role-lint | DevOps/ansible-role-gitea as it is | fails, 48 findings |
| | the same with a generated baseline | passes |
| | baseline plus a new `shell:` task | fails on the new task only |
| role lint (2026-11-16, G5 batch 2) | eom, discourse, mcp-servers, listmonk, prometheus with generated baselines, and gitea, www, dex again | all pass |
| | listmonk plus a new `tasks/extra.yml` with an unnamed `shell:` | fails on the new file |
| | eom with the previous step (no bare-name link) | fails: `tests/test.yml` names the role `eom` |
| role lint (2026-11-16, G6 batch 1) | debian, libvirt-guest, ingress-nginx, grafana, nextcloud, minio, tor-controller, mediawiki with generated baselines | all pass |
| | each of them plus a new `tasks/ci_plant.yml` with an unnamed `shell:` | all fail, on the new file |
| helm-lint | DevOps/helm-jamulus | passes |
| | a template referencing a missing value | fails in `helm lint` |
| | a Service with a string port | fails in kubeconform |
| helm-lint (2026-11-15, G7) | helm-mailman3 with `ci/` values, helm-pixelfed with its test values | pass (7 + 15, and 27 resources valid) |
| | helm-mailman3 with defaults only | fails in `helm lint` (missing `admin`) |
| | helm-mailman3 at 5facaf7 (before the G7 fix) | fails in `helm lint` (template parse error) |
| | a chart with no templates | fails: rendered no resources |
| | `render-defaults: false` with no values files | fails with a clear message |
| dockerfile-lint | `images/ci-tools/Dockerfile` | passes |
| | `FROM debian:latest` with unpinned apt | fails (DL3007, DL3008) |
| | repo with no Dockerfile | fails with a clear message |
`actionlint` 1.7.12 and `yamllint` pass on all the workflow files.
**Not tested yet:**
- Running on the Gitea runner at all, including `container.image` from an input.
- Calling a workflow from another repo with `uses: owner/repo/path@ref`, and
from a private repo calling this public one. This is the G2 acceptance check.
If the runner can't resolve cross-repo `uses:`, the fallback is to copy the
caller with the steps inlined, generated from these files.