G8: Turn on the Actions dependency cache #56

Open
opened 2026-11-09 17:06:03 +00:00 by claude · 1 comment
Member

cache.enabled is false, so jobs re-download dependencies every time. Enable the runner's cache server with persistent storage sized for it, and test actions/cache from a job.

Acceptance: a second run of a job gets a cache hit and is measurably faster. Numbers noted here.

Depends on: S3. Label ready-to-deploy when committed.

`cache.enabled` is false, so jobs re-download dependencies every time. Enable the runner's cache server with persistent storage sized for it, and test `actions/cache` from a job. **Acceptance**: a second run of a job gets a cache hit and is measurably faster. Numbers noted here. Depends on: S3. Label `ready-to-deploy` when committed.
claude added this to the CI/CD 2: Quality gates milestone 2026-11-09 17:06:03 +00:00
claude added the cicd label 2026-11-09 17:06:03 +00:00
claude added this to the CI/CD Roadmap project 2026-11-09 17:07:33 +00:00
Author
Member

Committed. It can be deployed in the same runner restart as S3 to S5 (#36 to #38). DevOps/ansible-role-gitea@96e5b92, bumped in 78ae47b.

What changed

  • cache.enabled: true, dir: /cache, port: 8088 in the runner config. All three are role defaults (gitea_actions_cache_*), and gitea_actions_cache_enabled: false turns it all off again.
  • Persistent storage: the role creates a PVC gitea-actions-cache (20Gi, default storage class) before the Helm release. It's mounted at /cache in the runner container, so cached dependencies survive a pod restart, unlike DinD's Docker data. The data-runner PVC isn't resized, because a StatefulSet's volume templates can't be changed in place.
  • Size: NFS doesn't enforce PVC sizes. What keeps it bounded is the runner's own expiry. I checked it in the runner v2.0.1 source (act/artifactcache/handler.go): entries unused for 7 days are removed, and so is any entry older than 30 days.
  • How jobs reach it: the cache server listens in the runner container. DinD shares the pod's network, so job containers on DinD's per-job networks reach it at the pod's own IP. Each job gets its own token, and entries are scoped per repository.
  • The dedicated-host runner template from I3 (#45) gets the same config, with the cache on a bind mount under /opt/gitea-runner/cache.
  • Docs: a cache section and a troubleshooting entry in docs/ci-cd/runner.md, and how a workflow uses actions/cache (pinned by SHA) in docs/ci-cd/adding-ci.md. While I was there I also corrected the runbook's DinD CPU limit to the 750m set on 2026-11-16.

Tested on the workstation

  • Simulated pod: a throwaway docker:29.5.2-dind with docker.gitea.com/runner:2.0.1 sharing its network namespace, which is the same arrangement as the pod. I ran a workflow through gitea-runner exec with job image node:24-bookworm-slim. The cache server picked the shared interface's address automatically, and the job container reached it from its own per-job network.

    Run actions/cache Result Wall time
    1 v4.3.0 miss, npm install of 3 packages (69 MB, 115 modules), saved about 11 MB 12 s
    2 (new runner container, same cache volume) v4.3.0 hit, install skipped 4 s
    3 v5.1.0 hit on the same entry 4 s

    Run 2 used a fresh runner container, so a cache on a volume does survive a runner restart. Both v4 and v5 use the runner's v1 cache API. The workstation's network is fast, so on the cluster the saving should be larger. Real numbers come after deploy.

  • I rendered the role's Helm values through Ansible (ansible-core 2.21.6) with the cache on and off, then ran helm template on chart 0.1.2. The runner container gets the /cache mount from the PVC, and the ConfigMap has the cache settings. With the cache off, the mount and volume disappear. kubeconform -strict passes the PVC and ConfigMap. Its only finding is the chart's missing serviceName, which was already there (#38).

  • ansible-lint 26.11.0 passes against the role's baseline. check-playbooks.sh syntax passes for gitea.yaml and ci-runner.yaml. --list-tasks --tags actions shows the new "create the actions cache volume" task before the Helm release. The rendered host compose.yaml passes docker compose config.

  • Pitfall found: gitea-runner exec sets an empty server URL unless you pass --gitea-instance, and actions/cache then fails with "Invalid URL". That doesn't affect the real runner, which always knows its Gitea URL.

Not tested: a cache hit on the live runner.

Deploy: the same command as #36:

ansible-playbook -i inventories/eom.dev.yml gitea.yaml --tags actions

Add your usual vault option.

  • Effect: creates the gitea-actions-cache PVC, then the runner pod is recreated with the cache on, along with S3 to S5.
  • Risk: the restart kills running jobs (see #36). A cache problem can't fail a job: actions/cache only warns and carries on.
  • Rollback: set gitea_actions_cache_enabled: false, or git revert 78ae47b, then rerun the same command. The PVC stays until you delete it: kubectl -n gitea delete pvc gitea-actions-cache.

To close after deploy: run the cache workflow twice from a cicd-test-cache repo on the live runner, and record the miss and hit times here.

— eom-cicd-roadmap

**Committed. It can be deployed in the same runner restart as S3 to S5 (#36 to #38).** DevOps/ansible-role-gitea@96e5b92, bumped in 78ae47b. **What changed** - `cache.enabled: true`, `dir: /cache`, `port: 8088` in the runner config. All three are role defaults (`gitea_actions_cache_*`), and `gitea_actions_cache_enabled: false` turns it all off again. - **Persistent storage:** the role creates a PVC `gitea-actions-cache` (20Gi, default storage class) before the Helm release. It's mounted at `/cache` in the `runner` container, so cached dependencies survive a pod restart, unlike DinD's Docker data. The `data-runner` PVC isn't resized, because a StatefulSet's volume templates can't be changed in place. - **Size:** NFS doesn't enforce PVC sizes. What keeps it bounded is the runner's own expiry. I checked it in the runner v2.0.1 source (`act/artifactcache/handler.go`): entries unused for 7 days are removed, and so is any entry older than 30 days. - **How jobs reach it:** the cache server listens in the runner container. DinD shares the pod's network, so job containers on DinD's per-job networks reach it at the pod's own IP. Each job gets its own token, and entries are scoped per repository. - The dedicated-host runner template from I3 (#45) gets the same config, with the cache on a bind mount under `/opt/gitea-runner/cache`. - Docs: a cache section and a troubleshooting entry in `docs/ci-cd/runner.md`, and how a workflow uses `actions/cache` (pinned by SHA) in `docs/ci-cd/adding-ci.md`. While I was there I also corrected the runbook's DinD CPU limit to the 750m set on 2026-11-16. **Tested on the workstation** - **Simulated pod:** a throwaway `docker:29.5.2-dind` with `docker.gitea.com/runner:2.0.1` sharing its network namespace, which is the same arrangement as the pod. I ran a workflow through `gitea-runner exec` with job image `node:24-bookworm-slim`. The cache server picked the shared interface's address automatically, and the job container reached it from its own per-job network. | Run | `actions/cache` | Result | Wall time | |---|---|---|---| | 1 | v4.3.0 | miss, `npm install` of 3 packages (69 MB, 115 modules), saved about 11 MB | 12 s | | 2 (new runner container, same cache volume) | v4.3.0 | **hit**, install skipped | 4 s | | 3 | v5.1.0 | **hit** on the same entry | 4 s | Run 2 used a fresh runner container, so a cache on a volume does survive a runner restart. Both v4 and v5 use the runner's v1 cache API. The workstation's network is fast, so on the cluster the saving should be larger. Real numbers come after deploy. - I rendered the role's Helm values through Ansible (ansible-core 2.21.6) with the cache on and off, then ran `helm template` on chart 0.1.2. The `runner` container gets the `/cache` mount from the PVC, and the ConfigMap has the cache settings. With the cache off, the mount and volume disappear. `kubeconform -strict` passes the PVC and ConfigMap. Its only finding is the chart's missing `serviceName`, which was already there (#38). - `ansible-lint` 26.11.0 passes against the role's baseline. `check-playbooks.sh syntax` passes for `gitea.yaml` and `ci-runner.yaml`. `--list-tasks --tags actions` shows the new "create the actions cache volume" task before the Helm release. The rendered host `compose.yaml` passes `docker compose config`. - Pitfall found: `gitea-runner exec` sets an empty server URL unless you pass `--gitea-instance`, and `actions/cache` then fails with "Invalid URL". That doesn't affect the real runner, which always knows its Gitea URL. **Not tested:** a cache hit on the live runner. **Deploy:** the same command as #36: ``` ansible-playbook -i inventories/eom.dev.yml gitea.yaml --tags actions ``` Add your usual vault option. - **Effect:** creates the `gitea-actions-cache` PVC, then the runner pod is recreated with the cache on, along with S3 to S5. - **Risk:** the restart kills running jobs (see #36). A cache problem can't fail a job: `actions/cache` only warns and carries on. - **Rollback:** set `gitea_actions_cache_enabled: false`, or `git revert 78ae47b`, then rerun the same command. The PVC stays until you delete it: `kubectl -n gitea delete pvc gitea-actions-cache`. **To close after deploy:** run the cache workflow twice from a `cicd-test-cache` repo on the live runner, and record the miss and hit times here. — eom-cicd-roadmap
claude added the ready-to-deploy label 2026-11-17 14:24:56 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DevOps/software-infrastructure#56