G9: reusable gitleaks secret scan, config and docs
Add a reusable workflow that downloads a pinned, checksum-verified gitleaks and scans the pushed commits, the PR's commits, or the current tree. Logs show only a count, never the rule, file, line or value, because the logs are public. This repo's caller is manual-only until S8 passes. .gitleaks.toml extends the default rules and allowlists Helm checksum/* annotations. docs/ci-cd/secret-scanning.md covers usage, what to do when it fails, and the policy for history scans (done privately, not in CI). Tested locally by running the step verbatim in a container against throwaway repos: a planted fake token fails on push, new-branch push and PR; clean ranges, checksum annotations and this repo's tree pass; a bad checksum fails; the value never appears in the output. actionlint and yamllint pass. Refs DevOps/software-infrastructure#57 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,112 @@
|
||||
# Reusable secret scan (gitleaks). DevOps/software-infrastructure#57 (G9).
|
||||
# Usage and policy: docs/ci-cd/secret-scanning.md.
|
||||
#
|
||||
# What it scans:
|
||||
# push the pushed commits (before..sha); the current tree if the
|
||||
# push created the branch or "before" can't be found
|
||||
# pull_request the PR's commits (base..head)
|
||||
# anything else the current tree only (not history)
|
||||
#
|
||||
# Repos are public and so are their Actions logs, so by default the log shows
|
||||
# only how many possible secrets were found, never which file, line or rule.
|
||||
# Run gitleaks locally to see details (the command is printed on failure).
|
||||
# Full-history scans are done privately, not in CI.
|
||||
|
||||
name: reusable-secret-scan
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
gitleaks-version:
|
||||
description: gitleaks release to download. Change gitleaks-sha256 with it.
|
||||
type: string
|
||||
default: "8.30.1"
|
||||
gitleaks-sha256:
|
||||
description: SHA-256 of gitleaks_<version>_linux_x64.tar.gz from the release's checksums file.
|
||||
type: string
|
||||
default: "551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb"
|
||||
show-details:
|
||||
description: Print rule, file and line (values stay redacted). Only for private repos.
|
||||
type: boolean
|
||||
default: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
gitleaks:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Scan for secrets
|
||||
env:
|
||||
GITLEAKS_VERSION: ${{ inputs.gitleaks-version }}
|
||||
GITLEAKS_SHA256: ${{ inputs.gitleaks-sha256 }}
|
||||
SHOW_DETAILS: ${{ inputs.show-details }}
|
||||
EVENT: ${{ github.event_name }}
|
||||
SHA: ${{ github.sha }}
|
||||
BEFORE: ${{ github.event.before }}
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
tmp="${RUNNER_TEMP:-/tmp}/secret-scan"
|
||||
mkdir -p "$tmp"
|
||||
|
||||
url="https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz"
|
||||
curl -fsSL --retry 3 -o "$tmp/gitleaks.tgz" "$url"
|
||||
echo "${GITLEAKS_SHA256} $tmp/gitleaks.tgz" | sha256sum -c --quiet -
|
||||
tar -xzf "$tmp/gitleaks.tgz" -C "$tmp" gitleaks
|
||||
gl="$tmp/gitleaks"
|
||||
|
||||
is_commit() { [ -n "$1" ] && ! [[ "$1" =~ ^0+$ ]] && git cat-file -e "$1^{commit}" 2>/dev/null; }
|
||||
|
||||
scope=(dir .)
|
||||
what="current tree"
|
||||
case "$EVENT" in
|
||||
pull_request|pull_request_target)
|
||||
if is_commit "$BASE_SHA" && is_commit "$HEAD_SHA"; then
|
||||
scope=(git "--log-opts=${BASE_SHA}..${HEAD_SHA}" .)
|
||||
what="commits ${BASE_SHA:0:8}..${HEAD_SHA:0:8}"
|
||||
fi ;;
|
||||
push)
|
||||
if is_commit "$BEFORE" && is_commit "$SHA"; then
|
||||
scope=(git "--log-opts=${BEFORE}..${SHA}" .)
|
||||
what="commits ${BEFORE:0:8}..${SHA:0:8}"
|
||||
fi ;;
|
||||
esac
|
||||
|
||||
args=(--no-banner --no-color --redact=100 --exit-code=3 --log-level=error
|
||||
--report-format=json --report-path="$tmp/report.json")
|
||||
if [ "$SHOW_DETAILS" = "true" ]; then args+=(--verbose); fi
|
||||
|
||||
echo "Scanning ${what} with gitleaks ${GITLEAKS_VERSION}"
|
||||
set +e
|
||||
"$gl" "${scope[@]}" "${args[@]}" >"$tmp/out.log" 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
|
||||
case "$rc" in
|
||||
0)
|
||||
echo "No secrets found." ;;
|
||||
3)
|
||||
n=$(grep -o '"RuleID"' "$tmp/report.json" | wc -l)
|
||||
if [ "$SHOW_DETAILS" = "true" ]; then cat "$tmp/out.log"; fi
|
||||
echo "::error::Secret scan found ${n} possible secret(s) in the ${what}."
|
||||
echo "Details are not printed here because this log may be public."
|
||||
echo "To see them, run gitleaks on your checkout, for example:"
|
||||
img="ghcr.io/gitleaks/gitleaks:v${GITLEAKS_VERSION}"
|
||||
echo " docker run --rm -v \"\$PWD:/repo\" ${img} ${scope[0]} --redact -v /repo"
|
||||
echo "If it is a real secret, rotate it first: removing it from git does not make it safe."
|
||||
echo "If it is a false positive, add an allowlist entry to .gitleaks.toml"
|
||||
echo "(see docs/ci-cd/secret-scanning.md in DevOps/software-infrastructure)."
|
||||
exit 1 ;;
|
||||
*)
|
||||
cat "$tmp/out.log"
|
||||
echo "::error::gitleaks failed to run (exit ${rc})."
|
||||
exit "$rc" ;;
|
||||
esac
|
||||
@@ -0,0 +1,23 @@
|
||||
# Secret scan for this repo. DevOps/software-infrastructure#57 (G9).
|
||||
#
|
||||
# Manual-only for now: the shared runner is still being stabilised (S3-S8,
|
||||
# #36-#41). After S8 passes, replace the trigger with:
|
||||
#
|
||||
# push:
|
||||
# branches: [main]
|
||||
# pull_request:
|
||||
#
|
||||
# A manual run scans the current tree, not history.
|
||||
|
||||
name: secret-scan
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: secret-scan-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
scan:
|
||||
uses: ./.gitea/workflows/reusable-secret-scan.yaml
|
||||
@@ -0,0 +1,11 @@
|
||||
# gitleaks configuration. DevOps/software-infrastructure#57 (G9).
|
||||
# Default rules, plus allowlists for known false positives.
|
||||
# Policy and how to add an entry: docs/ci-cd/secret-scanning.md.
|
||||
|
||||
[extend]
|
||||
useDefault = true
|
||||
|
||||
[[allowlists]]
|
||||
description = "Helm checksum/* annotations are SHA-256 hashes of rendered manifests, not secrets"
|
||||
regexTarget = "line"
|
||||
regexes = ['''checksum/[A-Za-z0-9._-]+:\s*["']?[0-9a-f]{64}["']?\s*$''']
|
||||
@@ -0,0 +1,103 @@
|
||||
# Secret scanning (G9)
|
||||
|
||||
Refs: DevOps/software-infrastructure#57.
|
||||
|
||||
CI runs [gitleaks](https://github.com/gitleaks/gitleaks) to stop new secrets
|
||||
from being committed. Most eom.dev repos and their Actions logs are public, so
|
||||
the check is built never to show a secret, or where one is, in a log.
|
||||
|
||||
## Files
|
||||
|
||||
| File | What it is |
|
||||
|---|---|
|
||||
| `.gitea/workflows/reusable-secret-scan.yaml` | Reusable workflow (`workflow_call`). Downloads a pinned gitleaks release, checks its SHA-256, and scans. |
|
||||
| `.gitea/workflows/secret-scan.yaml` | This repo's caller. Manual-only until S8 (#41) passes, then `push` to `main` and `pull_request`. |
|
||||
| `.gitleaks.toml` | This repo's config: gitleaks' default rules plus allowlists for known false positives. |
|
||||
|
||||
## What gets scanned
|
||||
|
||||
| Event | Scope |
|
||||
|---|---|
|
||||
| `push` | The pushed commits (`before..sha`). If the push created the branch, or `before` isn't in the clone, the current tree. |
|
||||
| `pull_request` | The PR's commits (`base..head`). |
|
||||
| Anything else (manual runs) | The current tree only. |
|
||||
|
||||
**CI doesn't scan full history.** A history scan of a public repo would put old
|
||||
findings into a public log. History scans are done privately (see below).
|
||||
|
||||
## What the log shows
|
||||
|
||||
By default, only the number of possible secrets and the commit range or tree
|
||||
that was scanned. It never shows the rule, file, line or value. The failure
|
||||
message prints the command to run locally to see details:
|
||||
|
||||
```bash
|
||||
docker run --rm -v "$PWD:/repo" ghcr.io/gitleaks/gitleaks:v8.30.1 git --redact -v /repo
|
||||
```
|
||||
|
||||
`show-details: true` prints the rule, file and line (values stay redacted). Use
|
||||
it only in private repos.
|
||||
|
||||
## Using it from another repo
|
||||
|
||||
Add a caller workflow and, if needed, a `.gitleaks.toml` at the repo root
|
||||
(gitleaks picks it up from the scanned directory):
|
||||
|
||||
```yaml
|
||||
name: secret-scan
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
concurrency:
|
||||
group: secret-scan-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
scan:
|
||||
uses: DevOps/software-infrastructure/.gitea/workflows/reusable-secret-scan.yaml@main
|
||||
```
|
||||
|
||||
Calling it from another repo hasn't been tried on this Gitea yet. G2 (#50)
|
||||
checks how the runner resolves cross-repo `uses:` and from private repos, and
|
||||
updates this section.
|
||||
|
||||
Inputs:
|
||||
|
||||
| Input | Default | Notes |
|
||||
|---|---|---|
|
||||
| `gitleaks-version` | `8.30.1` | Change `gitleaks-sha256` with it. |
|
||||
| `gitleaks-sha256` | SHA-256 of the 8.30.1 linux x64 tarball | From the release's `checksums.txt`. Same value as `images/ci-tools/Dockerfile`. |
|
||||
| `show-details` | `false` | See above. |
|
||||
|
||||
Once the CI tools image (G1, #49) is published, the job can run in it and skip
|
||||
the download.
|
||||
|
||||
## If the check fails
|
||||
|
||||
1. Run the local command above to see what was found.
|
||||
2. **If it's a real secret, rotate it first.** Deleting it from git, or
|
||||
rewriting history, doesn't make it safe once it has been pushed to a public
|
||||
repo. Then move it to the vault or an Actions secret.
|
||||
3. **If it's a false positive**, add a narrow allowlist entry to the repo's
|
||||
`.gitleaks.toml` with a `description` saying why it isn't secret. Prefer a
|
||||
regex on the line or a path over disabling a rule. For a single known
|
||||
finding, its fingerprint can go in `.gitleaksignore`. In a public repo that
|
||||
names the file and commit, so use it only for false positives.
|
||||
|
||||
Never paste a finding into an issue, commit message, PR, or Discourse post.
|
||||
|
||||
## Known false positives
|
||||
|
||||
| Pattern | Why it isn't secret | Allowlisted in |
|
||||
|---|---|---|
|
||||
| Helm `checksum/<name>: <64 hex>` annotations | SHA-256 of a rendered manifest, used to restart pods on change | `.gitleaks.toml` here |
|
||||
| Stripe `publishable-key="pk_..."` in HTML | Publishable keys are public by design | DevOps/www, when its caller is added |
|
||||
|
||||
## History scans
|
||||
|
||||
Done privately with the same gitleaks version, all refs, and `--redact`.
|
||||
Results go only in the private assessment, never in public issues or logs.
|
||||
|
||||
| Date | Repos | Result |
|
||||
|---|---|---|
|
||||
| 2026-11-14 | ansible-role-{gitea,eom,www,dex,discourse,mcp-servers,prometheus,listmonk}, www, software-infrastructure | Done; results are in the private assessment |
|
||||
Reference in New Issue
Block a user