G9: reusable gitleaks secret scan, config and docs
Add a reusable workflow that downloads a pinned, checksum-verified gitleaks and scans the pushed commits, the PR's commits, or the current tree. Logs show only a count, never the rule, file, line or value, because the logs are public. This repo's caller is manual-only until S8 passes. .gitleaks.toml extends the default rules and allowlists Helm checksum/* annotations. docs/ci-cd/secret-scanning.md covers usage, what to do when it fails, and the policy for history scans (done privately, not in CI). Tested locally by running the step verbatim in a container against throwaway repos: a planted fake token fails on push, new-branch push and PR; clean ranges, checksum annotations and this repo's tree pass; a bad checksum fails; the value never appears in the output. actionlint and yamllint pass. Refs DevOps/software-infrastructure#57 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,112 @@
|
|||||||
|
# Reusable secret scan (gitleaks). DevOps/software-infrastructure#57 (G9).
|
||||||
|
# Usage and policy: docs/ci-cd/secret-scanning.md.
|
||||||
|
#
|
||||||
|
# What it scans:
|
||||||
|
# push the pushed commits (before..sha); the current tree if the
|
||||||
|
# push created the branch or "before" can't be found
|
||||||
|
# pull_request the PR's commits (base..head)
|
||||||
|
# anything else the current tree only (not history)
|
||||||
|
#
|
||||||
|
# Repos are public and so are their Actions logs, so by default the log shows
|
||||||
|
# only how many possible secrets were found, never which file, line or rule.
|
||||||
|
# Run gitleaks locally to see details (the command is printed on failure).
|
||||||
|
# Full-history scans are done privately, not in CI.
|
||||||
|
|
||||||
|
name: reusable-secret-scan
|
||||||
|
|
||||||
|
on:
|
||||||
|
workflow_call:
|
||||||
|
inputs:
|
||||||
|
gitleaks-version:
|
||||||
|
description: gitleaks release to download. Change gitleaks-sha256 with it.
|
||||||
|
type: string
|
||||||
|
default: "8.30.1"
|
||||||
|
gitleaks-sha256:
|
||||||
|
description: SHA-256 of gitleaks_<version>_linux_x64.tar.gz from the release's checksums file.
|
||||||
|
type: string
|
||||||
|
default: "551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb"
|
||||||
|
show-details:
|
||||||
|
description: Print rule, file and line (values stay redacted). Only for private repos.
|
||||||
|
type: boolean
|
||||||
|
default: false
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
gitleaks:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 10
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
|
||||||
|
- name: Scan for secrets
|
||||||
|
env:
|
||||||
|
GITLEAKS_VERSION: ${{ inputs.gitleaks-version }}
|
||||||
|
GITLEAKS_SHA256: ${{ inputs.gitleaks-sha256 }}
|
||||||
|
SHOW_DETAILS: ${{ inputs.show-details }}
|
||||||
|
EVENT: ${{ github.event_name }}
|
||||||
|
SHA: ${{ github.sha }}
|
||||||
|
BEFORE: ${{ github.event.before }}
|
||||||
|
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||||
|
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
tmp="${RUNNER_TEMP:-/tmp}/secret-scan"
|
||||||
|
mkdir -p "$tmp"
|
||||||
|
|
||||||
|
url="https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz"
|
||||||
|
curl -fsSL --retry 3 -o "$tmp/gitleaks.tgz" "$url"
|
||||||
|
echo "${GITLEAKS_SHA256} $tmp/gitleaks.tgz" | sha256sum -c --quiet -
|
||||||
|
tar -xzf "$tmp/gitleaks.tgz" -C "$tmp" gitleaks
|
||||||
|
gl="$tmp/gitleaks"
|
||||||
|
|
||||||
|
is_commit() { [ -n "$1" ] && ! [[ "$1" =~ ^0+$ ]] && git cat-file -e "$1^{commit}" 2>/dev/null; }
|
||||||
|
|
||||||
|
scope=(dir .)
|
||||||
|
what="current tree"
|
||||||
|
case "$EVENT" in
|
||||||
|
pull_request|pull_request_target)
|
||||||
|
if is_commit "$BASE_SHA" && is_commit "$HEAD_SHA"; then
|
||||||
|
scope=(git "--log-opts=${BASE_SHA}..${HEAD_SHA}" .)
|
||||||
|
what="commits ${BASE_SHA:0:8}..${HEAD_SHA:0:8}"
|
||||||
|
fi ;;
|
||||||
|
push)
|
||||||
|
if is_commit "$BEFORE" && is_commit "$SHA"; then
|
||||||
|
scope=(git "--log-opts=${BEFORE}..${SHA}" .)
|
||||||
|
what="commits ${BEFORE:0:8}..${SHA:0:8}"
|
||||||
|
fi ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
args=(--no-banner --no-color --redact=100 --exit-code=3 --log-level=error
|
||||||
|
--report-format=json --report-path="$tmp/report.json")
|
||||||
|
if [ "$SHOW_DETAILS" = "true" ]; then args+=(--verbose); fi
|
||||||
|
|
||||||
|
echo "Scanning ${what} with gitleaks ${GITLEAKS_VERSION}"
|
||||||
|
set +e
|
||||||
|
"$gl" "${scope[@]}" "${args[@]}" >"$tmp/out.log" 2>&1
|
||||||
|
rc=$?
|
||||||
|
set -e
|
||||||
|
|
||||||
|
case "$rc" in
|
||||||
|
0)
|
||||||
|
echo "No secrets found." ;;
|
||||||
|
3)
|
||||||
|
n=$(grep -o '"RuleID"' "$tmp/report.json" | wc -l)
|
||||||
|
if [ "$SHOW_DETAILS" = "true" ]; then cat "$tmp/out.log"; fi
|
||||||
|
echo "::error::Secret scan found ${n} possible secret(s) in the ${what}."
|
||||||
|
echo "Details are not printed here because this log may be public."
|
||||||
|
echo "To see them, run gitleaks on your checkout, for example:"
|
||||||
|
img="ghcr.io/gitleaks/gitleaks:v${GITLEAKS_VERSION}"
|
||||||
|
echo " docker run --rm -v \"\$PWD:/repo\" ${img} ${scope[0]} --redact -v /repo"
|
||||||
|
echo "If it is a real secret, rotate it first: removing it from git does not make it safe."
|
||||||
|
echo "If it is a false positive, add an allowlist entry to .gitleaks.toml"
|
||||||
|
echo "(see docs/ci-cd/secret-scanning.md in DevOps/software-infrastructure)."
|
||||||
|
exit 1 ;;
|
||||||
|
*)
|
||||||
|
cat "$tmp/out.log"
|
||||||
|
echo "::error::gitleaks failed to run (exit ${rc})."
|
||||||
|
exit "$rc" ;;
|
||||||
|
esac
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
# Secret scan for this repo. DevOps/software-infrastructure#57 (G9).
|
||||||
|
#
|
||||||
|
# Manual-only for now: the shared runner is still being stabilised (S3-S8,
|
||||||
|
# #36-#41). After S8 passes, replace the trigger with:
|
||||||
|
#
|
||||||
|
# push:
|
||||||
|
# branches: [main]
|
||||||
|
# pull_request:
|
||||||
|
#
|
||||||
|
# A manual run scans the current tree, not history.
|
||||||
|
|
||||||
|
name: secret-scan
|
||||||
|
|
||||||
|
on:
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: secret-scan-${{ github.ref }}
|
||||||
|
cancel-in-progress: true
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
scan:
|
||||||
|
uses: ./.gitea/workflows/reusable-secret-scan.yaml
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
# gitleaks configuration. DevOps/software-infrastructure#57 (G9).
|
||||||
|
# Default rules, plus allowlists for known false positives.
|
||||||
|
# Policy and how to add an entry: docs/ci-cd/secret-scanning.md.
|
||||||
|
|
||||||
|
[extend]
|
||||||
|
useDefault = true
|
||||||
|
|
||||||
|
[[allowlists]]
|
||||||
|
description = "Helm checksum/* annotations are SHA-256 hashes of rendered manifests, not secrets"
|
||||||
|
regexTarget = "line"
|
||||||
|
regexes = ['''checksum/[A-Za-z0-9._-]+:\s*["']?[0-9a-f]{64}["']?\s*$''']
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
# Secret scanning (G9)
|
||||||
|
|
||||||
|
Refs: DevOps/software-infrastructure#57.
|
||||||
|
|
||||||
|
CI runs [gitleaks](https://github.com/gitleaks/gitleaks) to stop new secrets
|
||||||
|
from being committed. Most eom.dev repos and their Actions logs are public, so
|
||||||
|
the check is built never to show a secret, or where one is, in a log.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
| File | What it is |
|
||||||
|
|---|---|
|
||||||
|
| `.gitea/workflows/reusable-secret-scan.yaml` | Reusable workflow (`workflow_call`). Downloads a pinned gitleaks release, checks its SHA-256, and scans. |
|
||||||
|
| `.gitea/workflows/secret-scan.yaml` | This repo's caller. Manual-only until S8 (#41) passes, then `push` to `main` and `pull_request`. |
|
||||||
|
| `.gitleaks.toml` | This repo's config: gitleaks' default rules plus allowlists for known false positives. |
|
||||||
|
|
||||||
|
## What gets scanned
|
||||||
|
|
||||||
|
| Event | Scope |
|
||||||
|
|---|---|
|
||||||
|
| `push` | The pushed commits (`before..sha`). If the push created the branch, or `before` isn't in the clone, the current tree. |
|
||||||
|
| `pull_request` | The PR's commits (`base..head`). |
|
||||||
|
| Anything else (manual runs) | The current tree only. |
|
||||||
|
|
||||||
|
**CI doesn't scan full history.** A history scan of a public repo would put old
|
||||||
|
findings into a public log. History scans are done privately (see below).
|
||||||
|
|
||||||
|
## What the log shows
|
||||||
|
|
||||||
|
By default, only the number of possible secrets and the commit range or tree
|
||||||
|
that was scanned. It never shows the rule, file, line or value. The failure
|
||||||
|
message prints the command to run locally to see details:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run --rm -v "$PWD:/repo" ghcr.io/gitleaks/gitleaks:v8.30.1 git --redact -v /repo
|
||||||
|
```
|
||||||
|
|
||||||
|
`show-details: true` prints the rule, file and line (values stay redacted). Use
|
||||||
|
it only in private repos.
|
||||||
|
|
||||||
|
## Using it from another repo
|
||||||
|
|
||||||
|
Add a caller workflow and, if needed, a `.gitleaks.toml` at the repo root
|
||||||
|
(gitleaks picks it up from the scanned directory):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
name: secret-scan
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
pull_request:
|
||||||
|
concurrency:
|
||||||
|
group: secret-scan-${{ github.ref }}
|
||||||
|
cancel-in-progress: true
|
||||||
|
jobs:
|
||||||
|
scan:
|
||||||
|
uses: DevOps/software-infrastructure/.gitea/workflows/reusable-secret-scan.yaml@main
|
||||||
|
```
|
||||||
|
|
||||||
|
Calling it from another repo hasn't been tried on this Gitea yet. G2 (#50)
|
||||||
|
checks how the runner resolves cross-repo `uses:` and from private repos, and
|
||||||
|
updates this section.
|
||||||
|
|
||||||
|
Inputs:
|
||||||
|
|
||||||
|
| Input | Default | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `gitleaks-version` | `8.30.1` | Change `gitleaks-sha256` with it. |
|
||||||
|
| `gitleaks-sha256` | SHA-256 of the 8.30.1 linux x64 tarball | From the release's `checksums.txt`. Same value as `images/ci-tools/Dockerfile`. |
|
||||||
|
| `show-details` | `false` | See above. |
|
||||||
|
|
||||||
|
Once the CI tools image (G1, #49) is published, the job can run in it and skip
|
||||||
|
the download.
|
||||||
|
|
||||||
|
## If the check fails
|
||||||
|
|
||||||
|
1. Run the local command above to see what was found.
|
||||||
|
2. **If it's a real secret, rotate it first.** Deleting it from git, or
|
||||||
|
rewriting history, doesn't make it safe once it has been pushed to a public
|
||||||
|
repo. Then move it to the vault or an Actions secret.
|
||||||
|
3. **If it's a false positive**, add a narrow allowlist entry to the repo's
|
||||||
|
`.gitleaks.toml` with a `description` saying why it isn't secret. Prefer a
|
||||||
|
regex on the line or a path over disabling a rule. For a single known
|
||||||
|
finding, its fingerprint can go in `.gitleaksignore`. In a public repo that
|
||||||
|
names the file and commit, so use it only for false positives.
|
||||||
|
|
||||||
|
Never paste a finding into an issue, commit message, PR, or Discourse post.
|
||||||
|
|
||||||
|
## Known false positives
|
||||||
|
|
||||||
|
| Pattern | Why it isn't secret | Allowlisted in |
|
||||||
|
|---|---|---|
|
||||||
|
| Helm `checksum/<name>: <64 hex>` annotations | SHA-256 of a rendered manifest, used to restart pods on change | `.gitleaks.toml` here |
|
||||||
|
| Stripe `publishable-key="pk_..."` in HTML | Publishable keys are public by design | DevOps/www, when its caller is added |
|
||||||
|
|
||||||
|
## History scans
|
||||||
|
|
||||||
|
Done privately with the same gitleaks version, all refs, and `--redact`.
|
||||||
|
Results go only in the private assessment, never in public issues or logs.
|
||||||
|
|
||||||
|
| Date | Repos | Result |
|
||||||
|
|---|---|---|
|
||||||
|
| 2026-11-14 | ansible-role-{gitea,eom,www,dex,discourse,mcp-servers,prometheus,listmonk}, www, software-infrastructure | Done; results are in the private assessment |
|
||||||
Reference in New Issue
Block a user