Security audit, 2026-11-12
First written audit for issue #11. It covers main at commit ebdbe19 (after PR #23). Two findings (F7 and F13) were fixed by PR #24 after the audit and are marked in the table. Every statement has a label: Observed (we read it), Measured (we ran it), Estimated (a calculation) or Proposed (not running yet).
Summary
- Measured:
shellcheckreports 0 findings on 8 scripts.actionlintreports 0 findings on 6 workflows andaction.yml.bats tests/passes 337 of 337 tests. - Measured: each of the 5 architecture rules fails its test when we break the rule on a scratch copy. One rule, “no write outside the workspace”, fails only for the paths its test names (F6).
- Observed: 17 findings: 0 critical, 1 high, 3 medium, 11 low, 2 info.
- Observed: three gaps matter most. They are the v2 image (F1), a token in
.git/configthat is reported only after the build (F3), and the missing release provenance (F4). - Proposed: every decision below is a proposal until the maintainer confirms it. The last section lists the 7 that need an answer.
What we looked at and the limits
| Item | Statement |
|---|---|
| Commit | Observed: ebdbe19, origin/main, the worktree was clean. Date of the audit: 2026-11-12. |
| Read | Observed: the issue, docs/threat-model.md, docs/architecture.md, docs/testing.md, SECURITY.md, action.yml, the 6 workflows, .github/dependabot.yml, tests/architecture.bats, tests/action.bats, and the test names of tests/failure-paths.bats and tests/validate-inputs.bats. |
| Read in part | Observed: the 8 scripts hold 1,810 lines. We read build.sh, the end of commit.sh, the first 110 lines of git-lib.sh and the head of install.sh. We read run_without_token in lib.sh and the registry checks of validate-inputs.sh. We searched the rest with grep. We did not review every line. |
| Run | Measured: shellcheck, actionlint, zizmor, bats, the rule-break experiments, and a demo of finding F3. Versions: shellcheck 0.11.0, actionlint 1.7.12, zizmor 1.32.0, bats 1.14.0. |
| Environment | Measured: macOS with bash 3.2.57 and Node.js 22.23.1. CI uses Node.js 24. The CI results come from gh, we did not run CI. |
| Settings | Observed: read with gh api, as the repository owner, on 2026-11-12. Read only. We changed no setting. |
| Not seen | The internals of the third-party actions (they are pinned by SHA, we did not read them). What actions/setup-node does with its default github.token input. A Trivy scan, because main builds no v2 image. Real pushes and real pull requests: CI uses a local bare remote and a stand-in gh. |
| Scratch files | The rule-break copies were in a scratch directory outside the worktree. They held fake values only. We deleted them. |
What was measured
Tool runs
| Check | Command | Result |
|---|---|---|
| Shell lint | shellcheck scripts/*.sh | Measured: 0 findings, exit 0. CI runs it on git ls-files '*.sh', which is the same 8 files. |
| Workflow lint | actionlint | Measured: 0 findings, exit 0. |
| Workflow security | zizmor --offline --min-severity medium . | Measured: 0 findings, exit 0. With the default persona the tool also reports 13 low findings, all self-repository (use $/ and not ./ for the local action). With --persona auditor it adds 1 medium, see F15. |
| Tests | bats tests/ | Measured: 337 tests, 337 pass, 0 fail, about 157 seconds. |
| Tests per file | bats --count per file | Measured: pr 60, package-managers 54, commit 44, check 39, validate-inputs 31. Then registries 27, action 16, architecture 15, aliases 14, install 13, build 10, failure-paths 8, token 6. The sum is 337. |
Jobs on main | gh api commits/ebdbe19/check-runs | Observed: 17 of 17 jobs succeeded on ebdbe19. |
Each architecture rule, broken on purpose
We copied the repository to a scratch directory, added one line that breaks the rule, and ran the matching test file. The control copy (no change) had 0 failures.
| Rule from #11 | Line we added to the scratch copy | Test that failed |
|---|---|---|
| The install and build scripts never read the token | echo "$GITHUB_TOKEN" in install.sh, ${NAB_TOKEN:-} in build.sh, $GH_TOKEN in check.sh | “install.sh and build.sh never read a token variable” (2 cases), “only the scripts on the token allowlist read a token variable” (1 case) |
| Only the commit script pushes | git push in check.sh and in build.sh, git fetch in validate-inputs.sh | “only the token scripts push to or fetch from a remote” (3 cases) |
| No network except the package manager | curl in install.sh, gh api in build.sh, wget in commit.sh, /dev/tcp in lib.sh | “no script calls a network tool, the package manager is the only way out” (4 cases) |
Every script starts with set -euo pipefail | -u removed from build.sh, the line removed from pr.sh, set +e and set +o pipefail added | “every script starts with the shebang and set -euo pipefail” (2 cases), “no script turns off errexit, nounset or pipefail” (2 cases) |
| Nothing written outside the workspace and the temporary directory | > "$HOME/.bashrc", > /etc/..., npm install -g, git config --global | “no script installs a tool for the whole machine or touches a system or home path” (4 cases) |
(extra) An input goes into a run: line | "${{ inputs.command }}" added to the build step | “no run: block of action.yml holds a ${{ }} expression” and “no run line interpolates an expression” |
| (extra) The token reaches another step | NAB_TOKEN added to the env of the Build step | “the token is only read by …” and “NAB_TOKEN is in the env of the commit step and the pr step, and of no other step” |
(extra) Project code starts outside run_without_token | bash -c "$cmd" in build.sh | “install.sh and build.sh start the project’s code only through run_without_token” |
Measured, a gap: 3 other writes passed the suite with 0 failures. They are > /home/runner/..., > /Users/... and > "${RUNNER_TEMP}/../...", each added to build.sh. See F6.
Measured, finding F3: in a scratch repository with an http.extraheader in .git/config, build.sh ran a build script that read the header key from that file. The run printed no warning and did not fail. The header value was a fake string, and we did not copy it anywhere.
Workflow rules
| Check | Result |
|---|---|
Every uses: pinned by a 40 character SHA | Measured: 24 external uses: lines, 24 pinned, 0 not pinned. 13 more lines are uses: ./, the local action. The CI check counts 37 lines. |
Every workflow has top level permissions: | Measured: 6 of 6. auto-merge-dependabot.yml sets {}, the other 5 set contents: read. |
| Job level write scopes | Observed: 3 jobs. dependabot-auto-merge has contents, pull-requests and issues write, only for Dependabot pull requests. codeql has security-events write. scorecard has security-events and id-token write. |
| Dangerous triggers | Observed: 0 uses of pull_request_target or workflow_run. |
| The pin rule fails on a break | Measured: actions/checkout@v4 in a scratch codeql.yml. The pinned-actions script exits 1 and prints the file and line. zizmor exits 14 with unpinned-uses. |
| The permissions rule fails on a break | Measured: top level permissions: removed from a scratch dependency-review.yml. zizmor reports excessive-permissions (medium) and exits 13. With permissions: contents: write at workflow level it reports high and exits 14. |
Latest result on main | Observed: CI, Workflow security, CodeQL and Scorecard all succeeded on ebdbe19 (started 2026-11-12 17:10 UTC). dependency-review runs only on pull requests, so it has no run on main. |
Repository settings (read with gh api)
| Setting | Value |
|---|---|
Branch protection on main, required checks | Observed: 17 checks: shellcheck, workflows, bats, actionlint, zizmor, pinned-actions, dependency-review, Analyze (actions), self-test (ubuntu-latest, macos-latest, windows-latest, commit, pr), self-test-check, self-test-pm (pnpm, yarn-classic, yarn-berry). |
| Other protection fields | Observed: linear history on, conversation resolution on, force push off, deletion off. Up-to-date branch required: off. Required reviews: none. enforce_admins: off. Signed commits: off. Rulesets: 0. |
| Secret scanning | Observed: enabled. Open alerts: 0. |
| Push protection | Observed: enabled. |
| Private vulnerability reporting | Observed: enabled. |
| Dependabot security updates | Observed: enabled. Open Dependabot alerts: 0. |
| Actions permissions | Observed: the default workflow token is read. Workflows cannot approve pull requests. All actions are allowed. “Require actions to be pinned to a full-length commit SHA” is off. |
| Secrets and environments | Observed: 0 repository secrets, 0 environments, 1 collaborator. |
| Tags and releases | Observed: 7 tags (v2 and v2.0.0 to v2.1.2), 6 releases, no v3 tag. |
Code scanning alerts
| # | Rule | State | Reason, as written on the alert |
|---|---|---|---|
| 5 | CIIBestPracticesID (Scorecard) | open, low | None. It needs registration at bestpractices.dev. |
| 4 | CodeReviewID | dismissed, won’t fix, 2026-11-12 | One maintainer, so nobody else can approve. Required checks and the protection of main stay on. |
| 3 | FuzzingID | dismissed, won’t fix, 2026-11-12 | Shell action with no parser or binary input. The inputs are tested with hostile values. |
| 2 | PinnedDependenciesID | fixed, 2026-11-12 | Fixed by PR #23. |
| 1 | BranchProtectionID | fixed, 2026-11-12 | The protection rules on main. |
Observed: the two dismissal reasons match docs/threat-model.md. The public Scorecard API shows a score of 8 on 2026-11-12. Scores of 0: Code-Review, CII-Best-Practices, Fuzzing. Branch-Protection, Packaging and Signed-Releases return -1 (not evaluated).
Checklist of issue #11
| Item | Status | Evidence |
|---|---|---|
| Threat model | Done | docs/threat-model.md, PR #20. See F12. |
SECURITY.md | Done | SECURITY.md, PR #20. The sunset date is TBD, see F2. |
| Review of the v2 findings | Done | “Known v2 findings and the v3 fix” in docs/threat-model.md, PR #20. |
Every uses: pinned, least-privilege permissions:, a check fails on a break | Done | 24 of 24 pinned, 6 of 6 with permissions:. Both rules failed on purpose on a scratch copy. The checks are the pinned-actions and zizmor jobs, PR #20. |
actionlint and zizmor in CI | Done | .github/workflows/workflow-security.yml, PR #20. Both are required checks. |
| Scorecard, result tracked, findings fixed | Done | .github/workflows/scorecard.yml, PRs #20 and #23. 1 open alert (F5), 2 dismissed, 2 fixed. |
| Secret scanning, push protection, private reporting | Done | Repository settings, read with gh api. No pull request. |
| v2 Docker image: digest, Trivy, sunset date | Not done | Dockerfile is gone from main (removed in PR #16). See F1 and F2. |
| Dependabot covers the actions, merge time for security updates | Partly | .github/dependabot.yml covers github-actions and npm, PR #13. No merge time is written anywhere, see F10. |
| v3 release provenance | Not done | Belongs to #9. See F4. |
Scripts split by responsibility, docs/architecture.md | Done | 8 scripts, PRs #16 to #22. docs/architecture.md has a table row for each, PR #22. |
| Architecture tests, each seen failing | Partly | 15 tests in tests/architecture.bats, PR #22. All 5 rules failed on purpose. The write rule covers fewer paths than the issue says, see F6. |
| Input validation tests | Done | tests/validate-inputs.bats (31 tests) and tests/failure-paths.bats, PRs #16 and #22. We read the test names and ran them. We did not break each one. |
| Failure-path tests | Done | Detached HEAD: commit.bats:239, pr.bats:545. Fork pull request: failure-paths.bats:146, :166. Protected branch: commit.bats:547. Failing build: build.bats:90, failure-paths.bats:190. Nothing changed and second run: commit.bats:154, pr.bats:287. Two runs at once: failure-paths.bats:208. |
| Written audit | Done by this file | docs/audits/2026-11-12.md. It was not merged when we wrote it. |
| Quarterly audit, as a recurring issue | Not done | See F11. |
Findings
Severity: critical, high, medium, low, info. The owner “Maintainer” is the repository owner. A decision marked “accepted” is a proposal until the maintainer confirms it. Decisions marked “Decided” or “Fixed” were made on 2026-11-12, after the measurements.
| # | Gap | Evidence | Severity | Owner | Decision |
|---|---|---|---|---|---|
| F1 | The v2 image is not pinned, not scanned and not patched. SECURITY.md promises security fixes for v2. main has no Dockerfile, and the repository has no v2 branch. | Observed: git show v2:Dockerfile has FROM node:20-alpine and apk add with no versions. Node.js 20 ended on 2026-04-30 (schedule.json of nodejs/Release). main has no Dockerfile (removed in PR #16). Branches: main and docs/7-v3-readme. dependabot.yml has no docker entry. | high | Maintainer | Decided 2026-11-12: v2 support ends when v3.0.0 is published. SECURITY.md says so. No v2.1.3 and no maintenance branch. |
| F2 | The v2 sunset date is still TBD. | Observed: SECURITY.md:33, “Sunset: TBD, set before the v3.0.0 release”. | medium | Maintainer | Fixed: the sunset is the day v3.0.0 is published, written in SECURITY.md. |
| F3 | A token kept in .git/config is reported after the build, not before. If actions/checkout keeps its credentials, install and build code can read the token. | Observed: the check is nab_warn_persisted_credentials at git-lib.sh:98. Only commit.sh:119 and pr.sh:351 call it, after the build. Measured: see the demo above. | medium | Maintainer | Fixed after the audit in the pull request of issue #26 (decided 2026-11-12). Check in validate-inputs.sh, before install. Fail in modes commit and pr, warn in the other two. |
| F4 | No provenance for the v3 release. | Observed: 0 v3 tags. Scorecard Signed-Releases returns -1. grep for “attest”, “provenance” and “slsa” in the workflows finds 0 hits. | medium | Maintainer | Follow-up in #9. Nothing is released from #11. |
| F5 | Scorecard CII-Best-Practices is open. | Observed: code scanning alert #5, score 0. | low | Maintainer | Blocked on Maintainer: register at bestpractices.dev and answer the questionnaire. |
| F6 | The “writes outside the workspace” test is narrower than the rule in #11. | Measured: writes to /home/runner, /Users and ${RUNNER_TEMP}/.. pass the suite. The test checks $HOME, ~/, /etc, /usr, /var, /opt, /root and global installs. docs/architecture.md states the narrower rule, so the doc is accurate. | low | Maintainer | Follow-up issue. Proposed: a positive list of the redirect targets that a script may write to. |
| F7 | docs-examples is not a required check. The open PR #24 adds it. | Observed: the 17 required checks do not include it. PR #24 adds the job and has mergeable_state: clean. | low | Maintainer | Fixed after the audit: PR #24 merged at b506d69, and docs-examples is now the 18th required check (read with gh api on 2026-11-12). |
| F8 | Branch protection does not require an up-to-date branch and does not apply to admins. | Observed: strict: false, enforce_admins: false, no required reviews. | low | Maintainer | Accepted. One maintainer, no second reviewer to protect. Review it if a second maintainer joins. |
| F9 | The Actions setting that requires SHA pins is off. The CI check catches an unpinned action only in a pull request. | Observed: sha_pinning_required: false, allowed_actions: all. | low | Maintainer | Fixed 2026-11-12: sha_pinning_required is now true (read back with gh api). |
| F10 | No stated time to merge a security update. | Observed: grep finds no merge time in SECURITY.md or the docs. The auto-merge job holds a patch update for 3 days and a minor update for 7 (auto-merge-dependabot.yml:85), for npm packages only. | low | Maintainer | Fixed: SECURITY.md states 7 days for high or critical and 30 days for the others. |
| F11 | The quarterly audit is not scheduled. | Observed: the open issues are #7, #8, #9, #10, #11 and #12. None is a recurring audit. | low | Maintainer | Fixed: issue #27, with the text below. |
| F12 | The threat model describes v2 as “today” and v3 only as a design. The accepted-risk column is not confirmed. It has no threat for a persisted checkout token. | Observed: docs/threat-model.md:3 and :7. The v3 controls now exist and are tested: run_without_token, tests/token.bats, tests/architecture.bats. | low | Maintainer | Follow-up. Proposed: add a “v3 status” column that cites the test for each threat, and add the F3 threat. |
| F13 | docs/README.md says the action “Runs in isolated Docker container”. main is a composite action. | Observed: docs/README.md:112. The docs/README.md of PR #24 does not have that line. | low | Maintainer | Fixed after the audit by PR #24 (b506d69). No new issue. |
| F14 | The check tools are pinned by hand and nothing updates them. | Observed: workflow-security.yml:34 (actionlint 1.7.12) and :70 (zizmor 1.32.0). The actionlint checksum comes from the same release, so it checks the download, not the source. bats and shellcheck come from the runner image. | low | Maintainer | Accepted. Re-check the versions in each audit. |
| F15 | A secret would enter every step of one job. AUTO_APPROVE_TOKEN is in the job level env:. | Observed: auto-merge-dependabot.yml:57. zizmor --persona auditor reports it (secrets-outside-env, medium). The secret does not exist (0 repository secrets), so the approve step is skipped. | low | Maintainer | Accepted while the secret is absent. If it is added, move it to an environment or to one step. |
| F16 | The Code-Review and Fuzzing alerts are accepted. | Observed: alerts #4 and #3 are dismissed with the reasons in docs/threat-model.md. | info | Maintainer | Accepted. The reasons are documented. |
| F17 | Scorecard cannot read the branch protection. It returns -1, so it neither passes nor fails the check. | Observed: the Scorecard API, check Branch-Protection. This audit read the rules with gh api instead. | info | Maintainer | Accepted. Read the rules by hand in each audit. |
Proposed text for the recurring issue (F11)
- Title:
Quarterly security audit. Label:security. - Next dates: 2027-02-12, and before the v3.0.0 release.
- Checklist: repeat the sections “What was measured” and “Checklist of issue #11” of this file. Write the result as
docs/audits/YYYY-MM-DD.md. Close the issue and open the next one. - Why manual: a scheduled workflow that opens issues needs
issues: write. That is one more write scope, and the action is small enough to track by hand.
Residual risk accepted
Proposed, until the maintainer confirms:
- One maintainer. Nobody else reads a change before it merges (F8, F16). The required checks and the 337 tests reduce the risk. A second maintainer would close it.
- Same-repository pull requests run project code with the workflow token in v2. v3 removes the token from install and build. The exception is a checkout that keeps its credentials (F3).
- Third-party actions and the runner image. They are pinned by SHA or by the runner version, and we did not read their code.
- Text-based tests. The architecture tests search for names. They cannot see
${!name}or a command built from pieces. This is stated indocs/architecture.md. @v2is a moving tag. Users who follow it get any change that moves it (threat T4).
Repeat before each major release
Run this audit again before v3.0.0, and before each later major release.
- Run
shellcheck,actionlint,zizmorandbats tests/. Record the counts. - Break each architecture rule on a scratch copy and record the failing test. Add one new break for each rule that changed.
- Read the branch protection, the security settings and the code scanning alerts with
gh api. Record the numbers. - Check that every
uses:is a 40 character SHA and every workflow haspermissions:. - Check that every checkbox in the release issue has a file or a command as evidence.
- Run the F3 demo again, and read the code of
git-lib.sh,commit.shandpr.shline by line.
Decisions for the maintainer
Decisions 1, 2, 3, 6 and 7 were answered on 2026-11-12, and the findings table says how. Decision 4 is done for F9, and F5 still needs the registration at bestpractices.dev. Decision 5 (accepted risks) is not confirmed yet.
- v2 image (F1). Recommended: cut a
v2maintenance branch at tagv2.1.2. Move the base image to a current LTS version pinned by digest, add Trivy there, and release v2.1.3. One release reaches every@v2user, because the tag moves. The cost is one branch to keep until the sunset. The other option is to end v2 support at the v3.0.0 release and changeSECURITY.mdto say that v2 gets no patches. Both options need a release or a policy change, which #11 may not make. - Sunset date (F2). Pick a date, for example a fixed number of days after v3.0.0, and write it in
SECURITY.md. - Persisted credentials (F3). Approve the follow-up: check before install, fail in modes commit and pr.
- Settings (F5, F9). Register at bestpractices.dev. Turn on “Require actions to be pinned to a full-length commit SHA”.
- Accepted risks (F8, F14, F15, F16, F17). Confirm them, or ask for a change.
- Security update merge time (F10). Confirm 7 and 30 days, or give other numbers.
- Recurring audit (F11). Approve the issue text above, and say who creates it.