feat(scripts): one-shot release backfill to the public mirror (all tags, newest 10 binaries)

[LukasWodka]

Summary

One-shot, idempotent backfill of this repository's historical releases onto the public deliverable mirror. mirror-publish.yml publishes each new release going forward; releases that predate the mirror are carried over once by scripts/backfill-releases.sh, run by a human.

The decision it implements (stated in the script header and scripts/RELEASE_CHECKLIST.md §8):

• every published release gets its tag, its GitHub release and its text assets (install.sh, install.ps1, SHA256SUMS, anything SHA256SUMS does not list);
• binaries (the files SHA256SUMS lists) and their .sig/.cert companions only for the newest BINARY_KEEP releases (default 10) — older pinned binary URLs 404 on the mirror; the documented answer is "re-run the installer";
• prereleases and drafts skipped unless --include-prerelease.

Tag anchoring. The mirror holds a README, so no tag can point at the commit a release was built from. Each mirror tag is an annotated tag on the mirror's default-branch head; the annotation carries the original date (the source tag's tagger date when annotated, else the release's created_at) and the original message, and states that it is a release marker, not a source snapshot. A tag already on the mirror is accepted only if it points at a commit the mirror has; a dangling tag is refused, never repointed. The release body gets a footer with the original publish date (GitHub does not let a created release carry a past date).

Reuse, not copies. publish-mirror.sh target decides the mirror name (unset / malformed / equal to source refused by the same rule the workflow applies). publish-guard.sh scans every text asset and every release body against .publish-forbidden (+ BACKFILL_EXTRA_FORBIDDEN private needles, + gitleaks) before any write; a hit refuses that release naming the tier. Binaries are opaque to the string scan by design and are instead verified against the source release's SHA256SUMS before upload — a mismatch refuses, naming the asset.

Fail closed. Any read that does not complete is exit 2 ("could not tell", naming the failing call) — never "treated as absent". A refused release is reported in the per-release table and the run continues; exit 1 at the end if anything was refused. Anything already on the mirror with an identical SHA256 is skipped, so a re-run over a complete mirror makes zero writes. --from-tag / --only-tag resume partial runs without changing the binary decision (computed over the full filtered list). Dry-run is the default; --apply writes.

Type

• Feature (tooling)
• Bug fix
• Refactor
• Docs

Test plan

scripts/tests/backfill-releases-verify.sh — offline; gh is a recording fake serving fixtures (12 stable releases, 1 prerelease, an annotated source tag) and mutating a JSON mirror state on every write. 30 assertions, covering everything in the ask:

• dry-run makes zero writes; --apply makes exactly 48 writes for 12 releases (tag object, ref, release, one upload each), oldest first, newest stable last and the only make_latest=true;
• a second --apply writes nothing and downloads nothing it can compare by digest;
• BINARY_KEEP boundary: the 10th newest carries binaries + sig/cert, the 11th exactly the three text assets; --include-prerelease shifts the boundary;
• SHA mismatch vs SHA256SUMS refuses naming the asset, nothing of that release is written, the other 11 proceed, exit 1;
• MIRROR_REPO unset / equal to source (case-insensitive) refused by publish-mirror's rule before any gh call;
• prerelease excluded by default; source repo derived from gh repo view, no repository name hardcoded;
• failing reads (source list, mirror list, an asset download) exit 2 naming the call; mid-run failure then resume completes the mirror; empty mirror refused with instructions;
• refuse-tier needle in a body refuses naming the tier and the notes file (text not echoed); --notes fixed alternative; --strict promotes the report tier;
• present-with-different-digest and a dangling mirror tag each refused by name; --only-tag / --from-tag.

--mutations mode: copies the script, breaks 6 anchored rules one at a time (releases-read-fail-closed, prerelease-filter, binary-keep, idempotent-skip, sha-check, guard-refusal), proves each mutation landed (anchor found exactly once, copy differs, mutant parses), runs the suite against the mutant and demands red — 6/6 caught.

Local evidence: shellcheck -S warning -x clean on both scripts (also parses under macOS bash 3.2), bash -n clean, suite 30/30, mutations 6/6, go build ./... unchanged (no Go changes), actionlint clean on build.yml. Both harness modes are wired into the Installer (shell) job (~1 min + ~1.5 min).

Open questions for review

1. Release notes default. The ask was "same body semantics", so --notes source is the default and carries the source body (guard-scanned, footer appended). The workflow uses fixed notes for new releases. Backfilled releases will therefore read differently from future ones unless the backfill is run with --notes fixed. Which should the runbook step use?
2. Report-tier needles in old bodies. Most historical bodies carry report-tier hits (internal ticket references). Default (non-strict) counts them, matching the workflow's default; --strict would refuse almost every release. Run non-strict, or --notes fixed?
3. Companions follow binaries. .sig/.cert are small text files but travel with their binary (a signature without the thing it signs is noise), so they are also cut at BINARY_KEEP. Confirm.

Checklist

• Read the full diff
• Lint + tests run locally (evidence above)
• No secrets, customer names, or internal references in code, comments, or this PR
• Docs updated (scripts/RELEASE_CHECKLIST.md §8; script header carries the contract)

🤖 Generated with Claude Code

---

Note

Medium Risk
Operator-run tooling that can write mirror releases and assets via gh, but dry-run is default and the same publish guard/mirror rules as production are enforced and heavily tested in CI.

Overview
Adds scripts/backfill-releases.sh, a human-run, idempotent tool to copy historical GitHub releases from the source repo onto the public mirror (complementing ongoing mirror-publish.yml). Every matching release gets tags, GitHub releases, and text assets; binaries and cosign .sig/.cert only for the newest BINARY_KEEP (default 10). Mirror tags are annotated markers on the default-branch head; default --notes fixed matches the workflow (with --notes source opt-in). Reuses publish-mirror.sh target and publish-guard.sh; binaries are checked against source SHA256SUMS; conflicting mirror assets and dangling tags are refused, not overwritten.

Documents the procedure in scripts/RELEASE_CHECKLIST.md §8. CI gains shellcheck/bash -n for the new scripts plus scripts/tests/backfill-releases-verify.sh (fake gh, ~30 property tests) and a --mutations step that breaks anchored rules and expects the suite to fail.

^{Reviewed by Cursor Bugbot for commit cc68221. Bugbot is set up for automated code reviews on this repo. Configure here.}

[LukasWodka]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[LukasWodka]

bugbot run

[LukasWodka]

Decision recorded in bf1e1e1: scripts/backfill-releases.sh now defaults to --notes fixed, the same fixed release notes mirror-publish.yml writes for new releases (plus the original-date footer). Reason: the historical release bodies are GitHub's generated pull-request lists, and nearly every one carries strings the publish guard's report tier counts; the public mirror should not repeat them, and under --strict they would refuse most of the backfill. --notes source stays available as an explicit opt-in and still goes through the guard. The header, RELEASE_CHECKLIST.md section 8 and the verify suite are updated; the default is pinned by a new notes-default-fixed mutation that reddens four tests when flipped back (7/7 mutations caught, 34/34 tests green).

[cursor]

Stale Bugbot comment from a previous run.

[LukasWodka]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[saqlainsyed007]

Reviewed the full script (scripts/backfill-releases.sh, 543 lines) and the
test harness, not just the diff hunks or PR description.

What I verified as SAFE (contrary to the "no re-run" framing in the title,
this is actually genuinely idempotent and non-destructive):

• No destructive git operations anywhere: grepped for force-push, tag -f,
  -X DELETE, --force — none exist. Every write is an additive POST
  (git/tags, git/refs, releases, release upload).
• Assets are compared by SHA256 and skipped if already matching; if an
  asset already on the mirror has a DIFFERENT digest than the source, it
  is refused, never overwritten ("a published asset is never replaced").
• Mirror tags: an existing tag is accepted only if it already points at a
  commit the mirror has; a dangling tag is refused, never repointed.
• Mirror target name comes from the existing, already-reviewed
  publish-mirror.sh target (unset/malformed/self-referential mirror is
  refused there) — not re-derived here.
• Every text asset and every release body is scanned by the existing
  publish-guard.sh (forbidden-string list + gitleaks) before any upload;
  binaries are verified byte-for-byte against the source's SHA256SUMS
  before upload and refused by name on mismatch.
• Prereleases and drafts are excluded by default (only via
  --include-prerelease), so nothing internal-only leaks to the public
  mirror by accident.
• The previously-flagged Bugbot finding ("error exits swallowed in
  substitutions") is fixed: die2() now writes to stderr (visible even
  when the call site is wrapped in $(...)), and I grepped the whole
  script for the anti-pattern (a die2-capable substitution living inside
  &&/||/[ ], where set -e is suspended) — found none; every
  jq_of/gh_read assignment that can call die2 is its own standalone
  statement, so the exit 2 from the subshell aborts the script.

Two Bugbot findings remain open and I confirmed both are real:

1. "Refused newest leaves no latest" (Medium) — confirmed at the
   REL_ACTION=create block: make_latest is only set true for
   NEWEST_STABLE, and every other release is created with
   make_latest=false. If NEWEST_STABLE itself gets refused (guard hit,
   SHA mismatch, dangling tag), no release ends up marked latest and
   the mirror's releases/latest will 404 until a human notices the
   refusal-table exit-1 and reruns --only-tag for that release. Not
   data-corrupting, but a real silent gap worth a one-line guard (e.g.
   promote the next-newest surviving stable release to latest, or at
   least call it out explicitly in the summary line).

2. "Scratch commit honors global gpgsign" (Medium) — confirmed at the
   guard's scratch-checkout commit: only -c user.name / -c user.email are set, no -c commit.gpgsign=false. On an operator
   machine with a global commit.gpgsign=true, this commit will try
   (and fail) to sign as backfill@localhost, and the whole run
   die2's before any release is even planned. Fails closed (no
   corruption risk), but it's an avoidable false negative for the
   human who's supposed to run this once — worth the one-line fix
   since this script (unlike the CI publisher using the same idiom)
   is explicitly run from a human's machine.

Recommend fixing both before merge — they're small, well-isolated fixes
and this is exactly the kind of one-shot public-mirror script where
"we'll deal with it if it comes up" isn't a great posture. Everything
else — the idempotency model, the guard integration, the SHA
verification, the binary retention window, the annotated-tag anchoring —
is careful and well-tested (30 assertions + 6/6 mutation kill rate).

[LukasWodka]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[saqlainsyed007]

Re-reviewed at head 24929ca3 (moved from 6cfa9bf0).

My two original findings are genuinely fixed, verified against the live code:

• "Refused newest leaves no latest" — NEWEST_STABLE_REFUSED is now set whenever the
  newest-stable tag is refused during its own create path, and the post-loop block
  marks the last stable release actually written this run as latest (or warns by name
  if none was written). Pinned by a dedicated harness case and a latest-fallback
  mutation anchor.
• "Scratch commit honors global gpgsign" — the scratch checkout commit now carries
  -c commit.gpgsign=false (scripts/backfill-releases.sh:264). Pinned by a harness case
  that runs under a forced-failing gpg.program and a scratch-commit-unsigned
  mutation anchor.

But Bugbot's new finding on this same commit is real, and the fix for the first item
introduces it: the fallback PATCH at line 552 —

gh_write "$TMP/latest.json" api -X PATCH "repos/$MIRROR/releases/$LAST_STABLE_CREATED_ID" -F make_latest=true

— uses -F, which types make_latest as a JSON boolean. GitHub's release-update API
takes make_latest as a string enum ("true"/"false"/"legacy"), which is exactly
why the create path two lines away (line 525) uses -f "make_latest=$LATEST" instead.
A boolean value here will 422, die2 fires, and the table never prints — so the exact
scenario this fallback exists to handle (newest-stable refused) ends with the fallback
itself failing closed and releases/latest still 404ing. Confirmed by reading the file
directly and comparing both call sites, not just trusting Bugbot's claim.

One-line fix: change -F make_latest=true to -f make_latest=true at line 552, to
match the string-typed create-path call.

Requesting changes for this one regression — everything else from my prior review
stands fixed.

[LukasWodka]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit cc68221. Configure here.}

[saqlainsyed007]

Fourth and final pass on this one. All three real bugs found across this review are now confirmed fixed by reading the actual file content at each step, not diffs or commit messages:

1. Newest-stable-refusal left no release marked latest — fixed via the NEWEST_STABLE_REFUSED fallback that marks the last stable release actually written as latest, or warns by name if none landed.
2. The guard's scratch git commit didn't override commit.gpgsign — fixed with -c commit.gpgsign=false.
3. The fallback's PATCH call typed make_latest as a JSON boolean (-F) instead of the string enum GitHub's API expects — fixed to -f make_latest=true, matching the create-path convention at line 525.

All CI is green (Build/Test/Lint/CodeQL/govulncheck/quality/*, both Cursor Bugbot and bugbot/review), mergeable, and all 4 review threads are resolved with 0 open. No new findings surfaced on this final commit.

Approving.