From 0ce11db7df23e2e564f2e0c6119984930e9b87d9 Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Thu, 20 Aug 2026 17:24:01 +0200 Subject: [PATCH 01/10] feat(setup-semstat): extract the semstat installer into a reusable action The installer, its suite and the release pin move out of semver-validation so that anything needing semstat installs it the same way instead of carrying a copy. The script keeps its own bats suite, which serves a release from disk through a stubbed curl and cosign, so the download, the checksum verification and the version cross-check all run for real. Two things arrive with the move. The install directory goes on GITHUB_PATH, because every consumer so far calls semstat from inside a bash function or a `while read` loop where a step output is not in scope, and the path output is kept for callers that would rather name the binary. And verify-signature cosign-verifies checksums.txt against semstat's release workflow at the exact tag before reading it, off by default because it costs a cosign install on the job. The two markers answer different questions and both are kept: the digest says the cached binary is the one that passed, the signature marker says the release it came out of was checked. A step asking to verify is not served by whatever an earlier unverified step left behind. The stale signature claim is dropped before this run's binary lands, so a run killed mid-install leaves a binary with no claim rather than a claim with no basis. DEFAULT_VERSION in the script is the pin, read by a renovate customManager, because the script is an entry point in its own right: an action in this repository runs it directly rather than pinning this action by SHA. Two defaults would be two pins that can disagree. The jq gate does not come along. jq is what semver-validation reads semstat's output with, not something installing semstat needs, and report.sh already names it with the same error. --- .github/actions/setup-semstat/README.md | 213 ++++++++++++ .github/actions/setup-semstat/action.yml | 50 +++ .../src/install-semstat.sh | 167 +++++++--- .../test/install-semstat.bats | 303 ++++++++++++++++-- .github/workflows/test-setup-semstat.yaml | 153 +++++++++ Makefile | 7 +- docs/CONVENTIONS.md | 2 +- renovate.json | 6 + 8 files changed, 840 insertions(+), 61 deletions(-) create mode 100644 .github/actions/setup-semstat/README.md create mode 100644 .github/actions/setup-semstat/action.yml rename .github/actions/{semver-validation => setup-semstat}/src/install-semstat.sh (59%) rename .github/actions/{semver-validation => setup-semstat}/test/install-semstat.bats (61%) create mode 100644 .github/workflows/test-setup-semstat.yaml diff --git a/.github/actions/setup-semstat/README.md b/.github/actions/setup-semstat/README.md new file mode 100644 index 00000000..e7d83de9 --- /dev/null +++ b/.github/actions/setup-semstat/README.md @@ -0,0 +1,213 @@ +# Setup semstat Action + +Installs [`semstat`](https://github.com/loft-sh/semstat), the semver CLI this +repository's release actions answer semver questions with, from a pinned release. +Resolves the runner's os and arch, downloads the archive and the release +`checksums.txt`, verifies the one line that covers the archive, unpacks it, and +proves the result runs by comparing `semstat version` against the tag that was +asked for. + +The binary lands on `PATH`, so the actions that call semstat from inside a shell +function or a `while read` loop can just run `semstat`. The absolute path is an +output as well, for a caller that would rather name the binary than depend on +`PATH` ordering. + +One pin lives here rather than one per action, so there is one place to bump and +one release under review rather than five that can drift apart. An action in this +repository runs the installer out of the same checkout and so is never behind it; +a consumer in another repository pins this action by SHA and gets a bump when that +pin moves. + +## Runner requirements + +A Linux or macOS runner with `curl`, `tar` and `sha256sum` (or `shasum`) on it, +and egress to `github.com/loft-sh/semstat/releases/download` **and** +`objects.githubusercontent.com`, which release-asset downloads redirect to. A +proxy allowlist naming only `github.com` fails the download. + +`verify-signature: true` needs two more hosts. `cosign` is downloaded from the +`github.com/sigstore/cosign` releases, by this action rather than by the runner +image, and cosign then checks the transparency log against a trusted root it +fetches from `tuf-repo-cdn.sigstore.dev`. An egress-restricted runner has to +allow both or the step fails inside cosign. + +## Inputs + + + +| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION | +|------------------|--------|----------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| verify-signature | string | false | `"false"` | Verify `checksums.txt` against its cosign bundle
before trusting it, proving the release
came from semstat's own release workflow
at this exact tag rather than
only that the download arrived intact.
Costs a cosign install on the
job, so it is off by
default; turn it on for jobs
that publish. | +| version | string | false | | Release of [loft-sh/semstat](https://github.com/loft-sh/semstat) to install. Empty
installs the release pinned in `src/install-semstat.sh`,
which is where Renovate bumps it
and where every entry point reads
it from. | + + + +## Outputs + + + +| OUTPUT | TYPE | DESCRIPTION | +|--------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------| +| path | string | Absolute path to the verified semstat
binary. The directory holding it is
also on PATH for later steps,
so a caller can just run
`semstat`. | + + + +## Usage + +### From a workflow, or from an action in another repository + +```yaml +- name: Install semstat + id: semstat + uses: loft-sh/github-actions/.github/actions/setup-semstat@ # setup-semstat/v1 + +- name: Pick the newest tag + shell: bash + run: | + newest="" + while read -r tag; do + semstat validate "$tag" 2>/dev/null || continue + if [ -z "$newest" ] || semstat gt "$tag" "$newest"; then + newest="$tag" + fi + done < <(git tag -l 'v*') + echo "$newest" +``` + +Pin the full commit SHA with the tag in a trailing comment, the way `cve-scan` +and `govulncheck` reference `ci-test-notify`. A fix to the installer reaches such +a consumer only when its SHA is bumped, so releasing this action means advancing +`setup-semstat/v1` and then moving each cross-repo pin onto the new commit. + +### From another action in this repository + +Run the script instead of pinning the action: + +```yaml +- name: Install semstat + id: install + shell: bash + run: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh +``` + +Neither form of `uses:` works for a sibling in the same repository. A relative +`uses: ./...` resolves against the caller's workspace rather than against this +repository, so it finds nothing once the action is consumed from `vcluster-pro` or +`loft-enterprise`. A full `loft-sh/github-actions/...@` pin does resolve, but +a sibling can only be pinned at a commit that predates the change needing it, and +it then keeps running that commit while fixes to the installer land beside it — +the pin drift from DEVOPS-1126 and DEVOPS-923, silent because the action still +works. `github.action_path` is inside a full checkout of this repository, so the +two ship from one commit and cannot drift apart. `semver-validation` is the +worked example. + +The script takes `SEMSTAT_VERSION` and `SEMSTAT_VERIFY_SIGNATURE` from the +environment, both optional. Clear `SEMSTAT_BASE_URL` in the same `env:` — a +composite step inherits the job's environment, so a workflow-level `env:` would +otherwise repoint the download root, and step env is what wins. Verifying also +needs `cosign` on `PATH`, which is a `uses:` step and so cannot be lent out by a +script: + +```yaml +- name: Install cosign + if: inputs.verify-signature == 'true' + uses: sigstore/cosign-installer@ # v4.1.2 + +- name: Install semstat + id: install + shell: bash + env: + SEMSTAT_VERIFY_SIGNATURE: ${{ inputs.verify-signature }} + SEMSTAT_BASE_URL: "" + run: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh +``` + +The step output is `semstat` rather than this action's `path` output, because the +script writes it directly: `${{ steps.install.outputs.semstat }}`. + +### Naming the binary instead of relying on PATH + +```yaml +- name: Install semstat + id: semstat + uses: loft-sh/github-actions/.github/actions/setup-semstat@ # setup-semstat/v1 + +- name: Order two versions + shell: bash + env: + SEMSTAT_BIN: ${{ steps.semstat.outputs.path }} + run: "$SEMSTAT_BIN" gt "$CANDIDATE" "$CURRENT" +``` + +### Verifying who produced the release + +```yaml +- name: Install semstat + uses: loft-sh/github-actions/.github/actions/setup-semstat@ # setup-semstat/v1 + with: + verify-signature: true +``` + +`checksums.txt` on its own proves the download arrived intact and that the asset +name resolved inside the release that was asked for. It does not prove who +produced that release, because the manifest comes from the same place as the +archive. With `verify-signature: true` the action installs `cosign` and verifies +`checksums.txt` against its Sigstore bundle before reading it, at the exact +identity: + +``` +https://github.com/loft-sh/semstat/.github/workflows/release.yaml@refs/tags/ +``` + +The tag is part of the identity, not a wildcard, so a bundle signed by the same +workflow on any other ref does not pass. It is off by default because it costs a +cosign install on every job that touches a release path, and because signature +verification of a first-party binary out of a first-party release is largely +ceremony. Turn it on for jobs that publish. + +## Reusing an install across steps + +The unpack directory is keyed by release and platform under `RUNNER_TEMP`, so a +job that installs semstat in several steps downloads the release once. A cached +install is reused only if the binary still runs, still agrees about its version, +and was verified as strictly as the step asks: a step with +`verify-signature: true` re-installs over whatever an earlier unverified step +left behind rather than trusting it. + +The runner *prepends* `GITHUB_PATH` entries, so where two steps install different +semstat releases into the same job, a later bare `semstat` resolves to whichever +installed last. Anything that cares which release it is talking to should name the +`path` output instead of relying on that order. + +## Upgrading the pinned release + +`DEFAULT_VERSION` in `src/install-semstat.sh` is the pin, carrying the +`# renovate: datasource=github-releases depName=loft-sh/semstat` comment that a +`customManager` in `renovate.json` reads. It lives in the script rather than in +`action.yml` because both entry points reach the script, and the `version` input +only overrides it; two defaults would be two pins that can disagree. + +Renovate opens one bump, and no consumer carries a pin of its own. An action in +this repository gets it in the same commit. A cross-repo consumer pins this action +by SHA, so for it the bump lands here first and ships when its pin moves — two +steps, the same way any change to this action ships. + +The action refuses a binary that reports a different version than the one asked +for, so a mismatched or truncated download fails the step rather than answering +wrongly. + +## Development + +```bash +make test-setup-semstat # bats suite +make lint # actionlint + zizmor +make generate-docs # refresh the tables above from action.yml +``` + +The suite serves a release from disk through a stubbed `curl` and a stubbed +`cosign`, so the download, the checksum verification, the signature check and the +version cross-check all run for real against artifacts the test builds. +`SEMSTAT_BASE_URL` is what repoints the download root, and it takes `file://` +URLs only: a remote value is refused rather than ignored, because a composite +step inherits the job's environment and a workflow-level `env:` would otherwise +repoint both the binary and the manifest it is checked against. diff --git a/.github/actions/setup-semstat/action.yml b/.github/actions/setup-semstat/action.yml new file mode 100644 index 00000000..1be5811c --- /dev/null +++ b/.github/actions/setup-semstat/action.yml @@ -0,0 +1,50 @@ +name: "Setup semstat" +description: "Installs a pinned release of the semstat semver CLI, verifies it, and puts it on PATH." +author: "vCluster Labs" +branding: + icon: "download" + color: "green" + +inputs: + version: + description: | + Release of [loft-sh/semstat](https://github.com/loft-sh/semstat) to install. + Empty installs the release pinned in `src/install-semstat.sh`, which is + where Renovate bumps it and where every entry point reads it from. + required: false + default: "" + verify-signature: + description: | + Verify `checksums.txt` against its cosign bundle before trusting it, proving the + release came from semstat's own release workflow at this exact tag rather than + only that the download arrived intact. Costs a cosign install on the job, so it + is off by default; turn it on for jobs that publish. + required: false + default: "false" + +outputs: + path: + description: "Absolute path to the verified semstat binary. The directory holding it is also on PATH for later steps, so a caller can just run `semstat`." + value: ${{ steps.install.outputs.semstat }} + +runs: + using: "composite" + steps: + - name: Install cosign + if: inputs.verify-signature == 'true' + uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2 + + - name: Install semstat + id: install + shell: bash + env: + SEMSTAT_VERSION: ${{ inputs.version }} + SEMSTAT_VERIFY_SIGNATURE: ${{ inputs.verify-signature }} + # Composite steps inherit the job's environment, and the installer takes + # a file:// download root for its own tests. Cleared here, where step env + # wins, so a workflow-level env: or an earlier step writing GITHUB_ENV + # cannot repoint which executable this action runs: the archive and the + # checksums.txt it is verified against come from the same root, so a + # planted release verifies against its own planted manifest. + SEMSTAT_BASE_URL: "" + run: ${{ github.action_path }}/src/install-semstat.sh diff --git a/.github/actions/semver-validation/src/install-semstat.sh b/.github/actions/setup-semstat/src/install-semstat.sh similarity index 59% rename from .github/actions/semver-validation/src/install-semstat.sh rename to .github/actions/setup-semstat/src/install-semstat.sh index 423e00d9..07afb18e 100755 --- a/.github/actions/semver-validation/src/install-semstat.sh +++ b/.github/actions/setup-semstat/src/install-semstat.sh @@ -1,26 +1,37 @@ #!/usr/bin/env bash # Downloads the semstat release binary, verifies it against the release -# checksums, and reports where it landed. +# checksums, puts it on PATH and reports where it landed. # # Emits one GitHub Actions step output: # # semstat absolute path to the verified, executable binary # +# and appends the directory holding it to GITHUB_PATH, because every consumer so +# far calls semstat from inside a bash function or a `while read` loop, where a +# step output is not in scope. The output is kept as well, for a caller that +# would rather name the binary than depend on PATH ordering. +# # Required environment: -# SEMSTAT_VERSION release tag to install, with or without a leading "v". # GITHUB_OUTPUT standard GitHub Actions step output file. +# GITHUB_PATH standard GitHub Actions path file. # RUNNER_TEMP where to unpack. Per-job and cleared by the runner. # # Optional environment: +# SEMSTAT_VERSION release tag to install, with or without a leading "v". +# Empty or unset installs DEFAULT_VERSION below. +# SEMSTAT_VERIFY_SIGNATURE "true" to cosign-verify checksums.txt against the +# release workflow before trusting it. Needs cosign +# on PATH; the action installs it when asked. # SEMSTAT_BASE_URL file:// release download root. For the tests; see below. set -euo pipefail : "${GITHUB_OUTPUT:?GITHUB_OUTPUT is required}" -: "${SEMSTAT_VERSION:?semstat-version is required}" +: "${GITHUB_PATH:?GITHUB_PATH is required}" # Not defaulted to /tmp: RUNNER_TEMP is private to one job, and the install # directory below is a name anyone could predict. On a runner with a shared # writable /tmp, the fallback would let another user plant a binary there for -# the reuse check to adopt. +# the reuse check to adopt, or plant an empty signature marker over it so that a +# step asking to verify takes the cache-hit path and never runs cosign. : "${RUNNER_TEMP:?RUNNER_TEMP is required; this action expects a GitHub Actions runner}" # Checked up front rather than where each is first reached. A runner missing one @@ -40,30 +51,12 @@ if ! command -v sha256sum >/dev/null 2>&1 && ! command -v shasum >/dev/null 2>&1 exit 1 fi -# Named here even though nothing below uses it: jq is what the reporting step of -# this composite reads semstat's output with, and the caller cannot run one step -# without the other. Said before the download rather than after it, so a runner -# missing jq is told so instead of paying for an install it cannot use. -if ! command -v jq >/dev/null 2>&1; then - echo "::error::jq is required to read semstat's output and is not on PATH" - exit 1 -fi - # A workflow command ends at a newline, so anything the downloaded binary prints # has to be folded onto one line before it is echoed back. fold_lines() { printf '%s' "$1" | tr '\r\n' ' ' } -# The path is the whole answer this step gives the next one, so a write that did -# not land has to fail the step rather than hand the caller an empty SEMSTAT_BIN. -emit_path() { - if ! echo "semstat=${binary}" >>"$GITHUB_OUTPUT"; then - echo "::error::could not write the semstat path to GITHUB_OUTPUT" - exit 1 - fi -} - # One of the two is known to be on PATH by the gate above. sha256_of() { if command -v sha256sum >/dev/null 2>&1; then @@ -75,6 +68,19 @@ sha256_of() { BASE_URL="https://github.com/loft-sh/semstat/releases/download" +# The pin lives here rather than in action.yml because both entry points reach +# this script: the action, and a sibling action in this repository running it off +# github.action_path to avoid pinning a sibling by SHA. One line to bump, and no +# way for the two to install different releases. +# renovate: datasource=github-releases depName=loft-sh/semstat +DEFAULT_VERSION=v0.0.2 + +# The workflow that publishes the release, and so the identity its signatures +# carry. Verified at the exact tag rather than at a branch or a glob: a bundle +# signed by this workflow on any other ref does not attest this release. +SIGNING_WORKFLOW="https://github.com/loft-sh/semstat/.github/workflows/release.yaml" +OIDC_ISSUER="https://token.actions.githubusercontent.com" + # The tests serve a release from disk. A remote override is refused rather than # ignored, because composite steps inherit the job's environment: a workflow-level # env: or an earlier step writing GITHUB_ENV would otherwise repoint which @@ -90,9 +96,21 @@ if [ -n "${SEMSTAT_BASE_URL-}" ]; then esac fi -case "$SEMSTAT_VERSION" in - v*) tag="$SEMSTAT_VERSION" ;; - *) tag="v${SEMSTAT_VERSION}" ;; +# Anything but the two booleans is a typo rather than a no, and reading a typo as +# "do not verify" would answer a request for the stricter check with the weaker one. +case "${SEMSTAT_VERIFY_SIGNATURE:-false}" in + true) verify_signature=true ;; + false) verify_signature=false ;; + *) + echo "::error::verify-signature takes true or false; got $(fold_lines "$SEMSTAT_VERIFY_SIGNATURE")" + exit 1 + ;; +esac + +requested="${SEMSTAT_VERSION:-$DEFAULT_VERSION}" +case "$requested" in + v*) tag="$requested" ;; + *) tag="v${requested}" ;; esac version="${tag#v}" @@ -102,7 +120,7 @@ version="${tag#v}" # own, and what keeps path segments out of the URL; the echoes downstream carry # it unfolded because it got through this. if ! [[ $version =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$ ]]; then - echo "::error::semstat-version must be a semantic version, with or without a leading v; got $(fold_lines "$SEMSTAT_VERSION")" + echo "::error::the version to install must be a semantic version, with or without a leading v; got $(fold_lines "$requested")" exit 1 fi @@ -131,23 +149,52 @@ archive="semstat_${version}_${os}_${arch}.tar.gz" # per-job and the runner clears it, so nothing is carried into another job. install_dir="${RUNNER_TEMP}/semstat-${version}-${os}-${arch}" binary="${install_dir}/semstat" + # Written only after the checksum and the version cross-check passed, and holds # the digest of the binary they passed for. Reuse asks the marker rather than # the binary, because a binary that prints the right version proves nothing # about the rest of itself. marker="${install_dir}/.verified" +# A second marker, because the two questions are different: the digest says this +# is the binary that passed, and this says the release it came out of was +# signature-checked. A step asking to verify is not handed an install an earlier +# unverified step left behind. +verified_marker="${install_dir}/.signature-verified" + +# PATH for the callers that reach semstat from a function or a loop, the output +# for the ones that would rather be explicit about which binary they mean. Both +# are the whole answer this step gives the next one, so a write that did not land +# has to fail the step rather than hand the caller an empty SEMSTAT_BIN. +report_install() { + if ! echo "semstat=${binary}" >>"$GITHUB_OUTPUT"; then + echo "::error::could not write the semstat path to GITHUB_OUTPUT" + exit 1 + fi + if ! printf '%s\n' "$install_dir" >>"$GITHUB_PATH"; then + echo "::error::could not add ${install_dir} to GITHUB_PATH" + exit 1 + fi +} + # An installed binary is reused only if this script verified it earlier in this -# job, it has not changed since, and it still runs and agrees about its version. +# job, it has not changed since, it still runs and agrees about its version, and +# it was verified as strictly as this step asks. if [ -x "$binary" ] && recorded="$(cat "$marker" 2>/dev/null)" && [ -n "$recorded" ] && [ "$recorded" = "$(sha256_of "$binary")" ] && - cached="$("$binary" version 2>/dev/null)" && [ "${cached#v}" = "$version" ]; then - emit_path + cached="$("$binary" version 2>/dev/null)" && [ "${cached#v}" = "$version" ] && + { [ "$verify_signature" = false ] || [ -f "$verified_marker" ]; }; then + report_install echo "semstat ${version} already installed at ${binary}" exit 0 fi +if [ "$verify_signature" = true ] && ! command -v cosign >/dev/null 2>&1; then + echo "::error::verify-signature needs cosign on PATH; the setup-semstat action installs it, so this is reachable only by running the script directly" + exit 1 +fi + if ! work="$(mktemp -d "${RUNNER_TEMP}/semstat.XXXXXX")" || [ -z "$work" ]; then echo "::error::could not create a working directory under RUNNER_TEMP to download semstat into" exit 1 @@ -159,9 +206,8 @@ trap 'rm -rf "$work"' EXIT # checksums.txt is fetched alongside the archive rather than pinned here, because # the version is an input. It proves the download arrived intact and that the # asset name resolved inside the release we asked for, and `semstat version` -# below proves we unpacked semstat. It is not a signature check and is not meant -# to be one: semstat is ours, so the release is trusted and this catches a bad -# transfer rather than an untrusted publisher. +# below proves we unpacked semstat. It says nothing about who published that +# release, which is the separate question SEMSTAT_VERIFY_SIGNATURE answers. for asset in "$archive" checksums.txt; do status=0 curl -fsSL --retry 3 --retry-connrefused --connect-timeout 10 --max-time 120 -o "${work}/${asset}" "${BASE_URL}/${tag}/${asset}" || status=$? @@ -173,6 +219,33 @@ for asset in "$archive" checksums.txt; do fi done +# Fetched on its own rather than in the loop above, because an absent bundle is a +# different failure than an absent release and must not be reported as one. +# semstat signs at the end of goreleaser's publish phase, so a Fulcio or Rekor +# outage there leaves the release published with the archives but no provenance. +if [ "$verify_signature" = true ]; then + bundle=checksums.txt.sigstore.json + if ! curl -fsSL --retry 3 --retry-connrefused -o "${work}/${bundle}" "${BASE_URL}/${tag}/${bundle}"; then + echo "::error::semstat ${tag} publishes no ${bundle}, so its signature cannot be verified; the release may have been published through a signing outage, which a re-dispatch of its release workflow repairs" + exit 1 + fi +fi + +# Before checksums.txt is read rather than after: the signature is what says this +# manifest came from the release workflow, and every integrity claim below rests +# on the manifest. +if [ "$verify_signature" = true ]; then + identity="${SIGNING_WORKFLOW}@refs/tags/${tag}" + if ! cosign_output="$(cosign verify-blob "${work}/checksums.txt" \ + --bundle "${work}/checksums.txt.sigstore.json" \ + --certificate-identity="$identity" \ + --certificate-oidc-issuer="$OIDC_ISSUER" 2>&1)"; then + echo "::error::cosign could not verify checksums.txt for semstat ${tag} as ${identity}: $(fold_lines "$cosign_output")" + exit 1 + fi + echo "cosign verified checksums.txt for semstat ${tag} as ${identity}" +fi + # Pick out our line rather than running `sha256sum -c` over the whole file: # checksums.txt covers every archive and SBOM in the release and only one of # them was downloaded, and an absent line has to be an error rather than the @@ -233,14 +306,32 @@ fi # Moved into place only once verified, so nothing half-installed is left behind # for the reuse check above to find. -if ! mkdir -p "$install_dir" || - ! mv "${work}/semstat" "$binary" || - ! sha256_of "$binary" >"$marker"; then - # The marker is written last, so a failure part-way leaves an install the reuse - # check above will not adopt. +if ! mkdir -p "$install_dir"; then + echo "::error::could not create ${install_dir} to install semstat into" + exit 1 +fi + +# An older signature claim is dropped before this run's binary lands rather than +# after. The reverse order leaves a window where a run killed mid-install has +# replaced the binary but not the marker, so an unverified download sits under an +# earlier run's claim and the reuse check above hands it to a step that asked to +# verify. +if [ "$verify_signature" = false ]; then + rm -f "$verified_marker" +fi + +if ! mv "${work}/semstat" "$binary" || ! sha256_of "$binary" >"$marker"; then + # The digest marker is written last, so a failure part-way leaves an install + # the reuse check above will not adopt. echo "::error::could not install the verified semstat to ${binary}" exit 1 fi -emit_path +# Written after the binary, so the marker never claims more than what is +# installed. +if [ "$verify_signature" = true ]; then + : >"$verified_marker" +fi + +report_install echo "semstat ${version} installed at ${binary}" diff --git a/.github/actions/semver-validation/test/install-semstat.bats b/.github/actions/setup-semstat/test/install-semstat.bats similarity index 61% rename from .github/actions/semver-validation/test/install-semstat.bats rename to .github/actions/setup-semstat/test/install-semstat.bats index 1e43ec73..64ea74b3 100644 --- a/.github/actions/semver-validation/test/install-semstat.bats +++ b/.github/actions/setup-semstat/test/install-semstat.bats @@ -1,5 +1,5 @@ #!/usr/bin/env bats -# Tests for semver-validation/src/install-semstat.sh +# Tests for setup-semstat/src/install-semstat.sh # # Stubs curl with a local release directory, so the download, the checksum # verification and the version cross-check all run for real against artifacts @@ -14,6 +14,9 @@ setup() { export GITHUB_OUTPUT="$TEST_DIR/github_output" : >"$GITHUB_OUTPUT" + export GITHUB_PATH="$TEST_DIR/github_path" + : >"$GITHUB_PATH" + export RUNNER_TEMP="$TEST_DIR/runner-temp" mkdir -p "$RUNNER_TEMP" @@ -49,6 +52,22 @@ cp "$src" "$dest" MOCK chmod +x "$MOCK_DIR/curl" + export COSIGN_ARGS="$TEST_DIR/cosign_args" + : >"$COSIGN_ARGS" + + # cosign: records the whole invocation and reports OK. COSIGN_EXIT stands in + # for a bundle that does not verify. + cat >"$MOCK_DIR/cosign" <<'MOCK' +#!/usr/bin/env bash +printf '%s\n' "$*" >>"$COSIGN_ARGS" +if [ "${COSIGN_EXIT:-0}" -ne 0 ]; then + echo "Error: no matching signatures" >&2 + exit "$COSIGN_EXIT" +fi +echo "Verified OK" >&2 +MOCK + chmod +x "$MOCK_DIR/cosign" + publish_release v1.2.3 1.2.3 } @@ -82,6 +101,11 @@ MOCK done (cd "$dir" && sha256sum ./*.tar.gz | sed 's| \./| |' >checksums.txt) + + # The signature bundle is opaque to the script, which only hands it to cosign, + # so its contents matter no more than that the asset is there to be fetched. + printf '{"mediaType":"application/vnd.dev.sigstore.bundle+json;version=0.3"}\n' \ + >"$dir/checksums.txt.sigstore.json" } # Reads an output back out of GITHUB_OUTPUT. @@ -89,6 +113,16 @@ output_value() { sed -n "s/^$1=//p" "$GITHUB_OUTPUT" | tail -n1 } +# The directories the run put on PATH for the steps that follow it. +path_entries() { + cat "$GITHUB_PATH" +} + +# The assets fetched from the release so far, by name. +requested_assets() { + sed 's|.*/||' "$CURL_URLS" +} + # The archive name the run asked the release for. requested_archive() { grep -o '[^/]*\.tar\.gz$' "$CURL_URLS" | tail -n1 @@ -264,13 +298,58 @@ MOCK [[ "$output" == *"::error::semstat has no build for ppc64le"* ]] } -@test "fails when no version is given" { +@test "installs the pinned release when SEMSTAT_VERSION is unset" { + pinned="$(sed -n 's/^DEFAULT_VERSION=//p' "$SCRIPT")" + [ -n "$pinned" ] + unset SEMSTAT_VERSION run "$SCRIPT" + # The test serves one release from disk and it is not that one, so the download + # fails; which release was asked for is what this is about. + [ "$status" -eq 1 ] + [[ "$(cat "$CURL_URLS")" == *"/${pinned}/"* ]] +} + +@test "fails when GITHUB_PATH is not set" { + unset GITHUB_PATH + + run "$SCRIPT" + + [ "$status" -ne 0 ] + [[ "$output" == *"GITHUB_PATH is required"* ]] +} + +# Refused rather than falling back to /tmp: the install directory carries the +# signature-verified marker, and on a shared /tmp anyone could forge it. +@test "fails when RUNNER_TEMP is not set" { + unset RUNNER_TEMP + + run "$SCRIPT" + [ "$status" -ne 0 ] - [[ "$output" == *"semstat-version is required"* ]] + [[ "$output" == *"RUNNER_TEMP is required"* ]] + [ ! -s "$CURL_URLS" ] +} + +@test "leaves no work directory behind once installed" { + run "$SCRIPT" + + [ "$status" -eq 0 ] + [ -z "$(find "$RUNNER_TEMP" -maxdepth 1 -name 'semstat.*' -print -quit)" ] +} + +# The archive and any half-unpacked binary go with it, on every exit path rather +# than only the one that succeeded. +@test "leaves no work directory behind when the install fails" { + publish_release v1.2.4 1.2.3 + export SEMSTAT_VERSION=v1.2.4 + + run "$SCRIPT" + + [ "$status" -eq 1 ] + [ -z "$(find "$RUNNER_TEMP" -maxdepth 1 -name 'semstat.*' -print -quit)" ] } @test "refuses a version that is not a semantic version" { @@ -306,17 +385,6 @@ MOCK [ "$("$(output_value semstat)" version)" = "1.4.0-rc.1" ] } -@test "fails when RUNNER_TEMP is not set" { - # The fallback this replaces put the install directory at a predictable path - # under a /tmp anyone on the runner can write to. - unset RUNNER_TEMP - - run "$SCRIPT" - - [ "$status" -ne 0 ] - [[ "$output" == *"RUNNER_TEMP is required"* ]] -} - @test "refuses an archive whose semstat member is a symlink" { local dir="$RELEASE_DIR/v1.5.0" local staging="$TEST_DIR/staging-symlink" @@ -513,16 +581,168 @@ gate_path() { [ ! -s "$CURL_URLS" ] } -@test "a runner without jq says so before the release is downloaded" { - # jq is not used here but in the reporting step of the same composite, so a - # runner missing it is told before the install it cannot use is paid for. - run /usr/bin/env "PATH=$(gate_path curl tar sha256sum)" "$SCRIPT" +# PATH, not just the output: every consumer calls semstat from inside a bash +# function or a `while read` loop, where a step output is not in scope. +@test "puts the directory holding the binary on PATH" { + run "$SCRIPT" + + [ "$status" -eq 0 ] + [ "$(path_entries)" = "$(dirname "$(output_value semstat)")" ] +} + +@test "puts the binary on PATH again when the install is reused" { + run "$SCRIPT" + [ "$status" -eq 0 ] + installed="$(path_entries)" + : >"$GITHUB_PATH" + + run "$SCRIPT" + + [ "$status" -eq 0 ] + [[ "$output" == *"already installed"* ]] + [ "$(path_entries)" = "$installed" ] +} + +@test "leaves the signature alone unless asked" { + run "$SCRIPT" + + [ "$status" -eq 0 ] + [[ "$(requested_assets)" != *sigstore* ]] + [ ! -s "$COSIGN_ARGS" ] +} + +@test "verifies checksums.txt against the release workflow at the exact tag" { + export SEMSTAT_VERIFY_SIGNATURE=true + + run "$SCRIPT" + + [ "$status" -eq 0 ] + [[ "$(requested_assets)" == *checksums.txt.sigstore.json* ]] + [[ "$(cat "$COSIGN_ARGS")" == *"verify-blob"* ]] + [[ "$(cat "$COSIGN_ARGS")" == *"--bundle "*"checksums.txt.sigstore.json"* ]] + [[ "$(cat "$COSIGN_ARGS")" == *"--certificate-identity=https://github.com/loft-sh/semstat/.github/workflows/release.yaml@refs/tags/v1.2.3"* ]] + [[ "$(cat "$COSIGN_ARGS")" == *"--certificate-oidc-issuer=https://token.actions.githubusercontent.com"* ]] +} + +# A release can carry its archives and checksums.txt and no bundle, so this must +# not be reported as the release being absent. +@test "names the missing signature when the release publishes no bundle" { + export SEMSTAT_VERIFY_SIGNATURE=true + rm "$RELEASE_DIR/v1.2.3/checksums.txt.sigstore.json" + + run "$SCRIPT" + + [ "$status" -eq 1 ] + [[ "$output" == *"publishes no checksums.txt.sigstore.json"* ]] + [[ "$output" != *"check that the release exists"* ]] + [ -z "$(output_value semstat)" ] +} + +@test "fails when cosign does not verify the bundle" { + export SEMSTAT_VERIFY_SIGNATURE=true + export COSIGN_EXIT=1 + + run "$SCRIPT" + + [ "$status" -eq 1 ] + [[ "$output" == *"::error::cosign could not verify checksums.txt"* ]] + [[ "$output" == *"no matching signatures"* ]] + [ -z "$(output_value semstat)" ] +} + +# The error names the action, because a workflow that ran the script itself is +# the only way to reach a verify-signature request with no cosign on PATH. +# +# PATH is narrowed to a farm holding only what the script needs rather than +# trimmed of the mock, so the result does not depend on whether the machine +# running the tests happens to have a real cosign installed. +@test "fails when verification is asked for and cosign is not installed" { + export SEMSTAT_VERIFY_SIGNATURE=true + + farm="$TEST_DIR/no-cosign" + mkdir -p "$farm" + # `|| continue` rather than `&&`: as the last statement of the loop body, a + # miss would otherwise carry its status out of the loop and trip bats' set -e + # on a machine without one of these, failing the test at an unrelated line. + # sha256sum and shasum are both listed because the script takes either. + for tool in bash env uname mktemp awk tar chmod mv rm cat cut tr sha256sum shasum; do + real="$(command -v "$tool")" || continue + ln -sf "$real" "$farm/$tool" + done + ln -sf "$MOCK_DIR/curl" "$farm/curl" + + # Narrowed for the script only. Narrowing the test's own PATH would leave bats + # unable to find the tools it cleans up with once teardown removes the farm. + run env PATH="$farm" "$SCRIPT" + + [ "$status" -eq 1 ] + [[ "$output" == *"::error::verify-signature needs cosign on PATH"* ]] + [ -z "$(output_value semstat)" ] +} + +@test "refuses a verify-signature value that is neither true nor false" { + export SEMSTAT_VERIFY_SIGNATURE=yes + + run "$SCRIPT" [ "$status" -eq 1 ] - [[ "$output" == *"::error::jq is required"* ]] + [[ "$output" == *"verify-signature takes true or false; got yes"* ]] [ ! -s "$CURL_URLS" ] } +# A step asking for verification cannot be served by whatever an earlier +# unverified step left in the shared install directory. +@test "re-installs over an install that was not signature-verified" { + run "$SCRIPT" + [ "$status" -eq 0 ] + downloaded="$(download_count)" + + export SEMSTAT_VERIFY_SIGNATURE=true + + run "$SCRIPT" + + [ "$status" -eq 0 ] + [ "$(download_count)" -gt "$downloaded" ] + [ -s "$COSIGN_ARGS" ] +} + +@test "reuses an install that was signature-verified" { + export SEMSTAT_VERIFY_SIGNATURE=true + + run "$SCRIPT" + [ "$status" -eq 0 ] + downloaded="$(download_count)" + + run "$SCRIPT" + + [ "$status" -eq 0 ] + [[ "$output" == *"already installed"* ]] + [ "$(download_count)" -eq "$downloaded" ] +} + +# An unverified re-install must not inherit the earlier run's claim: the next +# verifying step would otherwise reuse a binary nothing verified. +@test "drops the verified marker when a later run does not verify" { + export SEMSTAT_VERIFY_SIGNATURE=true + run "$SCRIPT" + [ "$status" -eq 0 ] + + unset SEMSTAT_VERIFY_SIGNATURE + # Only running it says the install is unusable, so this forces a re-install + # while leaving the marker from the verified run in place. + printf 'truncated' >"$(output_value semstat)" + run "$SCRIPT" + [ "$status" -eq 0 ] + + export SEMSTAT_VERIFY_SIGNATURE=true + downloaded="$(download_count)" + + run "$SCRIPT" + + [ "$status" -eq 0 ] + [ "$(download_count)" -gt "$downloaded" ] +} + @test "a checksums.txt with CRLF line endings still verifies the archive" { # The \r is not whitespace to awk, so unstripped it stayed on the asset name # and the lookup missed, blaming the release for a missing archive that was @@ -538,9 +758,52 @@ gate_path() { @test "a RUNNER_TEMP that cannot be worked in says so rather than dying on mktemp" { export RUNNER_TEMP="$TEST_DIR/never-created" - run "$SCRIPT" [ "$status" -eq 1 ] [[ "$output" == *"::error::could not create a working directory under RUNNER_TEMP"* ]] } + +# An untouched `version:` input arrives as an empty string rather than unset, and +# the pin lives in the script so the action and a sibling action running that +# script off github.action_path cannot install different releases. +@test "installs the pinned release when the version input is empty" { + pinned="$(sed -n 's/^DEFAULT_VERSION=//p' "$SCRIPT")" + [ -n "$pinned" ] + + export SEMSTAT_VERSION="" + run "$SCRIPT" + + [ "$status" -eq 1 ] + [[ "$(cat "$CURL_URLS")" == *"/${pinned}/"* ]] +} + +# The marker is cleared before the binary is moved into place, so a run that dies +# in between cannot leave this run's unverified download under an earlier run's +# claim. Failing the move is how that window is reached from a test. +@test "clears an earlier verified claim even when the install does not complete" { + export SEMSTAT_VERIFY_SIGNATURE=true + run "$SCRIPT" + [ "$status" -eq 0 ] + + marker="$(dirname "$(output_value semstat)")/.signature-verified" + [ -f "$marker" ] + + unset SEMSTAT_VERIFY_SIGNATURE + # Only running it says the install is unusable, so this forces a re-install. + printf 'truncated' >"$(output_value semstat)" + + break_dir="$TEST_DIR/break" + mkdir -p "$break_dir" + cat >"$break_dir/mv" <<'MOCK' +#!/usr/bin/env bash +echo "mv: interrupted" >&2 +exit 1 +MOCK + chmod +x "$break_dir/mv" + + run env PATH="$break_dir:$PATH" "$SCRIPT" + + [ "$status" -ne 0 ] + [ ! -f "$marker" ] +} diff --git a/.github/workflows/test-setup-semstat.yaml b/.github/workflows/test-setup-semstat.yaml new file mode 100644 index 00000000..b96c524d --- /dev/null +++ b/.github/workflows/test-setup-semstat.yaml @@ -0,0 +1,153 @@ +name: Test setup-semstat + +on: + push: + branches: [main] + paths: + - '.github/actions/setup-semstat/**' + - '.github/workflows/test-setup-semstat.yaml' + pull_request: + paths: + - '.github/actions/setup-semstat/**' + - '.github/workflows/test-setup-semstat.yaml' + +permissions: {} + +jobs: + bats: + name: Run bats tests + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + - uses: bats-core/bats-action@77d6fb60505b4d0d1d73e48bd035b55074bbfb43 # 4.0.0 + with: + support-install: false + assert-install: false + detik-install: false + file-install: false + - name: Run setup-semstat tests + run: bats .github/actions/setup-semstat/test/*.bats + + action: + name: Run the action end to end + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + - name: Install semstat + id: install + uses: ./.github/actions/setup-semstat + + # The reason the action appends to GITHUB_PATH at all: this is the shape + # every consumer uses, semstat called by name from inside a function. + - name: Call semstat by name from a shell function + run: | + set -euo pipefail + + newer_than() { + semstat gt "$1" "$2" + } + + if ! newer_than v4.9.0 v4.9.0-rc.2; then + echo "::error::a release should outrank its own candidate" + exit 1 + fi + + # gt is an answer, not a failure, so a no has to stay green. + if newer_than v4.9.0-rc.2 v4.9.0; then + echo "::error::a candidate should not outrank its release" + exit 1 + fi + + - name: Call semstat through the path output + env: + SEMSTAT_BIN: ${{ steps.install.outputs.path }} + run: | + set -euo pipefail + + if [ ! -x "$SEMSTAT_BIN" ]; then + echo "::error::the path output is not an executable: '$SEMSTAT_BIN'" + exit 1 + fi + + "$SEMSTAT_BIN" validate v4.9.0-rc.2 + + # The inode, not just the path: path is built from the version and platform + # alone, so it is identical whether the second call reused the install or + # re-downloaded over it. A re-download moves a new file into place, which + # changes the inode. + - name: Note which file the first install left behind + id: before + env: + SEMSTAT_BIN: ${{ steps.install.outputs.path }} + run: echo "inode=$(stat -c %i "$SEMSTAT_BIN")" >>"$GITHUB_OUTPUT" + + - name: Install semstat again + id: reinstall + uses: ./.github/actions/setup-semstat + + - name: Assert the second install reused the first + env: + FIRST: ${{ steps.install.outputs.path }} + SECOND: ${{ steps.reinstall.outputs.path }} + BEFORE_INODE: ${{ steps.before.outputs.inode }} + run: | + set -euo pipefail + + if [ "$FIRST" != "$SECOND" ]; then + echo "::error::expected the same path; got '$FIRST' then '$SECOND'" + exit 1 + fi + + after="$(stat -c %i "$SECOND")" + if [ "$after" != "$BEFORE_INODE" ]; then + echo "::error::the release was downloaded again; inode went from $BEFORE_INODE to $after" + exit 1 + fi + + signature: + name: Verify the release signature + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false + + # Runs the real cosign against the real bundle, which is the half the bats + # suite stubs out. A change to how semstat's release workflow signs, or to + # the identity it signs as, fails here. + - name: Install semstat with signature verification + id: install + uses: ./.github/actions/setup-semstat + with: + verify-signature: true + + - name: Assert it installed + env: + SEMSTAT_BIN: ${{ steps.install.outputs.path }} + run: | + set -euo pipefail + + if [ ! -x "$SEMSTAT_BIN" ]; then + echo "::error::the path output is not an executable: '$SEMSTAT_BIN'" + exit 1 + fi + + "$SEMSTAT_BIN" validate v4.9.0 + + - name: Reject a verify-signature value that is not a boolean + id: bad_input + continue-on-error: true + uses: ./.github/actions/setup-semstat + with: + verify-signature: sometimes + + - name: Assert the bad value failed the step + if: steps.bad_input.outcome != 'failure' + run: | + echo "::error::verify-signature: sometimes should have failed the install step" + exit 1 diff --git a/Makefile b/Makefile index d4655945..66ca9070 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: test test-semver-validation test-linear-pr-commenter test-link-backport-prs test-release-notification test-linear-release-sync test-aws-test-infra test-cleanup-head-charts test-ci-test-notify test-auto-approve-bot-prs test-ai-pr-review test-ai-step test-publish-helm-chart test-govulncheck test-go-licenses test-run-ginkgo test-sticky-pr-comment test-repository-dispatch test-parse-label-filter test-release-branch-freeze test-prerelease-setup test-vcluster-release test-subtree-mirror test-oss-commit-sync test-backport-legacy-allowlist test-backport-legacy-split test-promote-release test-wait-for-release test-cve-scan test-commitlint test-resolve-github-release build-linear-release-sync lint check-bats-jobs install-auto-doc generate-docs check-docs help +.PHONY: test test-semver-validation test-setup-semstat test-linear-pr-commenter test-link-backport-prs test-release-notification test-linear-release-sync test-aws-test-infra test-cleanup-head-charts test-ci-test-notify test-auto-approve-bot-prs test-ai-pr-review test-ai-step test-publish-helm-chart test-govulncheck test-go-licenses test-run-ginkgo test-sticky-pr-comment test-repository-dispatch test-parse-label-filter test-release-branch-freeze test-prerelease-setup test-vcluster-release test-subtree-mirror test-oss-commit-sync test-backport-legacy-allowlist test-backport-legacy-split test-promote-release test-wait-for-release test-cve-scan test-commitlint test-resolve-github-release build-linear-release-sync lint check-bats-jobs install-auto-doc generate-docs check-docs help ACTIONS_DIR := .github/actions WORKFLOWS_DIR := .github/workflows @@ -93,11 +93,14 @@ check-docs: generate-docs ## verify docs are up to date (fails if drift detected help: ## show this help @grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf " %-30s %s\n", $$1, $$2}' -test: test-semver-validation test-linear-pr-commenter test-link-backport-prs test-release-notification test-linear-release-sync test-aws-test-infra test-cleanup-head-charts test-auto-approve-bot-prs test-ai-pr-review test-ai-step test-ci-test-notify test-go-licenses test-publish-helm-chart test-govulncheck test-run-ginkgo test-sticky-pr-comment test-repository-dispatch test-parse-label-filter test-release-branch-freeze test-prerelease-setup test-vcluster-release test-subtree-mirror test-oss-commit-sync test-backport-legacy-allowlist test-backport-legacy-split test-promote-release test-wait-for-release test-cve-scan test-commitlint test-resolve-github-release ## run all action tests +test: test-semver-validation test-setup-semstat test-linear-pr-commenter test-link-backport-prs test-release-notification test-linear-release-sync test-aws-test-infra test-cleanup-head-charts test-auto-approve-bot-prs test-ai-pr-review test-ai-step test-ci-test-notify test-go-licenses test-publish-helm-chart test-govulncheck test-run-ginkgo test-sticky-pr-comment test-repository-dispatch test-parse-label-filter test-release-branch-freeze test-prerelease-setup test-vcluster-release test-subtree-mirror test-oss-commit-sync test-backport-legacy-allowlist test-backport-legacy-split test-promote-release test-wait-for-release test-cve-scan test-commitlint test-resolve-github-release ## run all action tests test-semver-validation: ## run semver-validation bats tests bats $(ACTIONS_DIR)/semver-validation/test/*.bats +test-setup-semstat: ## run setup-semstat bats tests + bats $(ACTIONS_DIR)/setup-semstat/test/*.bats + test-linear-pr-commenter: ## run linear-pr-commenter unit tests cd $(ACTIONS_DIR)/linear-pr-commenter/src && go test -v ./... diff --git a/docs/CONVENTIONS.md b/docs/CONVENTIONS.md index b4008642..271bde15 100644 --- a/docs/CONVENTIONS.md +++ b/docs/CONVENTIONS.md @@ -11,7 +11,7 @@ Three tiers based on complexity: |---|---|---| | **YAML-only composite** | Thin glue wiring inputs to third-party actions, no custom logic | `release-notification` | | **Compiled action (Go/Node.js)** | Business logic, API calls, data transformation | `linear-pr-commenter` (Go), `linear-release-sync` (Go) | -| **Composite over a released binary** | The logic already ships as a versioned CLI; the action installs it, verifies it, and maps its answers to outputs | `semver-validation` (over `semstat`) | +| **Composite over a released binary** | The logic already ships as a versioned CLI; one action installs and verifies it, the others map its answers to outputs | `setup-semstat` (installs `semstat`), `semver-validation` (maps it) | | **Reusable workflow** | Cross-repo orchestration of multiple jobs | `backport.yaml`, `actionlint.yaml` | **Key rule:** Business logic (branching, loops, parsing, API calls) MUST live in diff --git a/renovate.json b/renovate.json index 33d2907c..366c6fc2 100644 --- a/renovate.json +++ b/renovate.json @@ -43,6 +43,12 @@ "depNameTemplate": "renovate", "datasourceTemplate": "npm" }, + { + "customType": "regex", + "description": "Update the semstat release pinned in the setup-semstat installer", + "managerFilePatterns": ["/^\\.github/actions/setup-semstat/src/install-semstat\\.sh$/"], + "matchStrings": ["# renovate: datasource=(?\\S+) depName=(?\\S+)\\s+DEFAULT_VERSION=(?\\S+)"] + }, { "customType": "regex", "description": "Update versions annotated with `# renovate: datasource=X depName=Y` in workflow inputs and composite action inputs", From 9a03d1c642b117c2894b80213ff79fab999676a4 Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Thu, 20 Aug 2026 17:24:12 +0200 Subject: [PATCH 02/10] refactor(semver-validation): run the sibling installer instead of pinning it A sibling in the same repository cannot be pinned correctly: the commit that adds the dependency is always newer than any commit available to pin, so the pin names a commit that predates the change needing it and keeps running it while fixes to the installer land next to it. github.action_path is inside a full checkout of this repository, so running the script from there ships both actions from one commit and removes the drift class rather than guarding against it. A relative `uses: ./...` still does not work, which is why a pin was the alternative at all. There is no semstat_version input any more; the pin lives with the installer, so Renovate opens one bump rather than one per action that runs semstat. Nothing pins semver-validation/v4 yet, so the input goes rather than being deprecated. verify-signature is a passthrough. The installer documents verification as the thing to turn on for jobs that publish, and this action is what those jobs actually call, so the policy was otherwise unreachable. --- .github/actions/semver-validation/README.md | 98 +++++++++++++------ .github/actions/semver-validation/action.yml | 29 ++++-- .github/workflows/test-semver-validation.yaml | 23 +++++ README.md | 70 ++++++++++++- 4 files changed, 179 insertions(+), 41 deletions(-) diff --git a/.github/actions/semver-validation/README.md b/.github/actions/semver-validation/README.md index 348ef2c3..1f3c804d 100644 --- a/.github/actions/semver-validation/README.md +++ b/.github/actions/semver-validation/README.md @@ -4,10 +4,11 @@ Reports on a version string: whether it is valid [semver](https://semver.org/), what its parts are, which release channel it belongs to, and how it orders against another version. -The work is done by [`semstat`](https://github.com/loft-sh/semstat), which the -action downloads and checksum-verifies at the pinned release. One implementation -answers for the action and for the shell scripts in this repository, so there is -no second semver engine to disagree with the first. +The work is done by [`semstat`](https://github.com/loft-sh/semstat), installed and +checksum-verified by +[`setup-semstat`](../setup-semstat/README.md), which is where the release pin +lives. One implementation answers for the action and for the shell scripts in this +repository, so there is no second semver engine to disagree with the first. ## Runner requirements @@ -16,22 +17,33 @@ The action needs a Linux or macOS runner with `curl`, `tar`, `jq` and either `github.com/loft-sh/semstat/releases/download` **and** `objects.githubusercontent.com`, which release-asset downloads redirect to. A proxy allowlist that names only `github.com` fails the install step. - -That is new in `semver-validation/v4`. The tags before it (`v1`, `v2` and `v3`) all -point at the self-contained Node action, which needed neither the network nor those -tools. A caller on a runner without egress keeps working on those tags and fails the -install step on `v4`. `v1` through `v3` stay where they are for that reason; none of -them were advanced onto this rewrite. +`verify-signature: true` adds a `cosign` download from the +`github.com/sigstore/cosign` releases and egress to `tuf-repo-cdn.sigstore.dev`, +where cosign fetches the trusted root it checks the transparency log against. + +Calling it also puts semstat on `PATH` for the steps that follow, because that is +how the installer it shares with [`setup-semstat`](../setup-semstat/README.md) +installs it. That is a side effect on the caller's job, so a job with its own +semstat on `PATH` should name it by absolute path rather than let step order decide +which one a later bare `semstat` resolves to. + +All of that is new, so the rewrite ships as `semver-validation/v4`. The tags before +it (`v1`, `v2` and `v3`) all point at the self-contained Node action, which needed +neither the network, nor those tools, nor anything of the caller's `PATH`. A caller +on a runner without egress keeps working on those tags and fails the install step on +`v4`, so all three stay where they are and none is advanced onto this rewrite. Live +callers pin `v1` and `v3` as floating tags, so advancing either would hand them the +network and tool requirements with no version change to notice. ## Inputs -| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION | -|-----------------|--------|----------|------------|--------------------------------------------------------------------------------------------| -| compare_to | string | false | | Second version to order `version` against.
Leave empty to skip the comparison. | -| semstat_version | string | false | `"v0.0.2"` | Release of [loft-sh/semstat](https://github.com/loft-sh/semstat) to download and
run. | -| version | string | true | | Version string to validate against semver
format | +| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION | +|------------------|--------|----------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| compare_to | string | false | | Second version to order `version` against.
Leave empty to skip the comparison. | +| verify-signature | string | false | `"false"` | Verify the semstat release's `checksums.txt` against
its cosign bundle before trusting it,
proving the release came from semstat's
own release workflow at that exact
tag rather than only that the
download arrived intact. Costs a cosign
install on the job, so it
is off by default; turn it
on where this action's answer gates
a publish. | +| version | string | true | | Version string to validate against semver
format | @@ -80,8 +92,8 @@ them were advanced onto this rewrite. An invalid version is an answer, not a failure: the step stays green and sets `is_valid` to `false`, so the caller decides what that means. The step fails only -when it cannot answer at all: an empty `version`, a `semstat_version` with no -release, or a semstat that did not run. `version` and `compare_to` are trimmed +when it cannot answer at all: an empty `version`, a semstat release that could not +be installed, or a semstat that did not run. `version` and `compare_to` are trimmed before they are read, so surrounding whitespace does not change the answer, and a `version` that is only whitespace is an invalid version rather than a missing one. A binary that crashed must not report `is_valid=false` for a perfectly good tag, @@ -140,12 +152,32 @@ never affects it, and a prerelease always sorts below its final release, so `compare_to` is omitted or is not a version, which is not the same as `false`, so gate on `is_greater == 'true'`. +### Verify who produced the semstat release + +```yaml +- name: Is this tag newer than what is released? + id: semver + uses: loft-sh/github-actions/.github/actions/semver-validation@semver-validation/v4 + with: + version: ${{ github.ref_name }} + compare_to: ${{ steps.latest.outputs.tag }} + verify-signature: true +``` + +The release `checksums.txt` proves the download arrived intact and resolved inside +the release asked for, but it comes from the same place as the archive, so it says +nothing about who published either. `verify-signature: true` installs `cosign` and +checks `checksums.txt` against its Sigstore bundle at the exact signing identity +before reading it; [`setup-semstat`](../setup-semstat/README.md) documents the +identity and the cost. It is off by default because it adds a cosign install to +every job. Turn it on where this action's answer gates a publish. + ## From a shell script The action is a workflow step, so it cannot be called from inside a loop or a -shell function. Install -[`semstat`](https://github.com/loft-sh/semstat#install) and call it directly -instead: +shell function. Put semstat on `PATH` with +[`setup-semstat`](../setup-semstat/README.md) — or, outside a workflow, [install +it](https://github.com/loft-sh/semstat#install) — and call it directly instead: ```bash newest="" @@ -174,24 +206,30 @@ fi ## Upgrading the semstat release -`semstat_version` pins which release is downloaded, and Renovate opens the bump. -The action verifies the archive against the release checksums and refuses a -binary that reports a different version than the one asked for, so a mismatched -or truncated download fails the step rather than answering wrongly. That is a -transfer check, not a signature check, and deliberately so: semstat is ours, so -the release is trusted and the sigstore bundle it also publishes is left alone. +There is no `semstat_version` input: the pin lives in +[`setup-semstat`](../setup-semstat/README.md), so Renovate opens one bump there +rather than one per action that runs semstat. `setup-semstat` verifies the archive +against the release checksums and refuses a binary that reports a different version +than the one asked for, so a mismatched or truncated download fails the step rather +than answering wrongly. + +The installer runs out of the same checkout as this action rather than through a +`uses:` pin, so a Renovate bump of the release and a fix to the installer both +reach this action in the commit that makes them. See +[`setup-semstat`](../setup-semstat/README.md) for why a sibling in the same +repository cannot be pinned by SHA without stranding it. ## Development ```bash -make test-semver-validation # bats suites for both scripts +make test-semver-validation # bats suite for report.sh make lint # actionlint + zizmor make generate-docs # refresh the tables above from action.yml ``` -`src/install-semstat.sh` downloads and verifies the binary; `src/report.sh` runs -it and writes the outputs. semstat is stubbed in the tests, because what its -answers should be is settled by [its own +`src/report.sh` runs semstat and writes the outputs; installing it is +`setup-semstat`'s job and is tested there. semstat itself is stubbed in these +tests, because what its answers should be is settled by [its own suite](https://github.com/loft-sh/semstat); what is tested here is the translation into action outputs. `test-semver-validation.yaml` also runs the action end to end against the real release. diff --git a/.github/actions/semver-validation/action.yml b/.github/actions/semver-validation/action.yml index 27c2ce94..647bec0e 100644 --- a/.github/actions/semver-validation/action.yml +++ b/.github/actions/semver-validation/action.yml @@ -13,11 +13,15 @@ inputs: description: "Second version to order `version` against. Leave empty to skip the comparison." required: false default: "" - semstat_version: - description: "Release of [loft-sh/semstat](https://github.com/loft-sh/semstat) to download and run." + verify-signature: + description: | + Verify the semstat release's `checksums.txt` against its cosign bundle before + trusting it, proving the release came from semstat's own release workflow at + that exact tag rather than only that the download arrived intact. Costs a + cosign install on the job, so it is off by default; turn it on where this + action's answer gates a publish. required: false - # renovate: datasource=github-releases depName=loft-sh/semstat - default: v0.0.2 + default: "false" outputs: is_valid: @@ -60,11 +64,24 @@ outputs: runs: using: "composite" steps: + - name: Install cosign + if: inputs.verify-signature == 'true' + uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2 + + # setup-semstat's installer, run out of the checkout this action came in + # rather than through `uses:`. Neither form of `uses:` works for a sibling in + # the same repository: a relative `uses: ./...` resolves against the caller's + # workspace and finds nothing once this action is consumed from another repo, + # and a SHA pin can only name a commit from before the change that needs it, + # which then keeps running while fixes to the installer land elsewhere. Both + # actions ship from one commit, so running the script off github.action_path + # is what makes them impossible to drift apart. Which release is installed, + # and the Renovate comment tracking it, live with the script. - name: Install semstat id: install shell: bash env: - SEMSTAT_VERSION: ${{ inputs.semstat_version }} + SEMSTAT_VERIFY_SIGNATURE: ${{ inputs.verify-signature }} # Composite steps inherit the job's environment, and the installer takes # a file:// download root for its own tests. Cleared here, where step env # wins, so a workflow-level env: or an earlier step writing GITHUB_ENV @@ -72,7 +89,7 @@ runs: # checksums.txt it is verified against come from the same root, so a # planted release verifies against its own planted manifest. SEMSTAT_BASE_URL: "" - run: ${{ github.action_path }}/src/install-semstat.sh + run: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh - name: Report on the version id: report diff --git a/.github/workflows/test-semver-validation.yaml b/.github/workflows/test-semver-validation.yaml index 3b9063dc..b2a016cd 100644 --- a/.github/workflows/test-semver-validation.yaml +++ b/.github/workflows/test-semver-validation.yaml @@ -5,10 +5,16 @@ on: branches: [main] paths: - '.github/actions/semver-validation/**' + # The action runs setup-semstat's installer out of the same checkout, so a + # change there is a change to this one, and the end-to-end job runs it. + - '.github/actions/setup-semstat/**' - '.github/workflows/test-semver-validation.yaml' pull_request: paths: - '.github/actions/semver-validation/**' + # The action runs setup-semstat's installer out of the same checkout, so a + # change there is a change to this one, and the end-to-end job runs it. + - '.github/actions/setup-semstat/**' - '.github/workflows/test-semver-validation.yaml' permissions: {} @@ -89,6 +95,16 @@ jobs: version: ' v2.1.0 ' compare_to: ' v2.0.9 ' + # Runs the real cosign against the real bundle, and re-installs over the + # unverified semstat the steps above left in the job, which is the whole + # path a publish job asking to verify takes. + - name: A version answered from a signature-verified semstat + id: signed + uses: ./.github/actions/semver-validation + with: + version: v4.9.0 + verify-signature: true + - name: Assert the outputs env: STABLE_IS_VALID: ${{ steps.stable.outputs.is_valid }} @@ -122,6 +138,8 @@ jobs: PADDED_IS_VALID: ${{ steps.padded.outputs.is_valid }} PADDED_PARSED: ${{ steps.padded.outputs.parsed_version }} PADDED_COMPARISON: ${{ steps.padded.outputs.comparison }} + SIGNED_IS_VALID: ${{ steps.signed.outputs.is_valid }} + SIGNED_TYPE: ${{ steps.signed.outputs.release_type }} run: | set -euo pipefail @@ -180,4 +198,9 @@ jobs: '{"major":2,"minor":1,"patch":0,"prerelease":null,"build":null,"raw":"v2.1.0"}' assert "padded comparison" "$PADDED_COMPARISON" 1 + # Verifying the release changes how semstat was installed, not what it + # answers. + assert "signed is_valid" "$SIGNED_IS_VALID" true + assert "signed release_type" "$SIGNED_TYPE" stable + exit "$fail" diff --git a/README.md b/README.md index 9d900c39..d4ee7310 100644 --- a/README.md +++ b/README.md @@ -11,10 +11,10 @@ are written, tested, and structured. Reports on a version string: validity, its parts, its release channel, and how it orders against another version. Runs [`semstat`](https://github.com/loft-sh/semstat), -which it downloads and checksum-verifies at a pinned release, so the runner needs -egress to the semstat releases and `curl`, `tar`, `sha256sum` and `jq` on it. That -is new in `semver-validation/v4`; the tags before it (`v1`, `v2` and `v3`) point at -the self-contained Node action, which needed neither, and stay available. +installed by the same script as `setup-semstat` below, so the runner needs egress to +the semstat releases and `curl`, `tar`, `sha256sum` and `jq` on it. That is new in +`semver-validation/v4`; the tags before it (`v1`, `v2` and `v3`) point at the +self-contained Node action, which needed neither, and stay available. **Location:** `.github/actions/semver-validation` @@ -36,7 +36,8 @@ the self-contained Node action, which needed neither, and stay available. - `version` (required): Version string to validate - `compare_to` (optional): Second version to order `version` against -- `semstat_version` (optional): Release of `loft-sh/semstat` to download +- `verify-signature` (optional): cosign-verify the semstat release before + trusting it. Costs a cosign install, so off by default **Outputs:** @@ -51,6 +52,46 @@ the self-contained Node action, which needed neither, and stay available. See [semver-validation README](./.github/actions/semver-validation/README.md) for detailed documentation. +### Setup semstat Action + +Installs [`semstat`](https://github.com/loft-sh/semstat) from a pinned release, +checksum-verifies it, and puts it on `PATH`. This is where the installer and the +release pin live: the actions that answer semver questions run it rather than +carrying a copy of either. Actions in this repository run its script out of the +same checkout; workflows and other repositories use the action below. + +**Location:** `.github/actions/setup-semstat` + +**Usage:** + +```yaml +- name: Install semstat + id: semstat + uses: loft-sh/github-actions/.github/actions/setup-semstat@ # setup-semstat/v1 + with: + verify-signature: true # optional, off by default + +- name: Order two versions from a shell function + run: | + newer_than() { semstat gt "$1" "$2"; } + newer_than v4.9.0 v4.9.0-rc.2 +``` + +**Inputs:** + +- `version` (optional): Release of `loft-sh/semstat` to install. Empty takes the + Renovate-tracked pin in `src/install-semstat.sh` +- `verify-signature` (optional): cosign-verify `checksums.txt` against semstat's + release workflow at that exact tag. Costs a cosign install, so off by default + +**Outputs:** + +- `path`: Absolute path to the verified binary. The directory holding it is on + `PATH` for later steps as well, since consumers call semstat from inside shell + functions and loops where a step output is not in scope + +See [setup-semstat README](./.github/actions/setup-semstat/README.md) for detailed documentation. + ### Linear Release Sync Action Syncs Linear issues to the "Released" state when a GitHub release is published. Finds PRs between releases, extracts Linear issue IDs, and moves matching issues from "Ready for Release" to "Released". @@ -1006,6 +1047,7 @@ Each testable action has a dedicated workflow that runs its tests on PRs when the action's files change: - `test-semver-validation.yaml` - triggers on `.github/actions/semver-validation/**` +- `test-setup-semstat.yaml` - triggers on `.github/actions/setup-semstat/**` - `test-linear-pr-commenter.yaml` - triggers on `.github/actions/linear-pr-commenter/**` - `test-link-backport-prs.yaml` - triggers on `.github/actions/link-backport-prs/**` - `test-linear-release-sync.yaml` - triggers on `.github/actions/linear-release-sync/**` @@ -1144,6 +1186,24 @@ git tag -f action-name/v1 git push origin action-name/v1 --force ``` +These tags float, and callers pin them by name, so a force-push reaches every one +of those callers on their next run with no change on their side. That is the point +when shipping a fix, and the wrong tool when the new code asks something of the +caller that the old code did not: a runner requirement, network egress, a token, a +permission, a change to the job's environment. Cut the next major instead and leave +the existing tags where they are, so callers meet the new requirement when they +choose to move. + +`semver-validation` is the worked example. `v1`, `v2` and `v3` are the self-contained +Node implementation; the composite over `semstat` needs egress to the semstat +releases, needs `curl`/`tar`/`sha256sum`/`jq` on the runner, and puts `semstat` on +the job's `PATH`, so it went out as `v4` rather than over any of them. Before +force-pushing a tag, check who pins it: + +```bash +gh search code 'loft-sh/github-actions/.github/actions/@ org:loft-sh' +``` + ### Referencing Actions in Workflows ```yaml From 2af5d4b7ac01f8438c5e3bdb75c9720dad552ed6 Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Thu, 20 Aug 2026 17:24:27 +0200 Subject: [PATCH 03/10] docs: say when not to advance a floating action tag The release section taught `git tag -f` plus `--force` as the house idiom with no counterpart, which is the mistake this stack nearly shipped: advancing a floating tag reaches every caller pinned to it on their next run, with no version for them to notice. That is what you want for a fix and not for a change that adds a runner requirement, network egress, a token, a permission, or a write to the job environment. Records semver-validation as the worked example, and the gh search that answers who pins a tag before it is touched. --- CLAUDE.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/CLAUDE.md b/CLAUDE.md index 23feed1e..85d2fea1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -74,6 +74,18 @@ git push origin /v2 --force For `release-notification`, the `notify-release.yaml` wrapper and its inner composite both resolve at `@release-notification/v2` (see PR #147), so advancing that one tag moves the wrapper and composite together. +### The other half: do NOT advance a tag when the new code asks more of the caller + +Advancing is right for a fix and wrong for a change that adds a requirement, because it reaches every floating-tag caller with no change on their side and no version for them to notice. Cut the next major instead and leave the existing tags alone. Treat as a new major: a new runner requirement or tool, network egress, a token or permission, or anything that changes the caller's job environment (writing `GITHUB_PATH` or `GITHUB_ENV`). + +Check who pins the tag before touching it — the answer is often more repos than expected, and some pin the tag while others pin its SHA: + +```bash +gh search code 'loft-sh/github-actions/.github/actions/@ org:loft-sh' +``` + +`semver-validation` is the worked example (DEVOPS-1369). `v1`, `v2` and `v3` are the self-contained Node action; the rewrite as a composite over `semstat` needs egress, needs `curl`/`tar`/`sha256sum`/`jq`, and puts `semstat` on the caller's `PATH`, so it shipped as `v4`. Advancing `v1` or `v3` would have changed five live release workflows across five repos. + ### YAML-only / composite / Node.js actions (semver-validation, release-notification, etc.) - Update code and commit changes - Tag the release: `git tag -f /v1` From 1828e89d53810f429d656f9a1c493dac6699530f Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Thu, 20 Aug 2026 23:33:05 +0200 Subject: [PATCH 04/10] fix(setup-semstat): bind every name the installer reads, and name every failure Dropping the semstat_version input left SEMSTAT_VERSION unbound in semver-validation's install step. Composite steps inherit the job's environment, so a caller setting that name at workflow or job level, or an earlier step writing it to GITHUB_ENV, silently chose which semstat release the action installed. That is the same hole the SEMSTAT_BASE_URL clearing beside it exists to close, and it is now closed the same way, in both actions and in the recipe the README gives for running the script from a sibling. The bundle fetch reported every curl failure as a release that publishes no bundle, so a runner whose proxy allows github.com but not the objects.githubusercontent.com that release assets redirect to was told to re-dispatch semstat's release workflow, which repairs nothing it has. Only curl's HTTP-error status says the release answered and lacks the asset; anything else now names the transport. That download also had no timeouts, so a stalled connection hung the step until the job timed out rather than failing in 120s. The signature marker was the one write in the script with no ::error:: behind it, and a claim that does not land sends every later verifying step back to the network for the rest of the job. Both new paths are covered by tests that fail without the fix. The e2e jobs now run under a job environment that would repoint the installer if a binding were ever dropped again, which is the part bats cannot reach. --- .github/actions/semver-validation/action.yml | 15 +++-- .github/actions/setup-semstat/README.md | 12 ++-- .../setup-semstat/src/install-semstat.sh | 19 +++++-- .../setup-semstat/test/install-semstat.bats | 56 ++++++++++++++++--- .github/workflows/test-semver-validation.yaml | 20 +++++++ .github/workflows/test-setup-semstat.yaml | 20 +++++++ 6 files changed, 118 insertions(+), 24 deletions(-) diff --git a/.github/actions/semver-validation/action.yml b/.github/actions/semver-validation/action.yml index 647bec0e..d70fe6ad 100644 --- a/.github/actions/semver-validation/action.yml +++ b/.github/actions/semver-validation/action.yml @@ -82,12 +82,15 @@ runs: shell: bash env: SEMSTAT_VERIFY_SIGNATURE: ${{ inputs.verify-signature }} - # Composite steps inherit the job's environment, and the installer takes - # a file:// download root for its own tests. Cleared here, where step env - # wins, so a workflow-level env: or an earlier step writing GITHUB_ENV - # cannot repoint which executable this action runs: the archive and the - # checksums.txt it is verified against come from the same root, so a - # planted release verifies against its own planted manifest. + # Composite steps inherit the job's environment, and the installer reads + # both of these from it. Cleared here, where step env wins, so a + # workflow-level env: or an earlier step writing GITHUB_ENV cannot + # repoint which executable this action runs: the base URL because the + # archive and the checksums.txt it is verified against come from the same + # root, so a planted release verifies against its own planted manifest, + # and the version because this action ships one pinned release rather + # than whichever one the job asks for. + SEMSTAT_VERSION: "" SEMSTAT_BASE_URL: "" run: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh diff --git a/.github/actions/setup-semstat/README.md b/.github/actions/setup-semstat/README.md index e7d83de9..6bf46cd2 100644 --- a/.github/actions/setup-semstat/README.md +++ b/.github/actions/setup-semstat/README.md @@ -102,11 +102,12 @@ two ship from one commit and cannot drift apart. `semver-validation` is the worked example. The script takes `SEMSTAT_VERSION` and `SEMSTAT_VERIFY_SIGNATURE` from the -environment, both optional. Clear `SEMSTAT_BASE_URL` in the same `env:` — a -composite step inherits the job's environment, so a workflow-level `env:` would -otherwise repoint the download root, and step env is what wins. Verifying also -needs `cosign` on `PATH`, which is a `uses:` step and so cannot be lent out by a -script: +environment, both optional. Bind every variable the script reads in the same +`env:`, including the ones the action does not expose: a composite step inherits +the job's environment and step env is what wins, so a name left unbound lets a +workflow-level `env:` or an earlier `GITHUB_ENV` write repoint the download root +or the release that gets installed. Verifying also needs `cosign` on `PATH`, +which is a `uses:` step and so cannot be lent out by a script: ```yaml - name: Install cosign @@ -118,6 +119,7 @@ script: shell: bash env: SEMSTAT_VERIFY_SIGNATURE: ${{ inputs.verify-signature }} + SEMSTAT_VERSION: "" SEMSTAT_BASE_URL: "" run: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh ``` diff --git a/.github/actions/setup-semstat/src/install-semstat.sh b/.github/actions/setup-semstat/src/install-semstat.sh index 07afb18e..d0608878 100755 --- a/.github/actions/setup-semstat/src/install-semstat.sh +++ b/.github/actions/setup-semstat/src/install-semstat.sh @@ -225,9 +225,18 @@ done # outage there leaves the release published with the archives but no provenance. if [ "$verify_signature" = true ]; then bundle=checksums.txt.sigstore.json - if ! curl -fsSL --retry 3 --retry-connrefused -o "${work}/${bundle}" "${BASE_URL}/${tag}/${bundle}"; then + status=0 + curl -fsSL --retry 3 --retry-connrefused --connect-timeout 10 --max-time 120 -o "${work}/${bundle}" "${BASE_URL}/${tag}/${bundle}" || status=$? + # 22 is curl's HTTP-error status, so the release answered and does not carry + # the asset. Any other status is this runner not reaching it, and telling that + # runner to re-dispatch semstat's release workflow sends it after the wrong + # repair. + if [ "$status" -eq 22 ]; then echo "::error::semstat ${tag} publishes no ${bundle}, so its signature cannot be verified; the release may have been published through a signing outage, which a re-dispatch of its release workflow repairs" exit 1 + elif [ "$status" -ne 0 ]; then + echo "::error::could not download ${bundle} for semstat ${tag}: curl exited ${status}" + exit 1 fi fi @@ -328,9 +337,11 @@ if ! mv "${work}/semstat" "$binary" || ! sha256_of "$binary" >"$marker"; then fi # Written after the binary, so the marker never claims more than what is -# installed. -if [ "$verify_signature" = true ]; then - : >"$verified_marker" +# installed. A claim that could not be written has to fail the step: every later +# step asking to verify would otherwise re-download for the whole job. +if [ "$verify_signature" = true ] && ! : >"$verified_marker"; then + echo "::error::could not record that semstat ${tag} was signature-verified at ${verified_marker}" + exit 1 fi report_install diff --git a/.github/actions/setup-semstat/test/install-semstat.bats b/.github/actions/setup-semstat/test/install-semstat.bats index 64ea74b3..e2cf9936 100644 --- a/.github/actions/setup-semstat/test/install-semstat.bats +++ b/.github/actions/setup-semstat/test/install-semstat.bats @@ -342,7 +342,7 @@ MOCK # The archive and any half-unpacked binary go with it, on every exit path rather # than only the one that succeeded. -@test "leaves no work directory behind when the install fails" { +@test "leaves no work directory behind when the download unpacks the wrong version" { publish_release v1.2.4 1.2.3 export SEMSTAT_VERSION=v1.2.4 @@ -520,7 +520,7 @@ PLANTED [ -z "$(output_value semstat)" ] } -@test "a failed install leaves no work directory behind under RUNNER_TEMP" { +@test "leaves no work directory behind when the release does not exist" { export SEMSTAT_VERSION=v9.9.9 run "$SCRIPT" @@ -529,13 +529,6 @@ PLANTED [ -z "$(find "$RUNNER_TEMP" -maxdepth 1 -name 'semstat.*' -print -quit)" ] } -@test "a successful install leaves no work directory behind either" { - run "$SCRIPT" - - [ "$status" -eq 0 ] - [ -z "$(find "$RUNNER_TEMP" -maxdepth 1 -name 'semstat.*' -print -quit)" ] -} - # A PATH holding bash, so the shebang resolves, plus the named tools and nothing # else. Used to take one tool away from the gate at a time. gate_path() { @@ -638,6 +631,33 @@ gate_path() { [ -z "$(output_value semstat)" ] } +@test "a bundle that could not be fetched is not read as a release without one" { + export SEMSTAT_VERIFY_SIGNATURE=true + + # Fails the bundle alone and hands every other asset to the real mock, so the + # release is there and reachable and only this download is not. + farm="$TEST_DIR/no-bundle-egress" + mkdir -p "$farm" + cat >"$farm/curl" <"$(output_value semstat)" + + run "$SCRIPT" + + [ "$status" -eq 1 ] + [[ "$output" == *"::error::could not record that semstat v1.2.3 was signature-verified"* ]] +} + @test "reuses an install that was signature-verified" { export SEMSTAT_VERIFY_SIGNATURE=true diff --git a/.github/workflows/test-semver-validation.yaml b/.github/workflows/test-semver-validation.yaml index b2a016cd..1ffaebe2 100644 --- a/.github/workflows/test-semver-validation.yaml +++ b/.github/workflows/test-semver-validation.yaml @@ -41,6 +41,13 @@ jobs: name: Run the action end to end runs-on: ubuntu-latest timeout-minutes: 10 + # A job environment that would repoint the installer if the action left + # either name unbound: an older release, and a download root serving nothing. + # Composite steps inherit it, so the steps below answering at all is the + # assertion. bats covers the script; only a real job covers the binding. + env: + SEMSTAT_VERSION: v0.0.1 + SEMSTAT_BASE_URL: file:///nonexistent steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: @@ -105,6 +112,19 @@ jobs: version: v4.9.0 verify-signature: true + - name: Assert the job environment did not repoint the install + run: | + set -euo pipefail + + pinned="$(sed -n 's/^DEFAULT_VERSION=//p' \ + .github/actions/setup-semstat/src/install-semstat.sh)" + installed="$(semstat version)" + + if [ "${installed#v}" != "${pinned#v}" ]; then + echo "::error::expected the pinned ${pinned}, got ${installed}" + exit 1 + fi + - name: Assert the outputs env: STABLE_IS_VALID: ${{ steps.stable.outputs.is_valid }} diff --git a/.github/workflows/test-setup-semstat.yaml b/.github/workflows/test-setup-semstat.yaml index b96c524d..760d0114 100644 --- a/.github/workflows/test-setup-semstat.yaml +++ b/.github/workflows/test-setup-semstat.yaml @@ -33,6 +33,13 @@ jobs: action: name: Run the action end to end runs-on: ubuntu-latest + # A job environment that would repoint the installer if the action left + # either name unbound: an older release, and a download root serving nothing. + # Composite steps inherit it, so the steps below answering at all is the + # assertion. bats covers the script; only a real job covers the binding. + env: + SEMSTAT_VERSION: v0.0.1 + SEMSTAT_BASE_URL: file:///nonexistent steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: @@ -63,6 +70,19 @@ jobs: exit 1 fi + - name: Assert the job environment did not repoint the install + run: | + set -euo pipefail + + pinned="$(sed -n 's/^DEFAULT_VERSION=//p' \ + .github/actions/setup-semstat/src/install-semstat.sh)" + installed="$(semstat version)" + + if [ "${installed#v}" != "${pinned#v}" ]; then + echo "::error::expected the pinned ${pinned}, got ${installed}" + exit 1 + fi + - name: Call semstat through the path output env: SEMSTAT_BIN: ${{ steps.install.outputs.path }} From ac419a11810815d54737a9b6e28958d946629f37 Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Fri, 21 Aug 2026 00:15:47 +0200 Subject: [PATCH 05/10] fix(setup-semstat): guard the sibling script, install cosign once per job github.action_path reaches outside semver-validation's own directory, and a sparse single-action checkout died there on bash's bare complaint. The jq check the shared installer dropped moves to semver-validation, which is what still needs it. cosign-installer runs once per job rather than once per call, and the installer takes a PATH opt-out for callers that only ever name the binary. --- .github/actions/semver-validation/README.md | 16 ++--- .github/actions/semver-validation/action.yml | 59 +++++++++++++++- .github/actions/setup-semstat/README.md | 70 ++++++++++++------- .github/actions/setup-semstat/action.yml | 37 +++++++++- .../setup-semstat/src/install-semstat.sh | 31 ++++++-- .../setup-semstat/test/install-semstat.bats | 57 ++++++++++++++- .github/workflows/test-semver-validation.yaml | 55 +++++++++++++-- .github/workflows/test-setup-semstat.yaml | 33 +++++++-- 8 files changed, 305 insertions(+), 53 deletions(-) diff --git a/.github/actions/semver-validation/README.md b/.github/actions/semver-validation/README.md index 1f3c804d..f968f400 100644 --- a/.github/actions/semver-validation/README.md +++ b/.github/actions/semver-validation/README.md @@ -21,17 +21,17 @@ proxy allowlist that names only `github.com` fails the install step. `github.com/sigstore/cosign` releases and egress to `tuf-repo-cdn.sigstore.dev`, where cosign fetches the trusted root it checks the transparency log against. -Calling it also puts semstat on `PATH` for the steps that follow, because that is -how the installer it shares with [`setup-semstat`](../setup-semstat/README.md) -installs it. That is a side effect on the caller's job, so a job with its own -semstat on `PATH` should name it by absolute path rather than let step order decide -which one a later bare `semstat` resolves to. +Calling it leaves the caller's `PATH` alone. The installer it shares with +[`setup-semstat`](../setup-semstat/README.md) can put semstat there, and does for +callers that run it as a bare command, but this action names the binary by absolute +path and asks the installer to skip the append, so a job with its own semstat on +`PATH` keeps resolving to that one. All of that is new, so the rewrite ships as `semver-validation/v4`. The tags before it (`v1`, `v2` and `v3`) all point at the self-contained Node action, which needed -neither the network, nor those tools, nor anything of the caller's `PATH`. A caller -on a runner without egress keeps working on those tags and fails the install step on -`v4`, so all three stay where they are and none is advanced onto this rewrite. Live +neither the network nor those tools. A caller on a runner without egress keeps +working on those tags and fails the install step on `v4`, so all three stay where +they are and none is advanced onto this rewrite. Live callers pin `v1` and `v3` as floating tags, so advancing either would hand them the network and tool requirements with no version change to notice. diff --git a/.github/actions/semver-validation/action.yml b/.github/actions/semver-validation/action.yml index d70fe6ad..812ec155 100644 --- a/.github/actions/semver-validation/action.yml +++ b/.github/actions/semver-validation/action.yml @@ -64,8 +64,50 @@ outputs: runs: using: "composite" steps: - - name: Install cosign + # jq belongs to the reporting step below, but a caller cannot run one step of + # this composite without the other, and saying so before the downloads is + # what keeps a runner missing jq from paying for tools it cannot use. + # report.sh checks again, for the runs that reach it directly. + - name: Check the runner can read semstat's output + shell: bash + run: | + if ! command -v jq >/dev/null 2>&1; then + echo "::error::jq is required to read semstat's output and is not on PATH" + exit 1 + fi + + # cosign-installer re-downloads its bootstrap binary and re-appends to + # GITHUB_PATH on every call, so a job that reports on several versions pays + # for it once per call while every semstat install after the first is a cache + # hit that never reaches cosign. The marker lives under RUNNER_TEMP and so is + # private to this job, which is what makes this skip an install this job + # already did rather than adopt whatever cosign a runner image shipped. + # + # Recorded before the install rather than after: a cosign-installer that + # fails takes the job with it, so nothing reads the marker afterwards, and a + # caller that swallows that failure meets the installer's own "needs cosign + # on PATH" check on the next call rather than a silent unverified install. + - name: Check whether this job still needs cosign + id: cosign if: inputs.verify-signature == 'true' + shell: bash + env: + COSIGN_MARKER: ${{ runner.temp }}/.setup-semstat-cosign-installed + run: | + install=true + if [ -f "$COSIGN_MARKER" ]; then + install=false + elif ! : >"$COSIGN_MARKER"; then + echo "::error::could not record that this job installs cosign at ${COSIGN_MARKER}" + exit 1 + fi + if ! echo "install=${install}" >>"$GITHUB_OUTPUT"; then + echo "::error::could not write whether this job needs cosign to GITHUB_OUTPUT" + exit 1 + fi + + - name: Install cosign + if: inputs.verify-signature == 'true' && steps.cosign.outputs.install == 'true' uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2 # setup-semstat's installer, run out of the checkout this action came in @@ -92,7 +134,20 @@ runs: # than whichever one the job asks for. SEMSTAT_VERSION: "" SEMSTAT_BASE_URL: "" - run: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh + # The reporting step names the binary through SEMSTAT_BIN and never runs + # a bare `semstat`, so the installer's PATH append would be a change to + # the caller's job with nothing here reading it. + SEMSTAT_SKIP_PATH: "true" + INSTALLER: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh + run: | + # A checkout that sparsely took only this action's directory leaves the + # sibling installer absent, and bash's own "No such file or directory" + # names neither what is missing nor what the checkout has to include. + if [ ! -x "$INSTALLER" ]; then + echo "::error::${INSTALLER} is missing or not executable; this action runs the installer that ships with the sibling setup-semstat action, so a sparse checkout has to take .github/actions/setup-semstat alongside .github/actions/semver-validation" + exit 1 + fi + "$INSTALLER" - name: Report on the version id: report diff --git a/.github/actions/setup-semstat/README.md b/.github/actions/setup-semstat/README.md index 6bf46cd2..ef6d4fb5 100644 --- a/.github/actions/setup-semstat/README.md +++ b/.github/actions/setup-semstat/README.md @@ -81,13 +81,30 @@ a consumer only when its SHA is bumped, so releasing this action means advancing ### From another action in this repository -Run the script instead of pinning the action: +Run the script instead of pinning the action. Copy this whole shape rather than +the one-line `run:` it reduces to: the `env:` block and the existence check are +both load-bearing, for the reasons below. ```yaml +- name: Install cosign + if: inputs.verify-signature == 'true' + uses: sigstore/cosign-installer@ # v4.1.2 + - name: Install semstat id: install shell: bash - run: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh + env: + SEMSTAT_VERIFY_SIGNATURE: ${{ inputs.verify-signature }} + SEMSTAT_VERSION: "" + SEMSTAT_BASE_URL: "" + SEMSTAT_SKIP_PATH: "" + INSTALLER: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh + run: | + if [ ! -x "$INSTALLER" ]; then + echo "::error::${INSTALLER} is missing or not executable; this action runs the installer that ships with the sibling setup-semstat action, so a sparse checkout has to take .github/actions/setup-semstat as well" + exit 1 + fi + "$INSTALLER" ``` Neither form of `uses:` works for a sibling in the same repository. A relative @@ -101,28 +118,29 @@ works. `github.action_path` is inside a full checkout of this repository, so the two ship from one commit and cannot drift apart. `semver-validation` is the worked example. -The script takes `SEMSTAT_VERSION` and `SEMSTAT_VERIFY_SIGNATURE` from the -environment, both optional. Bind every variable the script reads in the same -`env:`, including the ones the action does not expose: a composite step inherits -the job's environment and step env is what wins, so a name left unbound lets a -workflow-level `env:` or an earlier `GITHUB_ENV` write repoint the download root -or the release that gets installed. Verifying also needs `cosign` on `PATH`, -which is a `uses:` step and so cannot be lent out by a script: - -```yaml -- name: Install cosign - if: inputs.verify-signature == 'true' - uses: sigstore/cosign-installer@ # v4.1.2 - -- name: Install semstat - id: install - shell: bash - env: - SEMSTAT_VERIFY_SIGNATURE: ${{ inputs.verify-signature }} - SEMSTAT_VERSION: "" - SEMSTAT_BASE_URL: "" - run: ${{ github.action_path }}/../setup-semstat/src/install-semstat.sh -``` +Bind every variable the script reads in that same `env:`, including the ones the +calling action does not expose: a composite step inherits the job's environment +and step env is what wins, so a name left unbound lets a workflow-level `env:` or +an earlier `GITHUB_ENV` write repoint the download root, the release that gets +installed, or whether semstat lands on `PATH` at all. All four are optional to the +script and every one of them is worth binding. + +Check the script is there before running it. `github.action_path` reaches outside +the calling action's own directory, and this repository's workflows do sparse +single-action checkouts, so a caller that took only its own action directory +otherwise dies on bash's bare `No such file or directory` with no `::error::` line +naming what the checkout has to include. + +Verifying also needs `cosign` on `PATH`, which is a `uses:` step and so cannot be +lent out by a script. If the calling action can be invoked several times in one +job, guard that step the way `semver-validation` does: cosign-installer +re-downloads its bootstrap binary on every call, while every semstat install after +the first is a cache hit that never reaches cosign. + +Set `SEMSTAT_SKIP_PATH: "true"` where the calling action names the binary through +the step output and never runs a bare `semstat`, because the append is then a +change to the caller's job with nothing reading it. `semver-validation` does +exactly this. The step output is `semstat` rather than this action's `path` output, because the script writes it directly: `${{ steps.install.outputs.semstat }}`. @@ -179,7 +197,9 @@ left behind rather than trusting it. The runner *prepends* `GITHUB_PATH` entries, so where two steps install different semstat releases into the same job, a later bare `semstat` resolves to whichever installed last. Anything that cares which release it is talking to should name the -`path` output instead of relying on that order. +`path` output instead of relying on that order; an in-repository caller that only +ever names the output can set `SEMSTAT_SKIP_PATH` and leave the job's `PATH` out +of it altogether. ## Upgrading the pinned release diff --git a/.github/actions/setup-semstat/action.yml b/.github/actions/setup-semstat/action.yml index 1be5811c..20ca0d3b 100644 --- a/.github/actions/setup-semstat/action.yml +++ b/.github/actions/setup-semstat/action.yml @@ -30,8 +30,38 @@ outputs: runs: using: "composite" steps: - - name: Install cosign + # cosign-installer re-downloads its bootstrap binary and re-appends to + # GITHUB_PATH on every call, so a job that installs semstat in several steps + # pays for it once per step while every install after the first is a cache + # hit that never reaches cosign. The marker lives under RUNNER_TEMP and so is + # private to this job, which is what makes this skip an install this job + # already did rather than adopt whatever cosign a runner image shipped. + # + # Recorded before the install rather than after: a cosign-installer that + # fails takes the job with it, so nothing reads the marker afterwards, and a + # caller that swallows that failure meets the installer's own "needs cosign + # on PATH" check on the next call rather than a silent unverified install. + - name: Check whether this job still needs cosign + id: cosign if: inputs.verify-signature == 'true' + shell: bash + env: + COSIGN_MARKER: ${{ runner.temp }}/.setup-semstat-cosign-installed + run: | + install=true + if [ -f "$COSIGN_MARKER" ]; then + install=false + elif ! : >"$COSIGN_MARKER"; then + echo "::error::could not record that this job installs cosign at ${COSIGN_MARKER}" + exit 1 + fi + if ! echo "install=${install}" >>"$GITHUB_OUTPUT"; then + echo "::error::could not write whether this job needs cosign to GITHUB_OUTPUT" + exit 1 + fi + + - name: Install cosign + if: inputs.verify-signature == 'true' && steps.cosign.outputs.install == 'true' uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2 - name: Install semstat @@ -45,6 +75,9 @@ runs: # wins, so a workflow-level env: or an earlier step writing GITHUB_ENV # cannot repoint which executable this action runs: the archive and the # checksums.txt it is verified against come from the same root, so a - # planted release verifies against its own planted manifest. + # planted release verifies against its own planted manifest, and the + # PATH opt-out because putting semstat on PATH is this action's + # contract, not something a job's environment gets to withdraw. SEMSTAT_BASE_URL: "" + SEMSTAT_SKIP_PATH: "" run: ${{ github.action_path }}/src/install-semstat.sh diff --git a/.github/actions/setup-semstat/src/install-semstat.sh b/.github/actions/setup-semstat/src/install-semstat.sh index d0608878..0f3abb59 100755 --- a/.github/actions/setup-semstat/src/install-semstat.sh +++ b/.github/actions/setup-semstat/src/install-semstat.sh @@ -6,10 +6,10 @@ # # semstat absolute path to the verified, executable binary # -# and appends the directory holding it to GITHUB_PATH, because every consumer so -# far calls semstat from inside a bash function or a `while read` loop, where a -# step output is not in scope. The output is kept as well, for a caller that -# would rather name the binary than depend on PATH ordering. +# and, unless asked not to, appends the directory holding it to GITHUB_PATH, +# because most consumers call semstat from inside a bash function or a `while +# read` loop, where a step output is not in scope. The output is kept as well, +# for a caller that would rather name the binary than depend on PATH ordering. # # Required environment: # GITHUB_OUTPUT standard GitHub Actions step output file. @@ -23,6 +23,9 @@ # release workflow before trusting it. Needs cosign # on PATH; the action installs it when asked. # SEMSTAT_BASE_URL file:// release download root. For the tests; see below. +# SEMSTAT_SKIP_PATH "true" to report the binary through the step output only +# and leave GITHUB_PATH alone. For a caller that names the +# binary and has no use for a change to the job's PATH. set -euo pipefail : "${GITHUB_OUTPUT:?GITHUB_OUTPUT is required}" @@ -107,6 +110,18 @@ case "${SEMSTAT_VERIFY_SIGNATURE:-false}" in ;; esac +# Read the same way as verify-signature, and for the same reason: a typo taken +# for a no would leave the job's PATH changed under a caller that asked for it +# not to be. +case "${SEMSTAT_SKIP_PATH:-false}" in + true) skip_path=true ;; + false) skip_path=false ;; + *) + echo "::error::SEMSTAT_SKIP_PATH takes true or false; got $(fold_lines "$SEMSTAT_SKIP_PATH")" + exit 1 + ;; +esac + requested="${SEMSTAT_VERSION:-$DEFAULT_VERSION}" case "$requested" in v*) tag="$requested" ;; @@ -171,6 +186,9 @@ report_install() { echo "::error::could not write the semstat path to GITHUB_OUTPUT" exit 1 fi + if [ "$skip_path" = true ]; then + return + fi if ! printf '%s\n' "$install_dir" >>"$GITHUB_PATH"; then echo "::error::could not add ${install_dir} to GITHUB_PATH" exit 1 @@ -325,8 +343,9 @@ fi # replaced the binary but not the marker, so an unverified download sits under an # earlier run's claim and the reuse check above hands it to a step that asked to # verify. -if [ "$verify_signature" = false ]; then - rm -f "$verified_marker" +if [ "$verify_signature" = false ] && ! rm -f "$verified_marker"; then + echo "::error::could not drop the earlier signature claim at ${verified_marker}, which would leave this unverified install standing under it for a later step asking to verify" + exit 1 fi if ! mv "${work}/semstat" "$binary" || ! sha256_of "$binary" >"$marker"; then diff --git a/.github/actions/setup-semstat/test/install-semstat.bats b/.github/actions/setup-semstat/test/install-semstat.bats index e2cf9936..f2f9320b 100644 --- a/.github/actions/setup-semstat/test/install-semstat.bats +++ b/.github/actions/setup-semstat/test/install-semstat.bats @@ -596,6 +596,41 @@ gate_path() { [ "$(path_entries)" = "$installed" ] } +# semver-validation runs semstat through SEMSTAT_BIN and never as a bare +# `semstat`, so for it the append is a change to the caller's job with nothing +# reading it. +@test "leaves PATH alone when asked to" { + export SEMSTAT_SKIP_PATH=true + + run "$SCRIPT" + + [ "$status" -eq 0 ] + [ -x "$(output_value semstat)" ] + [ ! -s "$GITHUB_PATH" ] +} + +@test "leaves PATH alone when a reused install is reported" { + export SEMSTAT_SKIP_PATH=true + run "$SCRIPT" + [ "$status" -eq 0 ] + + run "$SCRIPT" + + [ "$status" -eq 0 ] + [[ "$output" == *"already installed"* ]] + [ ! -s "$GITHUB_PATH" ] +} + +@test "refuses a skip-path value that is neither true nor false" { + export SEMSTAT_SKIP_PATH=yes + + run "$SCRIPT" + + [ "$status" -eq 1 ] + [[ "$output" == *"SEMSTAT_SKIP_PATH takes true or false; got yes"* ]] + [ ! -s "$CURL_URLS" ] +} + @test "leaves the signature alone unless asked" { run "$SCRIPT" @@ -627,7 +662,7 @@ gate_path() { [ "$status" -eq 1 ] [[ "$output" == *"publishes no checksums.txt.sigstore.json"* ]] - [[ "$output" != *"check that the release exists"* ]] + [[ "$output" != *"could not download checksums.txt.sigstore.json"* ]] [ -z "$(output_value semstat)" ] } @@ -744,6 +779,26 @@ MOCK [[ "$output" == *"::error::could not record that semstat v1.2.3 was signature-verified"* ]] } +# The reverse of the test above: an unverified run that cannot clear the earlier +# claim has to fail rather than install under it, because the reuse check reads +# the claim and would hand this download to a step that asked to verify. +@test "a stale signature claim that cannot be dropped fails the step" { + run "$SCRIPT" + [ "$status" -eq 0 ] + + # A directory where the marker file goes: rm cannot remove it. + dir="$(dirname "$(output_value semstat)")" + mkdir "$dir/.signature-verified" + + # Only running it says the install is unusable, so this forces a re-install. + printf 'truncated' >"$(output_value semstat)" + + run "$SCRIPT" + + [ "$status" -eq 1 ] + [[ "$output" == *"::error::could not drop the earlier signature claim"* ]] +} + @test "reuses an install that was signature-verified" { export SEMSTAT_VERIFY_SIGNATURE=true diff --git a/.github/workflows/test-semver-validation.yaml b/.github/workflows/test-semver-validation.yaml index 1ffaebe2..73b3f8e8 100644 --- a/.github/workflows/test-semver-validation.yaml +++ b/.github/workflows/test-semver-validation.yaml @@ -41,13 +41,15 @@ jobs: name: Run the action end to end runs-on: ubuntu-latest timeout-minutes: 10 - # A job environment that would repoint the installer if the action left - # either name unbound: an older release, and a download root serving nothing. - # Composite steps inherit it, so the steps below answering at all is the - # assertion. bats covers the script; only a real job covers the binding. + # A job environment that would repoint the installer if the action left any + # of these unbound: an older release, a download root serving nothing, and a + # demand that semstat land on the job's PATH after all. Composite steps + # inherit it, so the steps below answering at all is the assertion. bats + # covers the script; only a real job covers the binding. env: SEMSTAT_VERSION: v0.0.1 SEMSTAT_BASE_URL: file:///nonexistent + SEMSTAT_SKIP_PATH: "false" steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: @@ -112,19 +114,62 @@ jobs: version: v4.9.0 verify-signature: true + # The second verified call is where the cosign install is skipped, the + # first having recorded that this job already ran cosign-installer. + # Verification still has to answer, which is what says the skip left + # cosign reachable for it. + - name: A second version answered from the verified semstat + id: signed_again + uses: ./.github/actions/semver-validation + with: + version: v4.9.1 + verify-signature: true + + - name: Assert the second verified call answered + env: + IS_VALID: ${{ steps.signed_again.outputs.is_valid }} + run: | + set -euo pipefail + + if [ "$IS_VALID" != "true" ]; then + echo "::error::the second verify-signature call reported is_valid='${IS_VALID}'" + exit 1 + fi + + # Read off the install directory rather than off PATH, because this action + # asks the installer to leave PATH alone. Exactly one install also says the + # job environment's v0.0.1 never got downloaded, which a PATH lookup could + # not have shown: it only ever names whichever landed last. - name: Assert the job environment did not repoint the install run: | set -euo pipefail pinned="$(sed -n 's/^DEFAULT_VERSION=//p' \ .github/actions/setup-semstat/src/install-semstat.sh)" - installed="$(semstat version)" + shopt -s nullglob + installs=("$RUNNER_TEMP"/semstat-*/semstat) + + if [ "${#installs[@]}" -ne 1 ]; then + echo "::error::expected one semstat install under RUNNER_TEMP, found ${#installs[@]}: ${installs[*]}" + exit 1 + fi + + installed="$("${installs[0]}" version)" if [ "${installed#v}" != "${pinned#v}" ]; then echo "::error::expected the pinned ${pinned}, got ${installed}" exit 1 fi + - name: Assert the action left the job's PATH alone + run: | + set -euo pipefail + + if command -v semstat >/dev/null 2>&1; then + echo "::error::semstat is on this job's PATH; this action names the binary and asks the installer to skip the append" + exit 1 + fi + - name: Assert the outputs env: STABLE_IS_VALID: ${{ steps.stable.outputs.is_valid }} diff --git a/.github/workflows/test-setup-semstat.yaml b/.github/workflows/test-setup-semstat.yaml index 760d0114..7dd60f3e 100644 --- a/.github/workflows/test-setup-semstat.yaml +++ b/.github/workflows/test-setup-semstat.yaml @@ -33,13 +33,15 @@ jobs: action: name: Run the action end to end runs-on: ubuntu-latest - # A job environment that would repoint the installer if the action left - # either name unbound: an older release, and a download root serving nothing. - # Composite steps inherit it, so the steps below answering at all is the - # assertion. bats covers the script; only a real job covers the binding. + # A job environment that would repoint the installer if the action left any + # of these unbound: an older release, a download root serving nothing, and a + # demand that semstat stay off the job's PATH. Composite steps inherit it, so + # the steps below answering at all is the assertion. bats covers the script; + # only a real job covers the binding. env: SEMSTAT_VERSION: v0.0.1 SEMSTAT_BASE_URL: file:///nonexistent + SEMSTAT_SKIP_PATH: "true" steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: @@ -159,6 +161,29 @@ jobs: "$SEMSTAT_BIN" validate v4.9.0 + # The second call is where the cosign install is skipped: the first one + # recorded that this job already ran cosign-installer. Verification still + # has to answer, which is what says the skip left cosign reachable for it. + - name: Install again with signature verification + id: reverify + uses: ./.github/actions/setup-semstat + with: + verify-signature: true + + - name: Assert the second verified install answered too + env: + FIRST: ${{ steps.install.outputs.path }} + SECOND: ${{ steps.reverify.outputs.path }} + run: | + set -euo pipefail + + if [ "$SECOND" != "$FIRST" ]; then + echo "::error::the second install reported '$SECOND', expected '$FIRST'" + exit 1 + fi + + "$SECOND" validate v4.9.0 + - name: Reject a verify-signature value that is not a boolean id: bad_input continue-on-error: true From c571504b8f44673dd3a50aca695f36d8d76147ca Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Fri, 21 Aug 2026 11:00:59 +0200 Subject: [PATCH 06/10] fix(setup-semstat): make the cosign-skip assertions actually reach cosign Both workflows claimed a second verified install proved the skipped cosign-installer left cosign reachable. It proved nothing: the reuse fast path exits before the `command -v cosign` gate and before verify-blob, so the assertion held even if the skip had left no cosign at all. Drop the semstat install between the two calls, keep the cosign marker, and the second call re-downloads and verifies with cosign-installer skipped. Also corrects the recorded reason for cutting semver-validation/v4: it binds SEMSTAT_SKIP_PATH and leaves the caller's PATH alone, so "puts semstat on the caller's PATH" was never one of the reasons. --- .github/actions/semver-validation/action.yml | 16 ++++++++- .github/actions/setup-semstat/README.md | 35 ++++++++++++++++--- .github/actions/setup-semstat/action.yml | 6 ++++ .github/workflows/test-semver-validation.yaml | 32 ++++++++++++++--- .github/workflows/test-setup-semstat.yaml | 35 +++++++++++++++++-- CLAUDE.md | 2 +- README.md | 7 ++-- 7 files changed, 116 insertions(+), 17 deletions(-) diff --git a/.github/actions/semver-validation/action.yml b/.github/actions/semver-validation/action.yml index 812ec155..fecd2205 100644 --- a/.github/actions/semver-validation/action.yml +++ b/.github/actions/semver-validation/action.yml @@ -68,10 +68,18 @@ runs: # this composite without the other, and saying so before the downloads is # what keeps a runner missing jq from paying for tools it cannot use. # report.sh checks again, for the runs that reach it directly. + # + # Held back when `version:` is absent, so report.sh stays the one that + # answers that case: it checks the missing input first, deliberately above + # its own jq check, and emits the outputs alongside the error. Failing here + # instead would swap that report for a jq message and leave the outputs + # empty. The two checks have to keep agreeing about which comes first. - name: Check the runner can read semstat's output shell: bash + env: + INPUT_VERSION: ${{ inputs.version }} run: | - if ! command -v jq >/dev/null 2>&1; then + if [ -n "${INPUT_VERSION-}" ] && ! command -v jq >/dev/null 2>&1; then echo "::error::jq is required to read semstat's output and is not on PATH" exit 1 fi @@ -87,6 +95,12 @@ runs: # fails takes the job with it, so nothing reads the marker afterwards, and a # caller that swallows that failure meets the installer's own "needs cosign # on PATH" check on the next call rather than a silent unverified install. + # + # This gate is duplicated in setup-semstat/action.yml, because a composite + # cannot share steps with a sibling. The two copies coordinate through the + # marker path below, so a job mixing both actions only installs cosign once: + # change the path or the cosign-installer pin in one and the pair silently + # stops agreeing. Edit them together. - name: Check whether this job still needs cosign id: cosign if: inputs.verify-signature == 'true' diff --git a/.github/actions/setup-semstat/README.md b/.github/actions/setup-semstat/README.md index ef6d4fb5..aab98f9c 100644 --- a/.github/actions/setup-semstat/README.md +++ b/.github/actions/setup-semstat/README.md @@ -86,8 +86,27 @@ the one-line `run:` it reduces to: the `env:` block and the existence check are both load-bearing, for the reasons below. ```yaml -- name: Install cosign +- name: Check whether this job still needs cosign + id: cosign if: inputs.verify-signature == 'true' + shell: bash + env: + COSIGN_MARKER: ${{ runner.temp }}/.setup-semstat-cosign-installed + run: | + install=true + if [ -f "$COSIGN_MARKER" ]; then + install=false + elif ! : >"$COSIGN_MARKER"; then + echo "::error::could not record that this job installs cosign at ${COSIGN_MARKER}" + exit 1 + fi + if ! echo "install=${install}" >>"$GITHUB_OUTPUT"; then + echo "::error::could not write whether this job needs cosign to GITHUB_OUTPUT" + exit 1 + fi + +- name: Install cosign + if: inputs.verify-signature == 'true' && steps.cosign.outputs.install == 'true' uses: sigstore/cosign-installer@ # v4.1.2 - name: Install semstat @@ -132,10 +151,16 @@ otherwise dies on bash's bare `No such file or directory` with no `::error::` li naming what the checkout has to include. Verifying also needs `cosign` on `PATH`, which is a `uses:` step and so cannot be -lent out by a script. If the calling action can be invoked several times in one -job, guard that step the way `semver-validation` does: cosign-installer -re-downloads its bootstrap binary on every call, while every semstat install after -the first is a cache hit that never reaches cosign. +lent out by a script. That is why the marker step above sits in front of it: +cosign-installer re-downloads its bootstrap binary on every call, while every +semstat install after the first is a cache hit that never reaches cosign, so an +action invoked several times in one job would pay for a bootstrap it does not use. +The marker lives under `RUNNER_TEMP`, private to the job, which is what makes the +skip reuse an install this job already did rather than adopt whatever `cosign` a +runner image happened to ship. Keep the marker path byte-identical to the one in +`setup-semstat/action.yml` and `semver-validation/action.yml`, since that shared +path is how a job mixing these actions installs cosign only once. Drop the step +only if the calling action can run at most once per job. Set `SEMSTAT_SKIP_PATH: "true"` where the calling action names the binary through the step output and never runs a bare `semstat`, because the append is then a diff --git a/.github/actions/setup-semstat/action.yml b/.github/actions/setup-semstat/action.yml index 20ca0d3b..c2b5ebd7 100644 --- a/.github/actions/setup-semstat/action.yml +++ b/.github/actions/setup-semstat/action.yml @@ -41,6 +41,12 @@ runs: # fails takes the job with it, so nothing reads the marker afterwards, and a # caller that swallows that failure meets the installer's own "needs cosign # on PATH" check on the next call rather than a silent unverified install. + # + # This gate is duplicated in semver-validation/action.yml, because a + # composite cannot share steps with a sibling. The two copies coordinate + # through the marker path below, so a job mixing both actions only installs + # cosign once: change the path or the cosign-installer pin in one and the + # pair silently stops agreeing. Edit them together. - name: Check whether this job still needs cosign id: cosign if: inputs.verify-signature == 'true' diff --git a/.github/workflows/test-semver-validation.yaml b/.github/workflows/test-semver-validation.yaml index 73b3f8e8..becfafea 100644 --- a/.github/workflows/test-semver-validation.yaml +++ b/.github/workflows/test-semver-validation.yaml @@ -114,10 +114,34 @@ jobs: version: v4.9.0 verify-signature: true - # The second verified call is where the cosign install is skipped, the - # first having recorded that this job already ran cosign-installer. - # Verification still has to answer, which is what says the skip left - # cosign reachable for it. + # Two versions are two inputs to this action, not two semstat releases, so + # the call below would otherwise reuse the install the step above verified + # and exit before the cosign gate. Dropping the install is what forces it + # to verify again; keeping the cosign marker is what makes it do so with + # cosign-installer skipped, which is the pairing under test. + - name: Drop the semstat install, keep the cosign marker + env: + COSIGN_MARKER: ${{ runner.temp }}/.setup-semstat-cosign-installed + run: | + set -euo pipefail + + if [ ! -f "$COSIGN_MARKER" ]; then + echo "::error::the verified call left no cosign marker at ${COSIGN_MARKER}, so the call below would install cosign again rather than exercise the skip" + exit 1 + fi + + rm -rf "${RUNNER_TEMP:?}"/semstat-* + + shopt -s nullglob + remaining=("$RUNNER_TEMP"/semstat-*) + + if [ "${#remaining[@]}" -ne 0 ]; then + echo "::error::a semstat install survived under ${RUNNER_TEMP}, so the next call would take the reuse path instead of verifying: ${remaining[*]}" + exit 1 + fi + + # Re-installs with cosign-installer skipped. If the skip left cosign + # unreachable the installer's own gate fails this step. - name: A second version answered from the verified semstat id: signed_again uses: ./.github/actions/semver-validation diff --git a/.github/workflows/test-setup-semstat.yaml b/.github/workflows/test-setup-semstat.yaml index 7dd60f3e..c00bc648 100644 --- a/.github/workflows/test-setup-semstat.yaml +++ b/.github/workflows/test-setup-semstat.yaml @@ -161,9 +161,38 @@ jobs: "$SEMSTAT_BIN" validate v4.9.0 - # The second call is where the cosign install is skipped: the first one - # recorded that this job already ran cosign-installer. Verification still - # has to answer, which is what says the skip left cosign reachable for it. + # Reaching the cosign gate a second time takes deleting the install: the + # reuse fast path exits before both the `command -v cosign` check and + # verify-blob, so a second call left to hit the cache proves only that the + # cache holds, and would stay green even if the skipped install had left no + # cosign at all. The cosign marker is deliberately kept, because that is + # what makes the call below skip cosign-installer and still have to verify. + - name: Drop the semstat install, keep the cosign marker + env: + COSIGN_MARKER: ${{ runner.temp }}/.setup-semstat-cosign-installed + run: | + set -euo pipefail + + if [ ! -f "$COSIGN_MARKER" ]; then + echo "::error::the first verified install left no cosign marker at ${COSIGN_MARKER}, so the call below would install cosign again rather than exercise the skip" + exit 1 + fi + + rm -rf "${RUNNER_TEMP:?}"/semstat-* + + # One surviving directory would put the next call back on the fast path + # and quietly restore the hole this step exists to close. + shopt -s nullglob + remaining=("$RUNNER_TEMP"/semstat-*) + + if [ "${#remaining[@]}" -ne 0 ]; then + echo "::error::a semstat install survived under ${RUNNER_TEMP}, so the next call would take the reuse path instead of verifying: ${remaining[*]}" + exit 1 + fi + + # A real install with cosign-installer skipped: it re-downloads, and the + # installer's own "verify-signature needs cosign on PATH" gate fails the + # step if the skip left cosign unreachable. That is the assertion. - name: Install again with signature verification id: reverify uses: ./.github/actions/setup-semstat diff --git a/CLAUDE.md b/CLAUDE.md index 85d2fea1..32393c8b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -84,7 +84,7 @@ Check who pins the tag before touching it — the answer is often more repos tha gh search code 'loft-sh/github-actions/.github/actions/@ org:loft-sh' ``` -`semver-validation` is the worked example (DEVOPS-1369). `v1`, `v2` and `v3` are the self-contained Node action; the rewrite as a composite over `semstat` needs egress, needs `curl`/`tar`/`sha256sum`/`jq`, and puts `semstat` on the caller's `PATH`, so it shipped as `v4`. Advancing `v1` or `v3` would have changed five live release workflows across five repos. +`semver-validation` is the worked example (DEVOPS-1369). `v1`, `v2` and `v3` are the self-contained Node action; the rewrite as a composite over `semstat` needs egress, needs `curl`/`tar`/`sha256sum`/`jq`, and drops the `semstat_version` input, so it shipped as `v4`. Advancing `v1` or `v3` would have changed five live release workflows across five repos. It does not touch the caller's `PATH`, which would have been a fourth reason on its own: it binds `SEMSTAT_SKIP_PATH` precisely so the rewrite stays invisible to the job around it. ### YAML-only / composite / Node.js actions (semver-validation, release-notification, etc.) - Update code and commit changes diff --git a/README.md b/README.md index d4ee7310..7a1026f0 100644 --- a/README.md +++ b/README.md @@ -1196,9 +1196,10 @@ choose to move. `semver-validation` is the worked example. `v1`, `v2` and `v3` are the self-contained Node implementation; the composite over `semstat` needs egress to the semstat -releases, needs `curl`/`tar`/`sha256sum`/`jq` on the runner, and puts `semstat` on -the job's `PATH`, so it went out as `v4` rather than over any of them. Before -force-pushing a tag, check who pins it: +releases, needs `curl`/`tar`/`sha256sum`/`jq` on the runner, and drops the +`semstat_version` input, so it went out as `v4` rather than over any of them. It +leaves the job's `PATH` alone, which would otherwise have been a reason on its +own. Before force-pushing a tag, check who pins it: ```bash gh search code 'loft-sh/github-actions/.github/actions/@ org:loft-sh' From 5e51c5b1adecf949a4d9be80c1a0f5e33391708d Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Fri, 21 Aug 2026 11:02:32 +0200 Subject: [PATCH 07/10] fix(setup-semstat): stop sending callers after a repair they cannot make The absent-bundle error told the reader a re-dispatch of semstat's release workflow repairs the release. That is semstat's repair, not something a consumer of this action can reach, and it stops working outright once a release is cut under immutability. Name a release that carries a bundle instead, which is advice the reader can act on either way. --- .../actions/setup-semstat/src/install-semstat.sh | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/.github/actions/setup-semstat/src/install-semstat.sh b/.github/actions/setup-semstat/src/install-semstat.sh index 0f3abb59..2f927f83 100755 --- a/.github/actions/setup-semstat/src/install-semstat.sh +++ b/.github/actions/setup-semstat/src/install-semstat.sh @@ -241,16 +241,21 @@ done # different failure than an absent release and must not be reported as one. # semstat signs at the end of goreleaser's publish phase, so a Fulcio or Rekor # outage there leaves the release published with the archives but no provenance. +# +# The message below names a different release rather than a re-dispatch of +# semstat's release workflow. Re-releasing a tag is semstat's repair and not +# something the reader of this error can reach, and it stops working outright on +# a release cut under immutability, so it is the wrong thing to send anyone after. if [ "$verify_signature" = true ]; then bundle=checksums.txt.sigstore.json status=0 curl -fsSL --retry 3 --retry-connrefused --connect-timeout 10 --max-time 120 -o "${work}/${bundle}" "${BASE_URL}/${tag}/${bundle}" || status=$? # 22 is curl's HTTP-error status, so the release answered and does not carry - # the asset. Any other status is this runner not reaching it, and telling that - # runner to re-dispatch semstat's release workflow sends it after the wrong - # repair. + # the asset. Any other status is this runner not reaching it, which is a + # different problem than a release without provenance and must not be reported + # as one. if [ "$status" -eq 22 ]; then - echo "::error::semstat ${tag} publishes no ${bundle}, so its signature cannot be verified; the release may have been published through a signing outage, which a re-dispatch of its release workflow repairs" + echo "::error::semstat ${tag} publishes no ${bundle}, so its signature cannot be verified; pin a release that carries one" exit 1 elif [ "$status" -ne 0 ]; then echo "::error::could not download ${bundle} for semstat ${tag}: curl exited ${status}" From f675dc6dd3d2af75cc6fa7c4d573c13de1382b5e Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Fri, 21 Aug 2026 12:05:52 +0200 Subject: [PATCH 08/10] docs(setup-semstat): say what an absent bundle means under the draft flow semstat now cuts its release as a draft and publishes it only after signing succeeds, so an outage leaves a draft rather than a published release with no provenance, and every published release carries its bundle. The absent -bundle branch therefore no longer describes an incomplete publish; it describes an asset missing from a release still mutable enough to lose one. --- .../actions/setup-semstat/src/install-semstat.sh | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/.github/actions/setup-semstat/src/install-semstat.sh b/.github/actions/setup-semstat/src/install-semstat.sh index 2f927f83..35f44736 100755 --- a/.github/actions/setup-semstat/src/install-semstat.sh +++ b/.github/actions/setup-semstat/src/install-semstat.sh @@ -239,13 +239,17 @@ done # Fetched on its own rather than in the loop above, because an absent bundle is a # different failure than an absent release and must not be reported as one. -# semstat signs at the end of goreleaser's publish phase, so a Fulcio or Rekor -# outage there leaves the release published with the archives but no provenance. +# semstat publishes a release only once signing has succeeded: goreleaser cuts it +# as a draft and the release workflow flips that off as its last step, so a Fulcio +# or Rekor outage leaves a deletable draft rather than a published release with no +# provenance. Every published release carries its bundle, the four cut before that +# flow included. # -# The message below names a different release rather than a re-dispatch of -# semstat's release workflow. Re-releasing a tag is semstat's repair and not -# something the reader of this error can reach, and it stops working outright on -# a release cut under immutability, so it is the wrong thing to send anyone after. +# So this branch no longer means an incomplete publish. It means the asset went +# missing from a release still mutable enough to lose it, which is nearer the +# tampering the signature check exists to catch. Either way the only repair the +# reader of this error can make is to name a release that has one: re-releasing a +# tag is semstat's to do, and cannot touch a release cut under immutability. if [ "$verify_signature" = true ]; then bundle=checksums.txt.sigstore.json status=0 From 362a68190cc85388cffeee27dc5df1935241ff17 Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Fri, 21 Aug 2026 13:30:51 +0200 Subject: [PATCH 09/10] feat(setup-semstat)!: verify the semstat signature by default checksums.txt is fetched from the same release as the archive it vouches for, so anyone able to replace one replaces the other in the same call. It proves the download arrived whole and nothing about who published it. The pinned release predates GitHub's release immutability, which is not retroactive, so its assets stay replaceable for good. The signature is the only link in the chain that does not come from the thing it is checking. Both actions are unreleased, so no caller changes behaviour under them. The cost is a cosign install per job and a dependency on Sigstore being reachable: verify-signature: false buys both back for a job that neither publishes nor gates a publish. The installer keeps defaulting to false when the variable is unset, because the actions are what install cosign and a direct run of the script has no such step. --- .github/actions/semver-validation/README.md | 34 ++++++++----- .github/actions/semver-validation/action.yml | 13 +++-- .github/actions/setup-semstat/README.md | 40 ++++++++++----- .github/actions/setup-semstat/action.yml | 11 ++-- .../setup-semstat/src/install-semstat.sh | 6 +++ .github/workflows/test-semver-validation.yaml | 13 +++++ .github/workflows/test-setup-semstat.yaml | 50 +++++++++++++++++++ CLAUDE.md | 2 +- README.md | 9 ++-- 9 files changed, 140 insertions(+), 38 deletions(-) diff --git a/.github/actions/semver-validation/README.md b/.github/actions/semver-validation/README.md index f968f400..19254236 100644 --- a/.github/actions/semver-validation/README.md +++ b/.github/actions/semver-validation/README.md @@ -17,9 +17,12 @@ The action needs a Linux or macOS runner with `curl`, `tar`, `jq` and either `github.com/loft-sh/semstat/releases/download` **and** `objects.githubusercontent.com`, which release-asset downloads redirect to. A proxy allowlist that names only `github.com` fails the install step. -`verify-signature: true` adds a `cosign` download from the -`github.com/sigstore/cosign` releases and egress to `tuf-repo-cdn.sigstore.dev`, -where cosign fetches the trusted root it checks the transparency log against. + +Because `verify-signature` defaults to true, two further hosts are needed by +default: a `cosign` download from the `github.com/sigstore/cosign` releases, and +egress to `tuf-repo-cdn.sigstore.dev`, where cosign fetches the trusted root it +checks the transparency log against. `verify-signature: false` drops both, and is +the setting for a runner whose allowlist cannot be changed. Calling it leaves the caller's `PATH` alone. The installer it shares with [`setup-semstat`](../setup-semstat/README.md) can put semstat there, and does for @@ -39,11 +42,11 @@ network and tool requirements with no version change to notice. -| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION | -|------------------|--------|----------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| compare_to | string | false | | Second version to order `version` against.
Leave empty to skip the comparison. | -| verify-signature | string | false | `"false"` | Verify the semstat release's `checksums.txt` against
its cosign bundle before trusting it,
proving the release came from semstat's
own release workflow at that exact
tag rather than only that the
download arrived intact. Costs a cosign
install on the job, so it
is off by default; turn it
on where this action's answer gates
a publish. | -| version | string | true | | Version string to validate against semver
format | +| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION | +|------------------|--------|----------|----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| compare_to | string | false | | Second version to order `version` against.
Leave empty to skip the comparison. | +| verify-signature | string | false | `"true"` | Verify the semstat release's `checksums.txt` against
its cosign bundle before trusting it,
proving the release came from semstat's
own release workflow at that exact
tag rather than only that the
download arrived intact. `checksums.txt` is fetched
from the same release as the
archive, so on its own it
proves the download arrived whole and
nothing about who published it. On
by default, because this action's `release_type`
and `is_stable` route publishes: a semstat
that misreports a prerelease as stable
sends an rc out as the
newest release. Set it to false
to trade that for a cosign
install on the job and a
dependency on Sigstore being reachable. | +| version | string | true | | Version string to validate against semver
format | @@ -166,11 +169,16 @@ gate on `is_greater == 'true'`. The release `checksums.txt` proves the download arrived intact and resolved inside the release asked for, but it comes from the same place as the archive, so it says -nothing about who published either. `verify-signature: true` installs `cosign` and -checks `checksums.txt` against its Sigstore bundle at the exact signing identity -before reading it; [`setup-semstat`](../setup-semstat/README.md) documents the -identity and the cost. It is off by default because it adds a cosign install to -every job. Turn it on where this action's answer gates a publish. +nothing about who published either. `verify-signature` installs `cosign` and checks +`checksums.txt` against its Sigstore bundle at the exact signing identity before +reading it; [`setup-semstat`](../setup-semstat/README.md) documents the identity +and the cost. + +It is on by default here, and the example above names it only to be explicit. +`release_type` and `is_stable` are what route a publish, so a semstat that +misreports a prerelease as stable is enough to send an rc out as the newest +release. Set it to false for a job that only reports a version to a human, which +buys back the cosign install and the dependency on Sigstore being reachable. ## From a shell script diff --git a/.github/actions/semver-validation/action.yml b/.github/actions/semver-validation/action.yml index fecd2205..291d1f84 100644 --- a/.github/actions/semver-validation/action.yml +++ b/.github/actions/semver-validation/action.yml @@ -17,11 +17,16 @@ inputs: description: | Verify the semstat release's `checksums.txt` against its cosign bundle before trusting it, proving the release came from semstat's own release workflow at - that exact tag rather than only that the download arrived intact. Costs a - cosign install on the job, so it is off by default; turn it on where this - action's answer gates a publish. + that exact tag rather than only that the download arrived intact. `checksums.txt` + is fetched from the same release as the archive, so on its own it proves the + download arrived whole and nothing about who published it. + + On by default, because this action's `release_type` and `is_stable` route + publishes: a semstat that misreports a prerelease as stable sends an rc out as + the newest release. Set it to false to trade that for a cosign install on the + job and a dependency on Sigstore being reachable. required: false - default: "false" + default: "true" outputs: is_valid: diff --git a/.github/actions/setup-semstat/README.md b/.github/actions/setup-semstat/README.md index aab98f9c..a35d9e0d 100644 --- a/.github/actions/setup-semstat/README.md +++ b/.github/actions/setup-semstat/README.md @@ -25,20 +25,23 @@ and egress to `github.com/loft-sh/semstat/releases/download` **and** `objects.githubusercontent.com`, which release-asset downloads redirect to. A proxy allowlist naming only `github.com` fails the download. -`verify-signature: true` needs two more hosts. `cosign` is downloaded from the -`github.com/sigstore/cosign` releases, by this action rather than by the runner -image, and cosign then checks the transparency log against a trusted root it -fetches from `tuf-repo-cdn.sigstore.dev`. An egress-restricted runner has to -allow both or the step fails inside cosign. +Two further hosts are needed **by default**, because `verify-signature` defaults to +true. `cosign` is downloaded from the `github.com/sigstore/cosign` releases, by this +action rather than by the runner image, and cosign then checks the transparency log +against a trusted root it fetches from `tuf-repo-cdn.sigstore.dev`. An +egress-restricted runner has to allow both, or the step fails inside cosign. + +`verify-signature: false` drops both, and is the setting for a runner whose +allowlist cannot be changed. ## Inputs -| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION | -|------------------|--------|----------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| verify-signature | string | false | `"false"` | Verify `checksums.txt` against its cosign bundle
before trusting it, proving the release
came from semstat's own release workflow
at this exact tag rather than
only that the download arrived intact.
Costs a cosign install on the
job, so it is off by
default; turn it on for jobs
that publish. | -| version | string | false | | Release of [loft-sh/semstat](https://github.com/loft-sh/semstat) to install. Empty
installs the release pinned in `src/install-semstat.sh`,
which is where Renovate bumps it
and where every entry point reads
it from. | +| INPUT | TYPE | REQUIRED | DEFAULT | DESCRIPTION | +|------------------|--------|----------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| verify-signature | string | false | `"true"` | Verify `checksums.txt` against its cosign bundle
before trusting it, proving the release
came from semstat's own release workflow
at this exact tag rather than
only that the download arrived intact.
`checksums.txt` is fetched from the same
release as the archive, so on
its own it proves the download
arrived whole and nothing about who
published it; this is the check
that answers that. On by default.
Set it to false to trade
the guarantee for a cosign install
on the job and a dependency
on Sigstore being reachable, which is
worth doing for a job that
neither publishes nor gates a publish. | +| version | string | false | | Release of [loft-sh/semstat](https://github.com/loft-sh/semstat) to install. Empty
installs the release pinned in `src/install-semstat.sh`,
which is where Renovate bumps it
and where every entry point reads
it from. | @@ -205,10 +208,21 @@ https://github.com/loft-sh/semstat/.github/workflows/release.yaml@refs/tags/@ org:loft-sh' ``` -`semver-validation` is the worked example (DEVOPS-1369). `v1`, `v2` and `v3` are the self-contained Node action; the rewrite as a composite over `semstat` needs egress, needs `curl`/`tar`/`sha256sum`/`jq`, and drops the `semstat_version` input, so it shipped as `v4`. Advancing `v1` or `v3` would have changed five live release workflows across five repos. It does not touch the caller's `PATH`, which would have been a fourth reason on its own: it binds `SEMSTAT_SKIP_PATH` precisely so the rewrite stays invisible to the job around it. +`semver-validation` is the worked example (DEVOPS-1369). `v1`, `v2` and `v3` are the self-contained Node action; the rewrite as a composite over `semstat` needs egress, needs `curl`/`tar`/`sha256sum`/`jq`, verifies its download by default and so needs Sigstore egress too, and drops the `semstat_version` input, so it shipped as `v4`. Advancing `v1` or `v3` would have changed five live release workflows across five repos. It does not touch the caller's `PATH`, which would have been a fourth reason on its own: it binds `SEMSTAT_SKIP_PATH` precisely so the rewrite stays invisible to the job around it. ### YAML-only / composite / Node.js actions (semver-validation, release-notification, etc.) - Update code and commit changes diff --git a/README.md b/README.md index 7a1026f0..b71e6121 100644 --- a/README.md +++ b/README.md @@ -1196,10 +1196,11 @@ choose to move. `semver-validation` is the worked example. `v1`, `v2` and `v3` are the self-contained Node implementation; the composite over `semstat` needs egress to the semstat -releases, needs `curl`/`tar`/`sha256sum`/`jq` on the runner, and drops the -`semstat_version` input, so it went out as `v4` rather than over any of them. It -leaves the job's `PATH` alone, which would otherwise have been a reason on its -own. Before force-pushing a tag, check who pins it: +releases, needs `curl`/`tar`/`sha256sum`/`jq` on the runner, verifies its download +by default and so needs Sigstore egress too, and drops the `semstat_version` input, +so it went out as `v4` rather than over any of them. It leaves the job's `PATH` +alone, which would otherwise have been a reason on its own. Before force-pushing a +tag, check who pins it: ```bash gh search code 'loft-sh/github-actions/.github/actions/@ org:loft-sh' From 043817c59f6724ebf6a556dc80abf37d4359f2f2 Mon Sep 17 00:00:00 2001 From: Dmytro Sydorov Date: Fri, 21 Aug 2026 13:37:07 +0200 Subject: [PATCH 10/10] test(setup-semstat): pin the reuse check off the new verifying default The second install in the end-to-end job inherited the flipped default, so it would have re-downloaded to verify, moved a new binary into place, and failed the inode reuse assertion for a reason that is not a reuse bug. Also gives every job in this workflow the timeout its sibling workflow already sets, which matters more now that a reachable Sigstore is on the default path. --- .github/workflows/test-setup-semstat.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/.github/workflows/test-setup-semstat.yaml b/.github/workflows/test-setup-semstat.yaml index e0ad095e..3200b751 100644 --- a/.github/workflows/test-setup-semstat.yaml +++ b/.github/workflows/test-setup-semstat.yaml @@ -17,6 +17,7 @@ jobs: bats: name: Run bats tests runs-on: ubuntu-latest + timeout-minutes: 10 steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: @@ -33,6 +34,7 @@ jobs: action: name: Run the action end to end runs-on: ubuntu-latest + timeout-minutes: 10 # A job environment that would repoint the installer if the action left any # of these unbound: an older release, a download root serving nothing, and a # demand that semstat stay off the job's PATH. Composite steps inherit it, so @@ -125,9 +127,16 @@ jobs: SEMSTAT_BIN: ${{ steps.install.outputs.path }} run: echo "inode=$(stat -c %i "$SEMSTAT_BIN")" >>"$GITHUB_OUTPUT" + # Has to ask for the same verification as the install above, or it is not + # the same install being asked for: the reuse path requires a signature + # claim at least as strong as the one requested, so leaving this to the + # default would re-download to verify, move a new binary into place, and + # fail the inode check below for a reason that is not a reuse bug. - name: Install semstat again id: reinstall uses: ./.github/actions/setup-semstat + with: + verify-signature: false - name: Assert the second install reused the first env: @@ -151,6 +160,7 @@ jobs: default_verifies: name: Verify by default runs-on: ubuntu-latest + timeout-minutes: 10 steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: @@ -184,6 +194,7 @@ jobs: signature: name: Verify the release signature runs-on: ubuntu-latest + timeout-minutes: 10 steps: - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 with: