diff --git a/.cursor/BUGBOT.md b/.cursor/BUGBOT.md deleted file mode 100644 index 51b42e1..0000000 --- a/.cursor/BUGBOT.md +++ /dev/null @@ -1,89 +0,0 @@ -# Bugbot guide — tracebloc/.github - -## Context - -**Public**, and it holds the reusable workflows every other repo calls. Roughly 120 -callers across 16 repos consume these at `@main`, so a change here reaches the whole -fleet on its first successful run — there is no per-consumer rollout. - -It also holds the conformance contract: `repo-inventory.yml` (what every repo must -have), `scripts/caller-drift.py` (the guard that checks it), and -`conformance-gate.yml` (the required check that refuses a contract change whose audit -did not pass). - -Three properties shape the real defects here: - -1. **A caller may only pass inputs the `@main` callee declares.** Add an input to a - caller before the reusable that accepts it has reached `main` and the whole call - dies with `startup_failure` — not a red job, a job that never starts. -2. **This repo is where "the check passed" is decided.** Most defects found here are - not wrong logic; they are a guard that reports success it did not verify. -3. **It is public.** No customer names, internal URLs, or internal paths in code, - comments, workflow prose, or fixtures. - -## Always flag - -- **Any path where a guard can report success without having checked.** This is the - house speciality and it has many shapes, all seen in this repo: an empty API - response read as "nothing found"; `|| echo 0` turning a failed call into a clean - count; a `grep -q` in a pipeline where `pipefail` turns a real hit into rc=141; a - required check whose job never runs on some PRs; a soft-fail default that makes a - "required" check exit 0 on findings. If a read can fail, the failure must produce an - UNREADABLE record, never a zero. - -- **A check that cannot fail, or cannot report.** Two mirror defects: a job that always - exits 0 (advisory by default while listed as required), and a job that is path- or - branch-filtered while being a *required* status check, so PRs outside the filter wait - forever at "Expected — waiting for status to be reported". Both look green. - -- **Anything that reads only ONE of GitHub's two protection systems.** A branch - protected solely by a ruleset returns 404 from `branches/{b}/protection` while the - branch list reports `protected: true`. Reading only the classic endpoint reports a - protected branch as unprotected. `bypass_actors` exists only on `/rulesets/{id}` — the - per-branch rules endpoint omits it entirely, so an allowlist asserted from that - endpoint asserts nothing. - -- **A mutation whose mutant does not parse.** In any `*-mutations.py` row, check that - the `new` string still leaves the target compilable: a literal `"\x00"` in a Python - replacement, or a `(0 && ` prepended at an anchor that stops mid-regex so the added - `)` closes nothing. Such a mutant reddens the WHOLE suite, the expected case is among - the failures by luck rather than by dependency, and the run scores `caught` about a - program that never ran — and `MISCAUGHT` cannot see it, because reddening everything - includes reddening the expected case. Four shipped this way in `.github#404` - (backend#3085). `pipefail-early-close-mutations.py` now parses every mutant first; - a harness that does not is worth flagging. Related: an anchor into a regex whose - character class carries quotes, backslashes or a `\001` should be GENERATED from the - real declaration, never retyped. - -- **A new input to a reusable that a caller starts passing in the same change.** See - property 1 — land the callee first, flip the caller in a follow-up. - -- **Changing a required check's job `name:`.** The name IS the contract; branch - protection matches on it. A rename silently turns the old required context into one - that can never report. - -## Known non-issues — do not flag - -- **`pii-gate / pii-check` failing red.** It was retired (backend#1409); a stale - required context can linger on old PRs. Not a leak. -- **The long incident comments.** Several files carry a paragraph explaining a specific - outage that shaped the code. They are load-bearing: each one is the reason a guard is - written the awkward way it is. Do not suggest trimming them for brevity. -- **`strict: false` on branch protection.** Deliberate fleet-wide (backend#1276) — the - release train handles stale bases better by re-evaluating and merging sha-pinned. -- **Duplicated prose between `org-standards.md` and repo CLAUDE.md files.** The block is - synced, not copy-pasted; never edit the consuming copy. - -## Tone - -The valuable finding here is almost always "on this path, the guard says yes without -having looked". Style findings on shell are low value; a fail-open is high value even -when it is currently unreachable, because this repo's whole job is to be the thing that -does not fail open. - -## Working with Bugbot findings (team norm) - -Triage every finding the same day: fix it, or reply on the thread saying why not. No -silent dismissals — unresolved threads block the merge and stall the release train's -settle stage. A finding that recurs becomes a rule: add it here, and if it is -grep-expressible, to code-quality's house-rules. diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 5987230..589e535 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1,32 +1,16 @@ # Code owners for tracebloc/.github # -# This repository is the org's enforcement root: 96 caller workflows across 14 -# repos reference `tracebloc/.github/.github/workflows/*.yml@main`, and every -# one of them passes `secrets: inherit`. A change merged here takes effect in -# every repo on the next workflow run, and reaches every secret those repos -# hold. +# This repository holds only the organization profile and the org-wide issue +# and pull request templates; the reusable workflows and scripts that made it +# the org's enforcement root moved to a private repository. # -# WHAT THIS FILE DOES, AND DOES NOT, DO -- measured 2026-08-16. -# -# It AUTO-REQUESTS a reviewer on the paths below. It does NOT gate the merge: -# `require_code_owner_reviews` is `false` on develop, staging and main, so any -# approving review turns the PR green. That is deliberate (a PR must never wait -# on one specific person to be approvable), and it is stated here because the -# header used to assert the opposite -- "branch protection on `main` already -# sets require_code_owner_reviews: true" -- which was false when checked. A -# CODEOWNERS file that misdescribes its own enforcement is worse than none: it -# reads as a control while gating nothing. -# -# The `*` line is GONE, deliberately. It auto-requested the same person on every -# PR in the repo they touch most, which reads in the UI as "waiting on Lukas" -# long after someone else has approved -- and it made this the second `*` rule -# in an org whose convention is narrow, security-only ownership with -# author-picks-reviewer everywhere else. The two paths below are the ones that -# actually carry the blast radius described at the top; a README or an issue -# template does not. +# What this file does: it AUTO-REQUESTS a reviewer on the paths below. It does +# NOT gate the merge -- `require_code_owner_reviews` is false on every branch, +# so any approving review turns a PR green. That is deliberate. -# The reusable workflows that define every gate in the org. -/.github/workflows/ @LukasWodka +# The organization landing page, rendered on github.com/tracebloc. +/profile/ @LukasWodka -# Scripts invoked by those workflows. -/scripts/ @LukasWodka +# Templates that appear in every repository's "new issue" / "new PR" chooser. +/.github/ISSUE_TEMPLATE/ @LukasWodka +/.github/pull_request_template.md @LukasWodka diff --git a/.github/workflows/actionlint.yml b/.github/workflows/actionlint.yml deleted file mode 100644 index 806fc9d..0000000 --- a/.github/workflows/actionlint.yml +++ /dev/null @@ -1,148 +0,0 @@ -name: Actionlint - -# Repo-local CI for this repo's OWN workflow files. Not a reusable workflow -- -# nothing calls it; it runs on pull requests here. -# -# WHY -# This repo publishes the reusable workflows that every other repo in the org -# consumes at `@main`, and until now nothing validated them before merge. A -# mistyped expression, a bad `needs:` edge, or a broken shell line shipped -# org-wide the moment it landed. actionlint parses each workflow, type-checks -# every `${{ }}` expression, validates the runs-on / uses / needs wiring, and -# runs shellcheck over every `run:` block -- i.e. it covers exactly the class -# of mistake that is invisible in review and expensive in production. -# -# BLOCKING BEHAVIOUR -# A hard gate from day one: any finding fails the job. That is affordable -# only because the pre-existing backlog (29 findings across 8 workflows) was -# cleared in the same change that added this file, so the tree starts at zero -# and the job is green immediately. Contrast code-quality.yml, which ships -# `soft-fail: true` because it is aimed at repos with an unlinted backlog -- -# a linter switched on as a required check against a dirty tree gets the -# check removed, not the tree fixed. -# -# Suppressions are per-line `# shellcheck disable=SCxxxx` directives placed at -# the site, each with a comment saying why. There is deliberately no -# `.github/actionlint.yaml` and no `-ignore` flag: a repo-wide exclusion would -# silently cover future code as well as the line it was written for. -# -# Before marking this a required status check, read the `paths:` filter below. -# It means the job does NOT run on a PR that touches no workflow file (a -# README-only change, say), and a required check that never runs leaves such a -# PR waiting for a status forever. Either drop the filter when you require the -# check, or keep the filter and leave the check advisory. -# -# SUPPLY CHAIN -# actionlint is installed from its release tarball, pinned by version AND -# verified against a pinned SHA-256, rather than through a wrapper action that -# downloads it for us -- a digest pin on the artefact actually executed is a -# stronger guarantee than a commit pin on the wrapper. `actions/checkout` is -# pinned to a full commit SHA. To bump: change ACTIONLINT_VERSION and -# ACTIONLINT_SHA256 together, taking the digest from the upstream -# `actionlint__checksums.txt` release asset. -# -# RUN IT LOCALLY (same tool, same checks as CI) -# brew install actionlint shellcheck # or: go install .../actionlint@latest -# actionlint - -# Deliberately NO `paths:` filter. A required status check that is filtered by -# path never runs on a PR that touches nothing matching it -- and a required -# check that never runs leaves that PR waiting forever for a status that cannot -# arrive. The job is ~7s, so running it on every PR costs less than that class -# of stuck PR. Keep it unfiltered for as long as it is a required check. -on: - pull_request: - types: [opened, reopened, synchronize, ready_for_review] - workflow_dispatch: - -# Supersede the previous run when the branch is pushed again, instead of -# stacking duplicate runs per push. -concurrency: - group: actionlint-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -# Read-only. The job checks out the tree and runs a linter -- it never writes -# to the repo, the API, or the project board. -permissions: - contents: read - -jobs: - actionlint: - name: actionlint - runs-on: ubuntu-latest - timeout-minutes: 5 - env: - ACTIONLINT_VERSION: "1.7.12" - ACTIONLINT_SHA256: "8aca8db96f1b94770f1b0d72b6dddcb1ebb8123cb3712530b08cc387b349a3d8" - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - persist-credentials: false - ref: ${{ github.event.pull_request.head.sha || github.sha }} - - - name: Install actionlint (version + digest pinned) - run: | - set -euo pipefail - TARBALL="actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" - URL="https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}/${TARBALL}" - curl -fsSL --retry 3 --retry-delay 2 -o "$RUNNER_TEMP/$TARBALL" "$URL" - echo "${ACTIONLINT_SHA256} $RUNNER_TEMP/$TARBALL" | sha256sum -c - - tar -xzf "$RUNNER_TEMP/$TARBALL" -C "$RUNNER_TEMP" actionlint - install -m 0755 "$RUNNER_TEMP/actionlint" /usr/local/bin/actionlint - actionlint -version - - - name: Check shellcheck is present - run: | - set -euo pipefail - # actionlint shells out to shellcheck for every `run:` block, but when - # the binary is missing it SKIPS that whole class of checks and still - # exits 0 -- the gate would stay green while checking far less. Fail - # loudly rather than degrade quietly. - if ! command -v shellcheck > /dev/null 2>&1; then - echo "::error::shellcheck is not on the runner -- actionlint would skip every shell check." - exit 1 - fi - shellcheck --version | sed -n '/version:/p' - - - name: Lint workflows - run: | - set -uo pipefail - # -shellcheck is passed explicitly so the integration cannot be turned - # off by an empty default. No -ignore and no config file on purpose: - # suppressions belong at the line they apply to. - actionlint -no-color -oneline -shellcheck shellcheck > "$RUNNER_TEMP/findings.txt" - STATUS=$? - - # Exit 0 = clean, 1 = findings, anything else = actionlint itself failed - # (bad flag, no workflows directory). Don't report that as a lint result. - if [ "$STATUS" -gt 1 ]; then - echo "::error::actionlint could not run (exit $STATUS). See the log above." - exit "$STATUS" - fi - - COUNT=$(wc -l < "$RUNNER_TEMP/findings.txt" | tr -d ' ') - if [ "$COUNT" = "0" ]; then - echo "actionlint: 0 findings." - echo "### actionlint: 0 findings" >> "$GITHUB_STEP_SUMMARY" - exit 0 - fi - - cat "$RUNNER_TEMP/findings.txt" - # Annotate each finding on the diff. -oneline emits "path:line:col: msg", - # so the location is the first three colon-separated fields. - awk -F: 'NF>=4 { - loc = $1 ":" $2 ":" $3 - printf "::error file=%s,line=%s,col=%s::%s\n", $1, $2, $3, substr($0, length(loc) + 3) - }' "$RUNNER_TEMP/findings.txt" - - { - echo "### actionlint: $COUNT finding(s)" - echo "" - echo '```' - cat "$RUNNER_TEMP/findings.txt" - echo '```' - } >> "$GITHUB_STEP_SUMMARY" - - echo "::error::actionlint reported $COUNT finding(s)." - echo "::error::Fix them, or add a targeted '# shellcheck disable=SCxxxx' at the specific line with a reason." - exit 1 diff --git a/.github/workflows/add-to-kanban.yml b/.github/workflows/add-to-kanban.yml deleted file mode 100644 index 2a3a432..0000000 --- a/.github/workflows/add-to-kanban.yml +++ /dev/null @@ -1,92 +0,0 @@ -name: Add to engineer kanban - -on: - issues: - types: [opened, reopened, transferred] - pull_request: - types: [opened, reopened, ready_for_review] - -jobs: - add-to-project: - runs-on: ubuntu-latest - # NO GITHUB_TOKEN AT ALL (saadqbal, #2181). Every call in this job authenticates - # as the App, so the workflow token needs nothing -- and an empty grant is the - # only version of that claim a reader can check. Free, and it means the least- - # privilege story covers both credentials in the job rather than just the loud one. - permissions: {} - steps: - # Board writes authenticate as the tracebloc-release-train App (backend#2036), - # not a human's PAT. `owner:` yields an ORG-scoped installation token; a - # repo-scoped one cannot write the org project. No fallback to the PAT: a - # fallback would let a broken App path keep working silently. - # - # This workflow also fires on DEPENDABOT PRs, which GitHub gates on a separate - # secret scope -- both app secrets are set there too, or Dependabot PRs would - # stop reaching the board with `Input required and not supplied`. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # SCOPED TO THIS REPO, or the two reads below land org-wide (saadqbal, - # #2181). `owner:` alone does not narrow anything -- run 32239403796 says - # so in as many words: "Input 'repositories' is not set. Creating token for - # all repositories owned by tracebloc." A token calling itself - # least-privilege while carrying issue+PR read across all 19 installed - # repos is the claim this PR exists to stop making. - # - # `organization_projects` is an ORG-level permission and is not affected by - # repo scoping, so the board write should be unchanged -- but that is an - # assumption, and it is the same class of assumption that broke the first - # attempt, so the verification run is what settles it rather than this - # comment. If it is wrong the failure is LOUD (see below), which is what - # makes trying it cheap. - repositories: ${{ github.event.repository.name }} - # Least privilege (backend#2166): without any `permission-*` the token - # carries the App's FULL installation grant. actions/add-to-project needs - # THREE scopes, not one: it must RESOLVE the triggering issue/PR node - # before it can add it to the board, so it needs read on both content - # types (this workflow fires on `issues` and `pull_request`) in addition - # to the project write. Projects-write alone leaves the node unresolvable - # -- the add fails with "Could not resolve to a node with the global id". - # - # WHAT IS ACTUALLY DEMONSTRATED, and what is not. Stated narrowly because - # two earlier versions of this paragraph each overclaimed in a different - # direction, and this text is copied verbatim into 17 repos -- a wrong - # argument here is a wrong argument 17 times, in a byte-compared file - # nobody re-derives. - # - # DEMONSTRATED: a MISSING READ scope fails loudly. Run 32239403796 on this - # branch, at commit 218f0b13 (projects-write only), errored with - # `Could not resolve to a node with the global id` and the job went RED -- - # `add-to-project` routes GraphQL errors through `setFailed`. - # - # NOT DEMONSTRATED: the case the FIRST version of this comment described -- - # the token resolving the node fine and then 403ing the BOARD WRITE. No run - # has ever produced it. So "fails loudly" is proven for the read scopes and - # is an expectation, not a measurement, for the write. - # - # AND ONE RUN THAT LOOKED LIKE EVIDENCE IS NOT (aptracebloc). The previous - # wording cited run 32237283072 as a second scope failure. It is not one: - # it ran on `develop`, whose file at that sha passes NO `permission-*` at - # all (the App's full grant), and it failed on - # `Could not resolve to a node with the global id of I_kwDONNfQt88...` -- - # a node a fully-privileged token also cannot see, i.e. an issue that no - # longer exists (this workflow fires on `issues: transferred`). Run - # 32237067262, the SAME develop sha, succeeded 2m34s earlier. A dead node, - # not a permission. - # - # The proof this is right is therefore still a LANDED CARD, not a passing - # mint: a mint can succeed with a scope the board write then needs and - # lacks, and that is the one path nothing here has exercised. - permission-issues: read - permission-pull-requests: read - permission-organization-projects: write - - - uses: actions/add-to-project@5afcf98fcd03f1c2f92c3c83f58ae24323cc57fd # v2.0.0 - with: - project-url: https://github.com/orgs/tracebloc/projects/2 - github-token: ${{ steps.app-token.outputs.token }} - diff --git a/.github/workflows/advance-deploy-env-caller.yml b/.github/workflows/advance-deploy-env-caller.yml deleted file mode 100644 index 3ed12c8..0000000 --- a/.github/workflows/advance-deploy-env-caller.yml +++ /dev/null @@ -1,17 +0,0 @@ -name: Advance deploy env - -# .github hosts the reusable advance-deploy-env.yml (workflow_call only), so — -# unlike every other repo — it had no push trigger of its own. Its kanban items -# therefore never advanced as their code moved through develop -> main: feature -# PRs stranded at "FR on dev" while their promotion PR alone reached Prod. This -# caller fires the reusable on this repo's own pushes, like the other 14 repos, -# so shipping to main auto-advances the contained tickets to Prod. - -on: - push: - branches: [develop, staging, master, main] - -jobs: - advance: - uses: tracebloc/.github/.github/workflows/advance-deploy-env.yml@main - secrets: inherit diff --git a/.github/workflows/advance-deploy-env-selftest.yml b/.github/workflows/advance-deploy-env-selftest.yml deleted file mode 100644 index 6d12668..0000000 --- a/.github/workflows/advance-deploy-env-selftest.yml +++ /dev/null @@ -1,46 +0,0 @@ -name: advance-deploy-env selftest - -# advance-deploy-env moves a merged PR's kanban card, and it decides WHICH card -# by attributing each pushed commit to a PR. It used to read that from the commit -# SUBJECT, which the org convention `type(scope): summary (backend#N)` breaks -- -# the `(#N)` slot holds a ticket, or an edited squash subject holds nothing -- so -# it advanced the wrong card or none (2/48 on the 2026-09-07 staging hop, -# backend#3365). The fix derives the PR from GitHub; this pins it with the two -# real misses as fixtures so the class cannot silently return. -# -# Offline: the suite builds throwaway git repos and stubs `gh` on PATH, so it -# needs no token and reaches no network. Path-filtered like the other selftests -- -# it only runs when the thing it tests changes. - -on: - pull_request: - paths: - - scripts/extract-advanced-prs.sh - - scripts/tests/extract-advanced-prs-selftest.sh - - .github/workflows/advance-deploy-env-selftest.yml - push: - branches: [main, develop, staging] - paths: - - scripts/extract-advanced-prs.sh - - scripts/tests/extract-advanced-prs-selftest.sh - - .github/workflows/advance-deploy-env-selftest.yml - -permissions: - contents: read - -concurrency: - group: advance-deploy-env-selftest-${{ github.ref }} - cancel-in-progress: true - -jobs: - selftest: - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # The suite creates real commits; a committer identity is not configured on - # a fresh runner. - - run: | - git config --global user.email "selftest@tracebloc.io" - git config --global user.name "advance-deploy-env selftest" - - run: bash scripts/tests/extract-advanced-prs-selftest.sh diff --git a/.github/workflows/advance-deploy-env.yml b/.github/workflows/advance-deploy-env.yml deleted file mode 100644 index 6df79e6..0000000 --- a/.github/workflows/advance-deploy-env.yml +++ /dev/null @@ -1,461 +0,0 @@ -name: Advance deploy environment - -# Reusable workflow. Called from each active repo on push to develop/staging/master/main. -# For each PR contained in the push, updates the Deploy environment project field -# and advances Status through the multi-stage validation flow: -# develop -> Status = "On dev" (automatic; no dev-side review — D6) -# staging -> Status = "FR on staging" (functional review on staging environment) -# master/main -> Status = "Prod" (shipped to production) -# -# Advancement is MONOTONIC: a card is never moved backward. Back-merges and -# fast-forwards re-push commits that already shipped further down the line -# (staging->develop back-merge, develop==main sync); those cards keep their -# furthest Status instead of being "un-shipped" on the board. -# -# The "Ready for prod" intermediate state is set manually via /fr-pass at staging -# (drag-and-drop on the kanban or via a /fr-pass comment) when the FR reviewer -# declares the validation passed but the deploy hasn't happened yet. -# -# -- Per-repo override via `.kanban.yml` -- -# For repos where the default branch isn't the prod-truth (e.g. averaging-service -# deploys a Docker image from staging without ever touching main), drop a -# `.kanban.yml` at the repo root: -# -# # .kanban.yml -# branch_status_map: -# staging: Prod # this repo's deploy ships staging-built artifacts -# -# The override merges with the default mapping. Anything you don't explicitly -# remap stays on the default. Keys are branch names. -# -# VALUES MUST BE A STATUS `ENV_FOR_STATUS` DECLARES, in -# tracebloc/.github's `scripts/branch_status_map.py` -- read the list there rather -# than from a copy here. The three names this comment used to list were a copy, and -# the table has five. -# A value outside it is REFUSED (backend#2324): the mapper exits non-zero, this -# workflow fails red and the card keeps whatever Status it had. It used to be -# accepted and passed straight to the board write, where it resolves to no option -# id -- and in the closure router that no-write let the project's built-in -# "Item closed" automation set `Cancelled` and archive the card within a day. - -on: - workflow_call: - inputs: - project-number: - description: "GitHub Projects v2 number (default: 2 = engineer kanban)" - type: number - default: 2 - org: - description: "GitHub org owning the project" - type: string - default: tracebloc - dry_run: - description: "Log intended card moves without writing them (test mode)" - type: boolean - default: false - -jobs: - advance: - # A branch CREATION or deletion is not a merge — its commits are inherited, - # not newly shipped. Without this guard, creating a branch (BEFORE = zero - # hash) falls through to the "last 50 commits" range below and mass-advances - # ~50 recent PRs' kanban items — e.g. every time a new `staging` branch is - # cut for a repo (RFC-BACKEND-0008 D8/#1274). Skip create/delete pushes. - if: github.event.created != true && github.event.deleted != true - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - - # The mapping comes from .github, not from a copy in this file: one - # definition of branch -> Status (backend#2243). - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: tracebloc/.github - ref: main - path: .kanban-map - persist-credentials: false - - - name: Resolve deploy env + status (one shared mapping) - id: env - env: - BRANCH: ${{ github.ref_name }} - # The workflow token, not the App token: this reads `.kanban.yml` - # from the caller's own repo, which is `contents: read` on itself. - GH_TOKEN: ${{ github.token }} - # ONE DEFINITION, SHARED WITH THE CLOSURE ROUTER (backend#2243). This step - # used to hold its own `case` plus its own `yq` read of `.kanban.yml`, and - # the router held two more copies that ignored the override entirely -- so - # with a `.kanban.yml` present the two workflows wrote DIFFERENT statuses - # for the same merge and run ordering decided which stuck. - # Third argument is the REF to read `.kanban.yml` from, which is the branch - # being mapped -- not the repo default (Bugbot, .github#295). - run: python3 .kanban-map/scripts/branch_status_map.py "$BRANCH" "$GITHUB_REPOSITORY" "$BRANCH" - - - name: Skip if branch not tracked - if: steps.env.outputs.env == '' - env: - BRANCH: ${{ github.ref_name }} - run: | - echo "Branch '$BRANCH' is not develop/staging/master/main (and no .kanban.yml override) - nothing to do." - - # THIS is the workflow backend#2036 was filed about. On 2026-08-14 at 06:22 - # UTC, run 31776053792 on client-runtime died with - # gh: API rate limit already exceeded for user ID 54042461 - # exit 1 -- because every board caller in the fleet, both crons, the - # conformance gate and that person's own `gh` shared ONE user PAT's 5,000/hr. - # The workflow failed CLOSED, which is right, and the cost was still a card - # left behind its own shipped code until someone noticed. - # - # An installation token has a budget that is not shared with a human's - # interactive use and does not evaporate when that person rotates a token or - # leaves. `owner:` makes it ORG-scoped; a repo-scoped token cannot write an - # org ProjectV2. - # - # Minted BEFORE the extract step, because the extract step is its first - # consumer (backend#3447): `/commits/{sha}/pulls` needs `pull-requests: read`, - # and the caller's `github.token` runs under the org default of restricted - # `read` -- contents and packages only -- so under that token the read 403s - # on every PRIVATE repo and succeeds on every public one. From .github#438 - # reaching `main` (2026-09-08 19:10 UTC) until this change, that was 100 % of - # private-repo pushes: the fail-closed branch below did its job and no card - # moved. A called workflow can only DOWNGRADE the caller's token, so the fix - # is not a `permissions:` block here -- it is this token, which already - # carries the grant. Gated on a tracked branch only: a push to an untracked - # branch still mints nothing. - # - # NO FALLBACK TO THE PAT: a fallback would let a broken App path keep - # working silently, so the migration would look complete while nothing had - # migrated (backend#1680's whole subject). - - name: Mint an installation token - id: app-token - if: steps.env.outputs.env != '' - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # LEAST PRIVILEGE, DERIVED FROM THIS JOB'S OWN CALLS (backend#2157). - # Without any `permission-*` the token carried the App's FULL installation - # grant -- contents:write included -- across every installed repo. The - # step below makes exactly five kinds of call, and this is the union of - # what they need: - # - # organization(login:).projectV2 field/option lookup projects read - # updateProjectV2ItemFieldValue projects WRITE - # repository.pullRequest(n).projectItems PRs read - # GET /repos/{r}/commits/{sha}/pulls (extract step) PRs read - # contents read - # - # The ONE repository-CONTENT read is the extract step's - # `/commits/{sha}/pulls`, so contents stays at `read` -- see the call-site - # block below before dropping it. Neither checkout above is on this token - # (both use `github.token` / `persist-credentials: false`), so contents - # stops at read -- and administration/actions/checks read still drop. - # - # `repositories:` STAYS UNNARROWED, but the reason it used to give is gone. - # It cited the closing-issue advancement, which was deliberately cross-repo; - # backend#2722 removed that block, so nothing here reaches outside the - # calling repo any more. `organization-projects` is an ORG-level grant and is - # not narrowed by `owner:` alone either way, so leaving this unnarrowed costs - # nothing measurable -- narrowing it is a plausible tidy-up, but it is a token - # -scope change that wants its own measurement rather than riding a - # behavioural fix. Do not re-cite the closing-issue block as the reason. - # - # WHAT IS NOT PROVEN HERE. These are the scopes the calls DOCUMENT a need - # for, not scopes a run has exercised. An under-scoped token does not fail - # at mint time -- it fails at the call, which for this workflow means a - # card left behind its own shipped code. The first push to a tracked - # branch carrying a PR is the real test; if it reddens, read the failing - # call rather than widening the list back to a full grant. - # `contents: read` IS FOR THE EXTRACT STEP'S CALL SITE, NOT THE MAPPER'S - # (backend#3447). Read this before dropping it again. - # - # saadqbal's finding on .github#324 was narrow and is STILL CORRECT, by its - # own test -- check the call site, not the prose. `contents: read` reached - # this file by analogy with kanban-closure-router, whose mapper calls run - # AFTER its mint under the App token; this workflow's ONE mapper call is - # `:94`, in the `env` step, which runs BEFORE this mint and under - # `GH_TOKEN: ${{ github.token }}`. It also maps `$GITHUB_REPOSITORY`, its - # own repo, where the router maps `$REPO_FULL`, the caller's. Same script, - # opposite side of the mint: the scope was required there and inert here. - # That reasoning is untouched, and the mapper is still not a reason to - # grant contents. Do not re-cite it as one. - # - # WHAT CHANGED IS A NEW CALL SITE ON THIS TOKEN. .github#438 put - # `GET /repos/{r}/commits/{sha}/pulls` in the extract step, and this PR - # moves that step to AFTER the mint -- so by the same call-site test its - # scope is now this token's business, which it was not when #324 was - # written. `fr-gate.yml`'s mint comment ("NOT DERIVED FROM A TEMPLATE") - # records that endpoint as `contents: read + pull-requests: read`, derived - # from its own calls under backend#2157 and running as a required check on - # every promotion since; no workflow in this repo reads that endpoint under - # `pull-requests: read` alone. Granting only PRs read - # would reproduce backend#3447 one scope over, in the same silent shape: - # the read 403s, the fail-closed branch refuses the subject fallback, and - # no card moves on any private repo. - # - # `permission-issues` DROPPED (backend#2722). The only issue reads were - # `repository.issue(n) {state, projectItems}` in the removed closing-issue - # block; with it gone, nothing in this workflow touches an issue. Re-add it - # only alongside a call that needs it. - permission-contents: read - permission-pull-requests: read - permission-organization-projects: write - - - name: Extract PR numbers from new commits - id: prs - if: steps.env.outputs.env != '' - env: - BEFORE: ${{ github.event.before }} - SHA: ${{ github.sha }} - # The App token minted ABOVE, not `github.token`: `/commits/{sha}/pulls` - # needs `pull-requests: read` + `contents: read`, and the org-default - # restricted workflow token holds only the latter on a private repo - # (backend#3447; the mint step's comment has the derivation). - GH_TOKEN: ${{ steps.app-token.outputs.token }} - # DERIVE each commit's PR from GitHub, not from its subject text - # (backend#3365). The org convention `type(scope): summary (backend#N)` - # puts a ticket in the `(#N)` slot, and an edited squash subject may carry - # no `(#N)` at all -- so the old subject-grep attributed the wrong card or - # none (2/48 on the 2026-09-07 staging hop). The script GETs each commit's - # merged PR and falls back to the subject only when the API is empty, with a - # `::warning::` so an unattributed commit is visible. It lives in .github - # (checked out at .kanban-map above), one definition; `git log` runs against - # the caller's checkout (cwd). Selftest: scripts/tests/extract-advanced-prs-selftest.sh. - run: bash .kanban-map/scripts/extract-advanced-prs.sh - - - name: Update project fields for each PR - if: steps.env.outputs.env != '' && steps.prs.outputs.prs != '' - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - ORG: ${{ inputs.org }} - PROJECT_NUMBER: ${{ inputs.project-number }} - DEPLOY_ENV: ${{ steps.env.outputs.env }} - STATUS_NAME: ${{ steps.env.outputs.status_name }} - REPO_FULL: ${{ github.repository }} - PR_NUMBERS: ${{ steps.prs.outputs.prs }} - DRY_RUN: ${{ inputs.dry_run }} - run: | - set -euo pipefail - REPO_NAME="${REPO_FULL#*/}" - - # Look up project ID + relevant field/option IDs (one query) - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - PROJ=$(gh api graphql -f query=' - query($org: String!, $num: Int!) { - organization(login: $org) { - projectV2(number: $num) { - id - fields(first: 50) { - nodes { - ... on ProjectV2SingleSelectField { id name options { id name } } - } - } - } - } - }' -F org="$ORG" -F num="$PROJECT_NUMBER") - - PROJECT_ID=$(echo "$PROJ" | jq -r '.data.organization.projectV2.id') - DEPLOY_FIELD=$(echo "$PROJ" | jq -r '.data.organization.projectV2.fields.nodes[] - | select(.name=="Deploy environment") | .id') - DEPLOY_OPT=$(echo "$PROJ" | jq -r --arg e "$DEPLOY_ENV" '.data.organization.projectV2.fields.nodes[] - | select(.name=="Deploy environment") | .options[] | select(.name==$e) | .id') - STATUS_FIELD=$(echo "$PROJ" | jq -r '.data.organization.projectV2.fields.nodes[] - | select(.name=="Status") | .id') - STATUS_OPT=$(echo "$PROJ" | jq -r --arg s "$STATUS_NAME" '.data.organization.projectV2.fields.nodes[] - | select(.name=="Status") | .options[] | select(.name==$s) | .id') - if [ -z "$DEPLOY_OPT" ] || [ "$DEPLOY_OPT" = "null" ]; then - echo "Could not resolve Deploy environment option for '$DEPLOY_ENV' - aborting" - exit 1 - fi - - # Status resolution does NOT degrade gracefully -- see the abort below. - # It used to, and that was the bug: a graceful degrade here means the - # board stops advancing while every run reports success. - # FAIL CLOSED. This used to warn and set SKIP_STATUS=1, so an unresolvable - # Status option meant the run stayed GREEN while no card advanced -- the - # board silently stops tracking the pipeline and the only signal is a - # warning nobody reads (Bugbot, .github#243, High). - # - # It is also the ONE asymmetry in this file and its siblings: the Deploy - # environment lookup twelve lines above aborts, and - # kanban-closure-router.yml aborts on exactly this condition. A column - # rename, a project renumbering or a `.kanban.yml` naming a column that - # does not exist are all misconfigurations, and every one of them is - # cheaper to find as a red run than as three weeks of un-advanced cards. - if [ -z "$STATUS_FIELD" ] || [ "$STATUS_FIELD" = "null" ] \ - || [ -z "$STATUS_OPT" ] || [ "$STATUS_OPT" = "null" ]; then - echo "::error::Could not resolve Status option '$STATUS_NAME' in project #$PROJECT_NUMBER." \ - "NOTHING WAS WRITTEN: this check runs before any field update, so no card" \ - "was touched and there is no half-applied state to repair." \ - "Check the board's Status column names against this workflow's branch map." >&2 - exit 1 - fi - # Now always 0 -- the branch that set it to 1 aborts instead. Kept so the - # two `[ "$SKIP_STATUS" != "1" ]` guards below stay valid without - # re-indenting their blocks; there is no live skip path. - SKIP_STATUS=0 - - # Pipeline order (mirrors fr-gate's rank). Advancement is monotonic: - # a push only ever moves a card FORWARD. Pushes routinely carry commits - # that already shipped further down the pipeline -- a staging->develop - # back-merge re-pushes Prod-shipped PRs to develop, a develop==main - # fast-forward does the same -- and demoting those cards would un-ship - # them on the board (seen live: .github#87-#92 and #95). - rank() { - case "$1" in - "Backlog") echo 1 ;; - "North Stars") echo 2 ;; - "Ready") echo 3 ;; - "In progress") echo 4 ;; - "Code review") echo 5 ;; - "On dev") echo 6 ;; - # RANK 7: the agent stage, between `On dev` and human FR. READ-ONLY for - # now -- nothing writes this value yet (#1578 does that, in a LATER - # hop). An unknown Status returns "" here, the guard below fails, and - # evaluation falls through to strict equality: the card BLOCKS every - # prod promotion carrying it. That is the backend#1411 shape, and the - # column already EXISTS on the board, so this was a live landmine - # waiting for the first card to land in it (#1577, RFC-BACKEND-1552 D5). - "Staging (agent review)") echo 7 ;; - # `Staging (human review)` IS GONE (saadqbal on .github#295). It ranked - # here as a shim so the monotonic guard stayed stable across the #1592 - # rename INSTANT -- and that instant has passed: measured against project - # #2, whose Status options are Backlog, North Stars, Ready, In progress, - # Code review, On dev, Staging (agent review), FR on staging, Ready for - # prod, Prod, Done, Cancelled. No card can carry the old name because the - # column does not exist. - # - # Keeping a shim for a completed rename is how this same file came to - # accept an override value naming a nonexistent column -- the headline - # finding of this PR. Retiring it is also what lets this file back into - # `kanban-columns-check.py`'s WRITERS: with the phantom gone, all twelve - # remaining names are live board options. - "FR on staging") echo 8 ;; - "Ready for prod") echo 9 ;; - "Prod") echo 10 ;; - # Done and Cancelled are TERMINAL: nothing may demote a card out of - # them. Both returned 0 here, which inverted the monotonic guard below - # so the next push carrying an old commit demoted a Done card -- and - # staging->develop back-merges re-carry old commits routinely - # (RFC-BACKEND-1405 D8, backend#1411). - "Done") echo 11 ;; - "Cancelled") echo 11 ;; - *) echo 0 ;; - esac - } - TARGET_RANK=$(rank "$STATUS_NAME") - - # Apply one single-select field update WITHOUT letting a single bad item - # abort the whole push. An unguarded mutation under `set -e` aborts the - # step on the first failure, leaving every later PR un-advanced. Archived - # items are already skipped above; this is defense-in-depth for any other - # per-item error -- log it, flag the run, but keep processing the rest. - RUN_FAILED=0 - update_field() { # $1=fieldId $2=optionId $3=human label - local err - if [ "${DRY_RUN:-false}" = "true" ]; then - echo "[DRY] would set $3 on item $ITEM_ID" - return 0 - fi - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - if err=$(gh api graphql -f query=' - mutation($p: ID!, $i: ID!, $f: ID!, $o: String!) { - updateProjectV2ItemFieldValue(input: { - projectId: $p, itemId: $i, fieldId: $f, - value: {singleSelectOptionId: $o} - }) { projectV2Item { id } } - }' -F p="$PROJECT_ID" -F i="$ITEM_ID" -F f="$1" -f o="$2" 2>&1 >/dev/null); then - return 0 - fi - echo "::warning::#$prnum $3 update failed: $err" - RUN_FAILED=1 - return 0 - } - - for prnum in $PR_NUMBERS; do - # Defensive: if the number isn't a PR (e.g. issue ref slipped through), - # the gh api call exits non-zero - swallow that and skip cleanly. - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - RESP=$(gh api graphql -f query=' - query($org: String!, $repo: String!, $num: Int!) { - repository(owner: $org, name: $repo) { - pullRequest(number: $num) { - projectItems(first: 10) { - nodes { - id isArchived project { number } - status: fieldValueByName(name: "Status") { - ... on ProjectV2ItemFieldSingleSelectValue { name } - } - } - } - } - } - }' -F org="$ORG" -F repo="$REPO_NAME" -F num="$prnum" 2>/dev/null) || RESP='{}' - - ITEM_ID=$(echo "$RESP" | jq -r --arg n "$PROJECT_NUMBER" '.data.repository.pullRequest.projectItems.nodes[]? - | select(.project.number == ($n | tonumber)) | .id' 2>/dev/null | head -1) - - if [ -z "$ITEM_ID" ] || [ "$ITEM_ID" = "null" ]; then - echo "#$prnum not on project (or not a PR) - skipping" - continue - fi - - # An archived project item cannot be updated: the mutation errors and, - # under `set -e`, used to abort the whole step -- stranding every PR - # after it in the push below its deploy column. Archived cards are - # intentionally out of the deploy flow, so skip them cleanly. - ARCHIVED=$(echo "$RESP" | jq -r --arg n "$PROJECT_NUMBER" '.data.repository.pullRequest.projectItems.nodes[]? - | select(.project.number == ($n | tonumber)) | .isArchived' 2>/dev/null | head -1) - if [ "$ARCHIVED" = "true" ]; then - echo "::notice::#$prnum project item is archived -- skipping (out of deploy flow)" - continue - fi - - CURRENT_STATUS=$(echo "$RESP" | jq -r --arg n "$PROJECT_NUMBER" '.data.repository.pullRequest.projectItems.nodes[]? - | select(.project.number == ($n | tonumber)) | .status.name // ""' 2>/dev/null | head -1) - if [ "$TARGET_RANK" -gt 0 ] && [ "$(rank "$CURRENT_STATUS")" -ge "$TARGET_RANK" ]; then - echo "::notice::#$prnum already at '${CURRENT_STATUS:-none}' (>= '$STATUS_NAME') -- not demoting" - continue - fi - - echo "-> PR #$prnum: Deploy env = $DEPLOY_ENV" - update_field "$DEPLOY_FIELD" "$DEPLOY_OPT" "Deploy env=$DEPLOY_ENV" - - if [ "$SKIP_STATUS" != "1" ]; then - echo "-> PR #$prnum: Status = $STATUS_NAME" - update_field "$STATUS_FIELD" "$STATUS_OPT" "Status=$STATUS_NAME" - fi - done - - # NO ISSUE ADVANCEMENT HERE. This workflow advances PRs only. - # - # It used to also advance the issues each promoted PR closes (backend#1600), - # because the closure router parked a PR-closed issue at `On dev` and nothing - # ever moved it when the code shipped -- it drifted permanently (2026-08-06: - # all 20 drifted cards were closed issues, 0 PRs). That was a real problem and - # #1600 was right to fix it; marching the card through the deploy columns was - # the wrong remedy. - # - # backend#2722 sends a completed issue straight to `Done` in - # kanban-closure-router.yml instead. `Done` is terminal, so there is nothing - # left to drift, and kanban-archive sweeps it off the board daily -- which - # answers #1600 more completely than advancing ever did. With no issue parked - # in a deploy column, this block had nothing left to advance; kept, it would - # pull those cards straight back in and undo the other half of the fix. - # - # Worth knowing what was deleted, because it was hard-won and should be - # reused rather than rewritten if issue advancement is ever needed again: - # the removed loop resolved closingIssuesReferences CROSS-REPO, failed CLOSED - # on a lookup error so a rate limit could not read as "closes no issues" - # (.github#166) while still skipping a non-PR number quietly (.github#181), - # and refused to advance a still-OPEN closing issue (.github#168) -- PRs merge - # to develop rather than the default branch, so GitHub does not auto-close - # them and the reference list is full of in-progress cards. - - if [ "$RUN_FAILED" -ne 0 ]; then - echo "::error::One or more project items failed to update (see warnings above)." - exit 1 - fi diff --git a/.github/workflows/blocked-gate-selftest.yml b/.github/workflows/blocked-gate-selftest.yml deleted file mode 100644 index 4961ea2..0000000 --- a/.github/workflows/blocked-gate-selftest.yml +++ /dev/null @@ -1,67 +0,0 @@ -name: Blocked gate selftest - -# blocked-gate.yml is a `workflow_call` reusable, so it cannot host a test job of -# its own — this workflow runs the marker table instead. Same split as -# version-bump-gate-selftest.yml, for the same reason. -# -# WHY IT EXISTS: the matching IS the gate. Everything else here is four lines of -# YAML; the entire risk lives in whether `blocked on X` fires and `unblocked`, -# `blocker`, `threshold`, `holder` and `wipe` do not. A false negative costs what -# we have today. A FALSE POSITIVE costs the gate itself — it fires on somebody's -# "unbreak dev + staging ingestion" title, gets called noise, and is switched off -# inside a week, at which point it catches nothing forever. house-rules.sh names -# that failure mode in its own design notes: a checker that cries wolf gets -# switched off. So the selftest is weighted towards false friends, and every -# blocked title in it is copied verbatim from a real tracebloc PR. -# -# The gate is consumed at @main by every repo that wires it up, so a regression -# in the matching reaches all of them on its first successful run. -# -# Every title below is a LITERAL written by us. No `${{ }}` appears in any -# `run:` here, and the gate itself never interpolates a PR title either — it -# reads the event payload from GITHUB_EVENT_PATH. - -on: - pull_request: - paths: - - scripts/blocked-marker.py - - scripts/tests/blocked-marker-selftest.py - - .github/workflows/blocked-gate.yml - - .github/workflows/blocked-gate-selftest.yml - push: - branches: [main, develop, staging] - paths: - - scripts/blocked-marker.py - - scripts/tests/blocked-marker-selftest.py - - .github/workflows/blocked-gate.yml - - .github/workflows/blocked-gate-selftest.yml - -permissions: - contents: read - -concurrency: - group: blocked-gate-selftest-${{ github.ref }} - cancel-in-progress: true - -jobs: - selftest: - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: '3.12' - # No dependencies on purpose: re, json and argparse only. A gate that can - # be broken by a pip resolution is a gate that blocks merges on a bad day - # at PyPI. - - run: python scripts/tests/blocked-marker-selftest.py - - # The gate must not fire on the PR that ships it. A self-reference bug — - # matching the WORD "blocked" in a path or filename rather than in prose — - # would show up here and nowhere else in the table. - - name: The gate does not fire on this workflow's own filenames - run: | - set -euo pipefail - python scripts/blocked-marker.py \ - --title "ci(gate): add blocked-gate.yml and blocked-marker.py" diff --git a/.github/workflows/blocked-gate.yml b/.github/workflows/blocked-gate.yml deleted file mode 100644 index 57c2898..0000000 --- a/.github/workflows/blocked-gate.yml +++ /dev/null @@ -1,73 +0,0 @@ -name: Blocked gate - -# Reusable workflow. Called from each active repo on pull_request events. -# Fails when a PR declares itself blocked — via the org-wide `blocked` label, -# or via a marker the author wrote into the title. -# -# WHY (backend#1729, backend#1752) -# data-ingestors#468 was titled "... (D10) [blocked on S2]". It collected THREE -# approvals and was merged by its author 105 seconds after S2 merged. The author -# did wait for the blocker they had written down — but "S2 merged" was never the -# real precondition, and nothing could tell the difference. dev and staging -# ingestion broke within hours and stayed broken for a day. -# -# The dependency was recorded as PROSE, in a place nothing reads. This makes the -# same sentence a merge gate. Two other PRs carry the shape right now: -# client#490 ("HOLD until v0.8.0 image") and client-runtime#192 ("DO NOT MERGE"). -# -# WHAT IT DOES NOT DO -# It cannot tell whether the blocker is genuinely resolved — #468's author -# satisfied the sentence they wrote and the real condition was still unmet. All -# this guarantees is that removing the marker is a deliberate act by someone who -# has looked. That is the whole claim; it is not "PRs are never merged early". -# -# NO OVERRIDE LABEL, deliberately. fr-gate has `skip-fr-gate` because satisfying -# it can take a release cycle. Here the fix is editing your own title, so an -# escape hatch would only ever be used to skip the thinking. -# -# The caller's `pull_request` trigger list must include `edited`, `labeled`, and -# `unlabeled`. `edited` so a title changed to add/remove a marker re-evaluates -# (or the gate is decided by whatever the title said when the PR opened); -# `labeled`/`unlabeled` so the `blocked` label added or removed after open also -# re-runs the gate — without them the label half stays green until the next -# synchronize. (Bugbot #229; version-bump-gate.yml documents the same for its -# own label path.) -# -# INJECTION: the PR title is attacker-controlled text. It is never interpolated -# into a shell command — no `${{ github.event.pull_request.title }}` appears -# anywhere here. The script reads the event payload from GITHUB_EVENT_PATH -# itself, so the title never transits the shell at all. - -on: - workflow_call: - -permissions: - # contents: read only — the script reads GITHUB_EVENT_PATH, never the API, so a - # wider grant would exceed a minimal (contents:read) caller and fail the - # reusable at startup with no jobs, like the other secretless reusables - # (code-quality.yml, version-bump-gate.yml). (Bugbot #229.) - contents: read - -jobs: - gate: - name: blocked - runs-on: ubuntu-24.04 - timeout-minutes: 5 - steps: - # Same shape as code-quality.yml's shared-checker step: the script lives - # here, not in the calling repo, so the reusable fetches it. `.github` is - # public, so this needs no token. - - name: Check out the shared marker checker - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: tracebloc/.github - # Same repo this workflow lives in, so `main` is the version that - # matches a `@main` caller. - ref: main - path: .blocked-gate-tools - persist-credentials: false - - # No API call and nothing to rate-limit: the event payload already carries - # the title and labels, and a reusable sees the caller's payload. - - name: Check for a blocked marker - run: python3 .blocked-gate-tools/scripts/blocked-marker.py diff --git a/.github/workflows/bricked-prs-selftest.yml b/.github/workflows/bricked-prs-selftest.yml deleted file mode 100644 index 182d1bf..0000000 --- a/.github/workflows/bricked-prs-selftest.yml +++ /dev/null @@ -1,46 +0,0 @@ -name: Bricked PRs selftest - -# The watcher's product is a DISTINCTION, not a list: a context absent because -# it will never report, versus absent because the run has not started, versus a -# branch that could not be read. Get any of those wrong and the report is either -# ignorable or misleading -- both were observed while building it (backend#1721), -# so the paths are asserted rather than trusted. -# -# Same shape as blocked-gate-selftest.yml: offline, no token, path-filtered -# because it only needs to run when the thing it tests changes. - -on: - pull_request: - paths: - - scripts/bricked-prs.py - - scripts/tests/bricked-prs-selftest.py - - .github/workflows/bricked-prs.yml - - .github/workflows/bricked-prs-selftest.yml - push: - branches: [main, develop, staging] - paths: - - scripts/bricked-prs.py - - scripts/tests/bricked-prs-selftest.py - - .github/workflows/bricked-prs.yml - - .github/workflows/bricked-prs-selftest.yml - -permissions: - contents: read - -concurrency: - group: bricked-prs-selftest-${{ github.ref }} - cancel-in-progress: true - -jobs: - selftest: - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: '3.12' - # bricked-prs.py imports caller-drift.py for the protection reader, and - # that module hard-fails without PyYAML by design. - - run: pip install --quiet pyyaml - - run: python scripts/tests/bricked-prs-selftest.py diff --git a/.github/workflows/bricked-prs.yml b/.github/workflows/bricked-prs.yml deleted file mode 100644 index 39539ca..0000000 --- a/.github/workflows/bricked-prs.yml +++ /dev/null @@ -1,92 +0,0 @@ -name: Bricked PRs - -# A required status check that never reports leaves a PR approved, with nothing -# red to point at, and permanently unmergeable (backend#1721). It is the one CI -# failure mode with NO red signal at all -- nobody is notified, and no reviewer -# sees a problem, because there is no failure, only an absence. So it needs a -# watcher; nothing inside a PR can detect it. -# -# Runs in tracebloc/.github only, like the other org-wide crons. - -on: - schedule: - # Every four hours. The failure is not urgent -- a bricked PR stays bricked - # -- but it is invisible, so the cost is the hours a human spends not - # realising. Four hours bounds that without adding noise. - - cron: "17 */4 * * *" - workflow_dispatch: {} - -permissions: - contents: read - -concurrency: - group: bricked-prs - cancel-in-progress: false - -jobs: - audit: - name: Required checks that never report - runs-on: ubuntu-latest - timeout-minutes: 20 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # Without this the `pip install` below hits PEP 668 on ubuntu-latest - # (24.04): the runner's Python is externally managed, pip refuses, the step - # fails and THE AUDIT NEVER RUNS. A scheduled guard that cannot start is - # worse than none — nothing reports, and silence reads as "no bricked PRs". - # bricked-prs-selftest.yml already does this before the same install; this - # is that shape, not a new idea (Bugbot, #243). - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: '3.12' - - name: Install PyYAML - run: pip install --quiet pyyaml - # Authenticates as the tracebloc-release-train App (backend#2036) instead of - # one human's PAT. `owner:` yields an ORG-scoped installation token, which is - # what an org-wide read needs. No fallback to the PAT: a fallback would let a - # broken App path keep working silently. - # - # SCOPES WERE MEASURED, NOT DERIVED (backend#2157, 2026-08-24). This audit was - # exempted from `scripts/mint-scope.py` because the protection endpoints are - # known to DEGRADE rather than error -- a narrower token can answer 200 with - # fewer fields, and a missing field reads as "not configured", which would make - # a protected branch look unprotected on a green run. Reading the docs cannot - # settle that, so it was run: five token scopes against the real endpoints, then - # `scripts/bricked-prs.py` itself over backend + client-runtime + client + - # .github. At the five scopes below the script produces output IDENTICAL to the - # full grant -- 4 findings, `0 COULD NOT AUDIT`, exit 1. - # - # Two of the five are non-obvious and BOTH were verified by removal: - # - # administration: read -- classic protection. Dropping it does not silently - # empty the required set; `branches/{b}/protection` answers 403, which - # read_protection() turns into an error (12 COULD NOT AUDIT, exit 2). It - # also keeps the 404 on an UNPROTECTED branch reading "Branch not - # protected", the fact read_protection() relies on to tell "no classic - # protection" apart from "could not read". - # actions: read -- NOT for any Actions API. `gh pr list --json - # statusCheckRollup` resolves `commit.status` underneath, and without - # actions:read GraphQL refuses that subfield on a PRIVATE repo while every - # other scope here is untouched (3 COULD NOT AUDIT, exit 2). Nothing in the - # docs connects the two; only removing it shows the link. - # - # The measurement drops contents from WRITE to read and removes issues:write and - # organization-projects:write entirely. Do not add them back to "be safe": this - # token reads, and the App holds bypass_reviews fleet-wide. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - permission-administration: read # classic branch protection - permission-actions: read # statusCheckRollup -> commit.status (see above) - permission-checks: read # check-suites, the head-age clock - permission-contents: read # branch list, commit fallback clock - permission-pull-requests: read # the open-PR list and its rollup - - name: Audit - env: - # Needs to read branch protection, rulesets and PRs across the org. - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: python3 scripts/bricked-prs.py diff --git a/.github/workflows/bugbot-gate-caller.yml b/.github/workflows/bugbot-gate-caller.yml deleted file mode 100644 index 67181cc..0000000 --- a/.github/workflows/bugbot-gate-caller.yml +++ /dev/null @@ -1,107 +0,0 @@ -name: Bugbot gate - -# THIN CALLER. The FLEET ROLLOUT of tracebloc/backend#2284 -- step 2 of the -# three-step arming order the reusable's own header sets out, continued from the -# pilot onto the rest of the org. -# -# WHERE THIS SITS. `bugbot-gate.yml` reached `.github`'s `main` in #305/#312 and -# then executed NOWHERE, because a reusable with no caller runs on nothing: every -# `repo-inventory.yml` row read `exempt`. claude-skills#36 gave it its first -# caller on 2026-08-25, on one low-traffic private repo chosen by measured PR -# volume. That pilot is the evidence this file rides on -- the gate was observed -# running against real PRs before going fleet-wide, which is CLAUDE.md rule 4 -# (arm while green) done in the only order that lets a misbehaviour be attributed -# to one repo instead of twenty. This is the other nineteen. -# -# THIS IS ADVICE, NOT A GATE, AND SAYING SO IS THE POINT -- it is the whole of -# what backend#2284 asks for at this step. `bugbot / review` is NOT added to this -# repo's required status contexts, and the PR carrying this file touches no -# branch protection at all. Step 3 is deliberately NOT taken here, and one -# measurement is why: -# -# BUGBOT DOES NOT REVIEW DEPENDABOT PRs. Sampled 2026-08-25 over the last 30 -# PRs each of .github, cli, release-train, tracebloc-website, averaging-service -# and backend: every NON-DRAFT pull request with no `Cursor Bugbot` check run -# on its head was authored by dependabot (cli#574, cli#575, -# tracebloc-website#510). The only other misses were drafts, which this gate -# passes by design. Bugbot re-runs only on a push or an explicit `bugbot run` -# comment, so a REQUIRED `bugbot / review` would park every Dependabot PR at a -# red check with no route to green. That question is unanswered, so the verdict -# is reported and nothing is required. As ADVICE the same PRs still go red -- -# after the callee's 900s wait, visible and costing only runner minutes, which -# is the honest way to leave a question open. -# -# `repo-inventory.yml` IS NOT TOUCHED BY THIS PR. `.github`'s caller state is -# read from its audit branch over the API, so a caller and its `required` row -# cannot land together: the row would be checked against a branch the caller is -# not on yet. Caller first, entry after -- the two-step blocked-gate and -# backend#2396 were both forced into. BETWEEN THE TWO, a caller sitting against -# an `exempt` row IS the stale-exemption finding and the org audit goes red. That -# window is the cost of this order rather than an oversight (the alternative is a -# PR that can never go green), and FLIPPING THE ROW TO `required` IS THE REQUIRED -# FOLLOW-UP -- one PR against tracebloc/.github for the whole fleet. -# -# NO INPUTS PASSED. The callee declares four -- `min-severity` (default `high`), -# `wait-seconds` (900), `poll-seconds` (20), `quality-ref` (`main`) -- and every -# one is left at its default, `min-severity` DELIBERATELY. Passing `high` -# explicitly would restate the callee's own default in twenty files, so changing -# the fleet threshold would take twenty PRs and would silently half-apply if one -# were missed: derive, never restate (CLAUDE.md rule 1). `high` is also the right -# threshold today precisely because it is green fleet-wide -- -# `required_conversation_resolution` is true on every measured branch, so no -# mergeable PR carries an open finding of ANY severity and starting stricter -# would buy nothing while risking a red gate on day one. A caller may only pass -# inputs the `@main` callee declares; passing one it does not have kills the run -# at `startup_failure`. Verified before writing this line: `bugbot-gate.yml` is -# blob 936771bb on `.github`'s `main` and `develop` alike, and declares all four. -# -# NO `paths:` FILTER, and none may be added. A path-filtered check never reports -# on a PR the filter skips, so once required it parks that PR at "Expected -- -# waiting for status" forever. This org has hit that twice (client#665, -# pii-gate/pii-check); the reusable's header, code-quality.yml's and -# selftests.yml's all carry the same warning. -# -# `ready_for_review` IS LOAD-BEARING, not boilerplate. The gate deliberately -# PASSES a draft -- a draft cannot merge, and Bugbot's behaviour on drafts is not -# this gate's business -- so leaving that type out means the exemption is never -# lifted and the check stays permanently green on anything opened as a draft. -# -# NO `secrets: inherit` -- RFC-BACKEND-1405 Q5. The callee runs on `github.token` -# with exactly the three read scopes granted below; inheriting would hand it -# every secret this repo holds, for no gain. -# -# NO `workflow_dispatch`. The callee reads `github.event.pull_request.number`, -# which a dispatch does not carry -- the run would abort with "PR_NUMBER must be -# a number" rather than checking anything. When the failure message says to -# RE-RUN this check after resolving a thread, it means "Re-run jobs" on the -# existing run, which replays the original pull_request payload. (Resolving a -# thread is a `pull_request_review_thread` event, which actionlint 1.7.12 -- a -# required check in tracebloc/.github -- does not know, so no caller can trigger -# on it yet.) - -on: - pull_request: - types: [opened, reopened, synchronize, ready_for_review] - -# Per-PR: `github.ref` is `refs/pull//merge` on a pull_request event. The -# callee polls for up to 900s, so without this a superseded run keeps a poll -# alive against a head nobody is merging. -concurrency: - group: bugbot-gate-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -# All three are load-bearing and a called workflow cannot hold more than its -# caller: `checks: read` reads the head's check runs, `pull-requests: read` -# reads the review threads, `contents: read` checks out the shared checker. -permissions: - contents: read - checks: read - pull-requests: read - -# The job id below and the callee's job id (`review`) together are the check -# CONTEXT name, `bugbot / review` -- which is the string branch protection would -# key on at step 3. Required checks reference job ids, never filenames, so this -# file may be renamed and that name may not. -jobs: - bugbot: - uses: tracebloc/.github/.github/workflows/bugbot-gate.yml@main diff --git a/.github/workflows/bugbot-gate.yml b/.github/workflows/bugbot-gate.yml deleted file mode 100644 index dc71784..0000000 --- a/.github/workflows/bugbot-gate.yml +++ /dev/null @@ -1,172 +0,0 @@ -name: Bugbot review gate - -# Reusable workflow. Makes Cursor Bugbot's review a GATE instead of advice -# (tracebloc/backend#2284). `scripts/bugbot-gate.py` carries the measurements and -# the reasoning; this file is the wiring, and only says what the script cannot. -# -# THE SHORT VERSION OF WHY THIS IS NOT "ADD `Cursor Bugbot` TO THE REQUIRED -# CONTEXTS". Measured 2026-08-22: Bugbot emits `success` when clean and `neutral` -# when it has findings, and never `failure`. That makes requiring the context a -# dichotomy with two bad horns: -# -# * if `neutral` SATISFIES a required context -- GitHub's documented behaviour -# -- requiring it gates nothing, because `failure` never occurs; -# * if it does NOT, requiring it permanently bricks every PR that ever received -# a finding, because Bugbot re-runs only on a push or an explicit -# `bugbot run`, so resolving a finding can never turn the check green again -# -- and a promotion PR may not be pushed to at all. client#786 and -# frontend-app#863 both merged to `main` on 2026-08-21 carrying exactly that -# state (`neutral` on the merged head, one resolved Medium beneath). -# -# So the verdict is REPORTED here and the decision is derived from the threads. -# -# ARMING THIS IS THREE STEPS, IN THIS ORDER, AND THE ORDER IS THE POINT. -# -# 1. this file reaches `main`. Every caller in this org pins -# `tracebloc/.github/...@main` (RFC-BACKEND-1405 Q3), so a caller added -# before the reusable is ON main references something main does not have and -# dies with a `startup_failure` -- a red check on the very PR introducing a -# gate. code-quality-caller.yml records the identical sequencing for its -# `action-pins` input: the job went to develop in #159 and was armed only -# once `main` carried it. That is why the PR adding THIS file adds no caller. -# 2. a caller is added, starting with tracebloc/.github itself. The gate is -# ADVICE at this point, and saying so plainly is the whole of backend#2284. -# 3. the `bugbot / review` context is added to branch protection, once step 2 -# has been observed green on real PRs. -# -# Never the reverse. backend#1976's lesson is that a contract claiming a context -# reality lacks is itself a finding; and a required context no workflow reports -# leaves every PR waiting forever (client#665, pii-gate/pii-check). Arm while -# green, then let the contract claim it. -# -# The TEST tier needs none of that and is armed from the start: -# `scripts/tests/bugbot-gate-selftest.py` and `-mutations.py` both run inside -# `selftests`, ALREADY a required context on develop/staging/main -- the same -# trick selftests.yml uses to arm a guard with no branch-protection edit. -# -# NO `paths:` FILTER, and the caller must not add one. A required check that is -# path-filtered never reports on a PR the filter skips, so the PR waits at -# "Expected -- waiting for status" forever. This org has hit that twice -# (client#665, pii-gate/pii-check); code-quality.yml's and selftests.yml's -# headers both warn about it. -# -# WHAT A CALLER MUST TRIGGER ON: `pull_request` with at least -# `[opened, reopened, synchronize, ready_for_review]`. `ready_for_review` is -# load-bearing, not boilerplate -- this gate deliberately PASSES a draft (a draft -# cannot merge, and Bugbot's behaviour on drafts is not this gate's business), so -# leaving that type out means the exemption is never lifted and the gate is -# permanently green on a PR that was opened as a draft. -# -# THE TRIGGER THIS WANTS AND CANNOT HAVE YET. Resolving a Bugbot thread is not a -# `pull_request` event, so `pull_request_review_thread: [resolved]` is the right -# trigger for the severity half of this gate. actionlint 1.7.12 -- a REQUIRED -# check in tracebloc/.github, run with no config file and no `-ignore` on purpose -# -- does not know that event name and rejects the workflow, so no caller can -# carry it without landing a red required check. Measured against the pinned -# binary: zero occurrences of `review_thread`. -# -# The consequence is written into the failure message rather than left to be -# discovered: resolve the thread, then RE-RUN this check. Not "push a commit", -# which is what a stale gate otherwise teaches. Adding the trigger when -# actionlint learns the event is a follow-up on backend#2284. -# -# THIS JOB WAITS, on purpose. Its central claim is that Bugbot reviewed the -# CURRENT head, which is false for the first minutes after every push -- so it -# polls. Measured over 40 Bugbot runs: p50 164s, p90 332s, max 635s. The wait -# runs concurrently with Bugbot's own work, so it adds latency only when Bugbot -# is slower than usual. `timeout-minutes` is deliberately above the script's own -# budget so the script reports WHY it gave up instead of the runner killing it -# with no message. - -on: - workflow_call: - inputs: - min-severity: - description: >- - Lowest Bugbot severity that blocks when its thread is open: one of - low | medium | high | critical. Anything outside that list is refused, - not defaulted. `high` is the default because it is green across the - fleet today -- `required_conversation_resolution` already blocks any - open thread, so no mergeable PR carries an open finding of any - severity, and starting stricter would buy nothing while risking a red - gate on day one. - type: string - default: "high" - wait-seconds: - description: >- - How long to wait for Bugbot to deliver a terminal verdict on the head. - Default 1500: the 900 that preceded it was ~1.4x the slowest of 40 - measured runs (635s), and Bugbot then took 15m22s on - client-runtime#544 (backend#3530) -- the gate gave up at 15m08s and - the red verdict stood over a head Bugbot passed fourteen seconds - later. Re-measured 2026-09-10 over 57 completed Cursor Bugbot runs on - five repos: p50 3.6m, p90 7.2m, p99 9.9m, max 10.3m, plus that 15.4m - outlier; 1500 s is ~1.6x the slowest observed. Five of the 24 gate - failures in the same window were this timeout, not a finding. - type: number - default: 1500 - poll-seconds: - description: "Gap between polls while waiting for Bugbot." - type: number - default: 20 - quality-ref: - description: >- - Ref of tracebloc/.github the checker is taken from. Callers pin this - workflow at `@main`, so `main` is the matching version. Override only - to test a change to the checker before it merges. - type: string - default: "main" - -# The CALLER must grant at least these three, or the run fails at startup with -# no jobs -- a called workflow cannot hold more than its caller. `checks: read` -# and `pull-requests: read` are both load-bearing: the first reads the head's -# check runs, the second reads the review threads. `contents: read` is for the -# checkout of the checker. -permissions: - contents: read - checks: read - pull-requests: read - -jobs: - review: - name: review - runs-on: ubuntu-latest - # Above the script's own 1500s budget on purpose: the script must be the thing - # that reports a timeout, with the measured latencies in the message. A - # runner-level kill produces no explanation at all. - timeout-minutes: 30 - steps: - # Same shape as code-quality.yml and blocked-gate.yml: the checker lives - # here, not in the calling repo, so the reusable fetches it. `.github` is - # public, so this needs no token. - - name: Check out the shared checker - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: tracebloc/.github - ref: ${{ inputs.quality-ref }} - path: .bugbot-gate-tools - persist-credentials: false - - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: '3.12' - - # Nothing is pip-installed: the checker imports only the standard library - # and shells out to `gh`, which is preinstalled on the runner. Asserted - # rather than assumed -- the selftest imports the module and would fail on - # a missing import, and it runs with no pip step in `selftests.yml`. - - name: Bugbot review gate - env: - # `github.token`, not `secrets.inherit`: this needs only the scopes - # declared above, and inheriting would hand it every secret the caller - # holds for no gain (RFC-BACKEND-1405 Q5). - GH_TOKEN: ${{ github.token }} - REPO: ${{ github.repository }} - PR_NUMBER: ${{ github.event.pull_request.number }} - MIN_SEVERITY: ${{ inputs.min-severity }} - WAIT_SECONDS: ${{ inputs.wait-seconds }} - POLL_SECONDS: ${{ inputs.poll-seconds }} - # No interpolation of any PR-controlled string into this shell: the - # checker reads everything it needs from the API itself, so a finding - # title or a branch name never transits a command line. - run: python3 .bugbot-gate-tools/scripts/bugbot-gate.py diff --git a/.github/workflows/caller-drift.yml b/.github/workflows/caller-drift.yml deleted file mode 100644 index 044c0e7..0000000 --- a/.github/workflows/caller-drift.yml +++ /dev/null @@ -1,483 +0,0 @@ -# Caller-inventory drift guard (tracebloc/backend#1415). -# -# WHY THIS EXISTS: nothing in the org detected a missing caller. -# merge-settings-drift.yml reads three booleans. kanban-reconcile.yml:436 probes a -# single filename to decide board scope, so a 403 there is indistinguishable from -# "repo not tracked". Eight repos drifted unnoticed, and e2e-test-agent#1 closed -# without routing because no closure caller exists in that repo at all. A caller -# that is simply absent produces no run, no annotation and no red check — the -# absence of a signal is not a signal, which is why it went unnoticed for months. -# -# WHAT IT COMPARES: /repo-inventory.yml, the single source of truth, against every -# active repo in the org. Matching is on the resolved `uses:` value of a parsed -# workflow, never on filename — two filename conventions are mixed within nearly -# every repo, kanban-closure-router's callers match neither, and code-quality.yml:60 -# is a commented-out example that a grep would count as a caller. It reads the -# develop-first branch: twelve of twenty repos default to main/master while work -# lands on develop, so a default-branch audit under-reports anything in flight. -# -# REPORT-ONLY on schedule, PR and push. The `create-prs` dispatch input is the only -# path that writes, and it writes ONE family: copies marked `required` in the -# inventory that are missing or drifted (backend#1608 item 4). Callers are not -# generated — their content is repo-specific, measured 2026-08-12 as eight different -# code-quality-caller.yml files across eight repos — and protection/rulesets are API -# settings no commit can change. Entries marked `divergent` or `exempt` carry a -# written reason and are never rewritten. -# -# `quality_files` (CLAUDE.md, .cursor/BUGBOT.md — backend#1608 increment 5) is also -# NOT remediable, and unlike the others that is a deliberate refusal rather than a -# technical limit: a commit could trivially create the file. But the family asserts -# that per-repo GUIDANCE exists, and auto-generating a placeholder would turn every -# finding green while adding nothing a tool can use — a check that can no longer -# fail, which is the one outcome this repo's guards exist to prevent. These findings -# are closed by a human writing the file. -# -# TOKEN: report-only needs org-wide read. create-prs additionally needs Contents: RW -# and Pull requests: RW — AND, because the callers it writes live under -# `.github/workflows/`, the `workflow` scope (classic PAT) or fine-grained -# "Workflows: write". Contents: RW alone is REFUSED for that path, so every -# remediation PUT would fail closed and no PR would open. standards-sync.yml writes -# CLAUDE.md, not workflow files, so it needs no `workflow` scope — the one scope -# that differs from what it exercises. (Bugbot #227.) -# -# FAIL-CLOSED: every read either yields a value or is recorded as unreadable, and -# one unreadable repo fails the run. There is no path from a 403, a rate limit, an -# unparseable workflow, a truncated tree or a missing inventory key to a green run. -# The final step fails on anything other than a literal exit code of 0, including -# an absent one — a skipped or crashed audit step must not read as all-clear. -# -# All interpolations reach shell through `env:` and are read as quoted variables, -# never spliced into a command line. - -name: Caller inventory drift - -on: - schedule: - - cron: '30 6 * * 1' # Mondays 06:30 UTC, right after merge-settings-drift - workflow_dispatch: - inputs: - create-prs: - description: 'Open remediation PRs for drifted/missing REQUIRED copies' - type: boolean - default: false - required: false - # Edits to the inventory or the guard are validated against reality on the PR - # that makes them, so a wrong inventory cannot reach develop unchallenged. - pull_request: - paths: - - repo-inventory.yml - - scripts/caller-drift.py - - scripts/tests/caller-drift-selftest.py - - .github/workflows/caller-drift.yml - - .github/workflows/conformance-gate.yml - push: - branches: [main, develop] - paths: - - repo-inventory.yml - - scripts/caller-drift.py - - scripts/tests/caller-drift-selftest.py - - .github/workflows/caller-drift.yml - - .github/workflows/conformance-gate.yml - -permissions: - contents: read - issues: write # the drift report lands on a tracking issue - -concurrency: - group: caller-drift-${{ github.ref }} - cancel-in-progress: false - -jobs: - # The guard's own fail-closed paths, asserted offline with a stubbed `gh`: a 403, - # a rate limit, a truncated tree, an unparseable workflow and a missing inventory - # key must each produce a FAILURE, never "no caller found". The audit needs this, - # so a regression here stops the audit from reporting at all rather than letting - # it report all-clear from a broken comparison. - selftest: - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: '3.12' - - run: python -m pip install --quiet --disable-pip-version-check 'pyyaml==6.0.2' - - run: python scripts/tests/caller-drift-selftest.py - - audit: - needs: selftest - runs-on: ubuntu-latest - timeout-minutes: 30 - env: - # Job-level so BOTH reporting steps below resolve it. Defined in one step's - # env: it was undefined in the other, and under `set -u` that step dies — - # loud rather than silent, but the drift comment would never post. - WATCHDOG_ISSUE: '1781' - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: '3.12' - - - name: Install PyYAML - # The guard parses workflows as YAML rather than grepping them, so there is - # no degraded mode worth having. If this step fails, the audit step is - # skipped, its exit_code output is empty, and the final step fails on the - # empty value rather than letting a skipped audit look clean. - run: python -m pip install --quiet --disable-pip-version-check 'pyyaml==6.0.2' - - # THE ONE AUDIT THAT DOES NOT AUTHENTICATE AS THE APP, and this is a decision - # rather than an omission (backend#2036, decided by Lukas 2026-08-17). - # - # Its three siblings -- standards-sync, merge-settings-drift, bricked-prs -- - # moved to the tracebloc-release-train App. This one cannot, because it is the - # only audit that asserts ruleset BYPASS ALLOWLISTS, and GitHub returns - # `bypass_actors` only to a caller with WRITE access to the ruleset: - # - # "To prevent leaking sensitive information, the bypass_actors property is - # only returned if the user making the API request has write access to the - # ruleset." - # - # So no read-level permission can fix it. `administration: read` was granted - # and measured: it fixed all 52 branch-protection reads and left every - # `bypass_actors` withheld. The only sufficient grant is - # `administration: write`, which would give an App invoked by ~14 workflows on - # every PR and push the power to REWRITE branch protection and every ruleset - # across the fleet -- including the `v*` tag trust root whose bypass list this - # audit exists to police. Granting write over the trust root in order to read - # who may bypass it makes the auditor one of the actors it audits. - # - # So this stays on PROJECTS_KANBAN_TOKEN, deliberately, and #2036 closes with - # the PAT alive for exactly this one consumer. That is the SAFEST place for a - # privileged credential to remain: a weekly cron, never event-triggered, so no - # outside contributor can influence when it runs, and it never writes. - # - # DO NOT "finish" #2036 by pointing this at the App. If a future GitHub release - # returns `bypass_actors` to a read-scoped caller, migrate then and delete this - # comment. Until it does, a green audit here means the allowlist was actually - # read -- which under the App it would not be (see the `bypass_present` note in - # caller-drift.py, and the 16 unreadable records that change produced). - - name: Check the audit token is present - # Guards an EMPTY token reaching the audit, which is what makes this guard - # fail open: the org listing returns nothing and every check passes against - # an empty scope. - env: - GH_TOKEN: ${{ secrets.PROJECTS_KANBAN_TOKEN }} - run: | - set -euo pipefail - if [ -z "${GH_TOKEN:-}" ]; then - echo "::error::PROJECTS_KANBAN_TOKEN is empty. Without it the org listing" \ - "returns nothing and every check would pass against an empty scope." >&2 - exit 1 - fi - - - name: Compare every active repo against repo-inventory.yml - id: audit - env: - GH_TOKEN: ${{ secrets.PROJECTS_KANBAN_TOKEN }} - # The event is asserted explicitly rather than inferred from `inputs` - # being unset — same expression standards-sync.yml uses. Writing to - # twenty repos is not something a cron may decide to do, and a - # PR-triggered audit that wrote to the fleet would be a supply-chain - # hole rather than a convenience. - CREATE_PRS: ${{ github.event_name == 'workflow_dispatch' && inputs.create-prs == true }} - run: | - set -uo pipefail - remediate="" - if [ "${CREATE_PRS:-false}" = "true" ]; then - remediate="--create-prs" - echo "Remediation ENABLED: required copies that are missing or drifted" - echo "will get a PR. divergent/exempt entries are never rewritten." - fi - # The exit code is recorded rather than raised so that the reporting step - # below still runs on a failure. The step itself therefore always exits 0 - # — the run is failed by the final step, from this recorded value. - set +e - python scripts/caller-drift.py \ - --inventory repo-inventory.yml \ - --source-dir . \ - ${remediate} - code=$? - set -e - echo "exit_code=$code" >> "$GITHUB_OUTPUT" - echo "The audit exited $code (0 clean, 1 drift, 2 could not evaluate)." - - - name: Rewrite the conformance matrix on the watchdog issue - # THE ONE SCREEN (backend#1608 item 3). - # - # The comment step below fires only on drift, deliberately — an all-clear - # per run trains people to ignore the issue (backend#1344). The cost of - # that, until now, was that a CONFORMANT fleet published nothing at all: - # the matrix existed only in a step summary on a run nobody opens, so - # "where does the fleet stand?" was still answered by running the script by - # hand. Comments are the history; this body is the current state. - # - # Rewritten on EVERY scheduled/manual run regardless of outcome, which is - # what makes staleness meaningful: the body carries the run timestamp and - # the audit is weekly, so a timestamp older than ~8 days means the audit - # itself stopped and the fleet is UNKNOWN. That is the backend#1530 - # cron-watchdog contract, and it only holds if a red run rewrites the body - # too — otherwise a fleet that broke in January still shows January's - # green and looks merely stale-ish. - # - # `always()`, and NOT gated on exit_code: a run that could not evaluate - # must overwrite the previous green with UNKNOWN. Leaving the last good - # matrix in place is the fail-open this whole guard exists to refuse. - if: >- - always() - && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') - env: - GH_TOKEN: ${{ secrets.PROJECTS_KANBAN_TOKEN }} - CODE: ${{ steps.audit.outputs.exit_code }} - REPORT: ${{ steps.audit.outputs.report }} - EVALUATED: ${{ steps.audit.outputs.evaluated }} - UNREADABLE: ${{ steps.audit.outputs.unreadable }} - CALLER_UNREADABLE: ${{ steps.audit.outputs.caller_unreadable }} - PROTECTION_UNREADABLE: ${{ steps.audit.outputs.protection_unreadable }} - RULESET_UNREADABLE: ${{ steps.audit.outputs.ruleset_unreadable }} - LISTING_UNREADABLE: ${{ steps.audit.outputs.listing_unreadable }} - FINDINGS: ${{ steps.audit.outputs.findings }} - FINDINGS_TOTAL: ${{ steps.audit.outputs.findings_total }} - REMEDIATED: ${{ steps.audit.outputs.remediated }} - REMEDIATION_FAILURES: ${{ steps.audit.outputs.remediation_failures }} - RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - run: | - set -euo pipefail - code="${CODE:-}" - now="$(date -u '+%Y-%m-%d %H:%M UTC')" - - # NAME WHICH READ FAILED, once, for every verdict below. UNREADABLE is the merged list, so wording - # it as "repo(s) could not be read" announced a failed protection or - # ruleset read as a repo nobody looked at -- true count, false name - # (Bugbot #238). The script emits the split; use it. - what="" - [ "${CALLER_UNREADABLE:-0}" != "0" ] && what="${CALLER_UNREADABLE} repo(s) could not be read (caller/copy state UNKNOWN)" - if [ "${PROTECTION_UNREADABLE:-0}" != "0" ]; then - [ -n "$what" ] && what="$what; " - what="${what}${PROTECTION_UNREADABLE} branch-protection read(s) failed (protection state UNKNOWN)" - fi - if [ "${RULESET_UNREADABLE:-0}" != "0" ]; then - [ -n "$what" ] && what="$what; " - what="${what}${RULESET_UNREADABLE} ruleset read(s) failed (ruleset state UNKNOWN)" - fi - # The fourth bucket, and the reason the fallback below is not enough on - # its own: it fires only when `what` is ENTIRELY empty, so a listing gap - # MIXED with any other cause was silently dropped from the sentence rather - # than mis-named -- the same true-count-false-name class, one step further - # along (Bugbot, #278). Its fix is a token-visibility one -- widen whatever - # credential this audit runs as -- which no other clause would suggest. - if [ "${LISTING_UNREADABLE:-0}" != "0" ]; then - [ -n "$what" ] && what="$what; " - what="${what}${LISTING_UNREADABLE} declared repo(s) missing from the org listing (fleet coverage UNKNOWN)" - fi - # Fall back to the merged count rather than an empty sentence if a - # future read type is added and not surfaced here. The selftest asserts - # that every `*_unreadable` output the script emits IS surfaced above, so - # this is a backstop rather than the plan. - [ -z "$what" ] && what="${UNREADABLE:-0} read(s) failed" - # Built once: TWO verdicts quote it, and fixing only the first would - # leave the second still saying "repo(s) could not be read". - - case "$code" in - 0) verdict='✅ **Conformant** — every repo read, every entry matched.' ;; - 1) verdict='🔴 **Drift** — see the latest comment for the findings.' ;; - 2) verdict='⚠️ **Could not evaluate** — the audit did not complete cleanly. Fleet conformance is UNKNOWN; anything not read is NOT known to comply.' ;; - *) verdict="⚠️ **No result** — the audit exited \`${code:-}\`, which it should never do. Treat fleet conformance as UNKNOWN." ;; - esac - - # Exit 2 is not only "an unreadable repo": die() also exits 2 for a bad - # inventory or a failed org enumeration, where UNREADABLE is 0. Name the - # actual cause from the count instead of always blaming unread repos - # (Bugbot #227). - if [ "$code" = "2" ]; then - if [ "${UNREADABLE:-0}" != "0" ]; then - verdict="⚠️ **Could not evaluate** — ${what}. Anything not read is NOT known to comply. Fleet conformance is UNKNOWN." - else - verdict='⚠️ **Could not evaluate** — the audit could not run to completion (a bad inventory or a failed org enumeration; see the log). Fleet conformance is UNKNOWN.' - fi - fi - - # Exit 2 has two causes and they need different words. Without this, a - # dispatch whose remediation PRs failed to open is headlined "repos that - # could not be read are NOT known to comply" - a true sentence about a - # thing that did not happen, sending the reader to the wrong problem. - # (Bugbot, #227.) - if [ "${REMEDIATION_FAILURES:-0}" != "0" ]; then - if [ "${UNREADABLE:-0}" != "0" ]; then - verdict="⚠️ **Remediation failed** — ${REMEDIATION_FAILURES} repo(s) still carry their drift, and ${what} — NOT known to comply." - else - verdict="⚠️ **Remediation failed** — the fleet was read successfully, but ${REMEDIATION_FAILURES} repo(s) could not be remediated and still carry their drift." - fi - fi - - # backend#1965: a REMEDIATED run is green, but it is not 'every entry - # matched' -- drift was found and is still on the fleet until those PRs - # merge. Saying Conformant here would be the mirror of the bug this - # change fixes: a true exit code under a false headline. - if [ "$code" = "0" ] && [ "${REMEDIATED:-0}" != "0" ]; then - verdict="✅ **Remediated** — ${FINDINGS_TOTAL:-?} drift finding(s), every one with a PR open. The fleet is NOT yet conformant, it is fixed-pending-merge: the next run goes red again if those PRs are closed unmerged." - fi - - # The verdict must not be able to contradict the counts printed beside it. - # A clean exit code alongside a non-zero unreadable/findings count is not a - # green fleet, it is a broken audit — and "✅ every repo read" printed above - # "3 unreadable" is the most confidently wrong thing this issue could say. - # The script does not currently produce that pair, which is exactly why it - # is worth pinning: the headline is DERIVED from the numbers, so it stays - # true if the exit-code contract ever changes. - # FINDINGS is the UN-REMEDIATED count (backend#1965), which is exactly - # what keeps this guard meaningful after that change: a clean exit beside - # a finding nothing fixed is still a contradiction, a remediated one is - # not. Comparing the TOTAL here would make every successful --create-prs - # run report 'Inconsistent result ... fix the audit' -- trading a false - # red for a worse one, since that headline tells the reader to stop - # trusting the audit itself. - if [ "$code" = "0" ] && { [ "${UNREADABLE:-0}" != "0" ] || [ "${FINDINGS:-0}" != "0" ]; }; then - verdict="⚠️ **Inconsistent result** — the audit exited clean while reporting ${UNREADABLE:-?} unreadable and ${FINDINGS:-?} finding(s). Treat fleet conformance as UNKNOWN and fix the audit." - fi - - # shellcheck disable=SC2016 # backticks are markdown code spans in the - # issue body, not command substitution — hence the single quotes. - { - printf 'This issue is the org'"'"'s repo-conformance surface (backend#1608). The weekly\n' - printf 'audit in `tracebloc/.github` (`caller-drift.yml`) rewrites this body on every\n' - printf 'run; drift arrives separately as a **comment**. The body is the current\n' - printf 'state, the comments are the history.\n\n' - printf '**Fail-closed contract:** the audit runs **Mondays 06:30 UTC**. If the\n' - printf 'timestamp below is stale by more than ~8 days, the audit itself has died —\n' - printf 'that staleness is the alarm, and fleet conformance is UNKNOWN rather than\n' - printf 'fine. Do not close this issue.\n\n' - printf -- '---\n\n' - printf '### Last audit: %s\n\n' "$now" - printf -- '- Repos evaluated: **%s** (%s unreadable)\n' "${EVALUATED:-?}" "${UNREADABLE:-?}" - printf -- '- Findings: **%s**' "${FINDINGS_TOTAL:-${FINDINGS:-?}}" - if [ "${REMEDIATED:-0}" != "0" ]; then - printf -- ' (%s with a PR open, %s still un-remediated)' "${REMEDIATED}" "${FINDINGS:-?}" - fi - printf -- '\n' - printf -- '- Verdict: %s\n\n' "$verdict" - if [ -n "${REPORT:-}" ]; then - printf '%s\n\n' "$REPORT" - else - # No report body on a run that was supposed to produce one is itself a - # finding: say so here rather than leaving the previous matrix visible. - printf 'The audit produced **no report body**, so no matrix could be rendered.\n' - printf 'Fleet conformance is UNKNOWN for this run — see the run log.\n\n' - fi - printf '[Run log](%s)\n' "$RUN_URL" - } > wd-body.md - - # GitHub rejects a body over 65536 characters. A rejected edit would leave - # the previous body in place — a stale green presented as current, the one - # outcome this step exists to prevent — so truncate and say that we did. - if [ "$(wc -c < wd-body.md)" -gt 60000 ]; then - head -c 59000 wd-body.md > wd-body.trunc - { - printf '\n\n---\n\n**TRUNCATED** — the full report exceeded GitHub'"'"'s issue-body\n' - printf 'limit. See the [run log](%s) for the complete matrix.\n' "$RUN_URL" - } >> wd-body.trunc - mv wd-body.trunc wd-body.md - fi - - gh issue edit "$WATCHDOG_ISSUE" --repo tracebloc/backend --body-file wd-body.md - - - name: Report on the tracking issue - # Only on scheduled and manual runs: a PR that edits the inventory gets its - # answer from the check itself, and a comment per push would train everyone - # to ignore the issue (backend#1344 — five silent failures nobody read). - # No all-clear comment ever, for the same reason. - if: >- - always() - && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') - && steps.audit.outputs.exit_code != '0' - env: - GH_TOKEN: ${{ secrets.PROJECTS_KANBAN_TOKEN }} - CODE: ${{ steps.audit.outputs.exit_code }} - REPORT: ${{ steps.audit.outputs.report }} - RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - run: | - set -euo pipefail - code="${CODE:-}" - - # shellcheck disable=SC2016 # the backticks below are markdown code spans - # in the issue body, not command substitution — they must stay literal, - # which is exactly why these printf strings are single-quoted. - { - case "$code" in - 1) - printf '## Caller inventory drift\n\n' - printf 'Reality has diverged from `repo-inventory.yml`. Either a repo lost a\n' - printf 'caller, or the inventory is out of date — the report below says which.\n\n' - ;; - 2) - printf '## Caller inventory could NOT be evaluated (or remediation failed)\n\n' - printf 'This is not a clean result. Either `repo-inventory.yml` failed schema\n' - printf 'validation, one or more repos could not be read, or remediation PRs\n' - printf 'could not be opened/refreshed. Repos that could not be read are **not**\n' - printf 'known to comply, and drift that could not be remediated still stands —\n' - printf 'fix the token, the inventory, or the remediation dispatch and re-run\n' - printf 'before drawing any conclusion.\n\n' - ;; - *) - printf '## Caller inventory guard did not report a result\n\n' - printf 'The audit step produced exit code `%s`, which it should never do. Treat\n' "${code:-}" - printf 'the caller state across the org as UNKNOWN until this is explained.\n\n' - ;; - esac - if [ -n "${REPORT:-}" ]; then - printf '%s\n\n' "$REPORT" - else - printf 'The audit produced no report body — see the run log.\n\n' - fi - printf '[Run log](%s)\n' "$RUN_URL" - } > body.md - # backend#1781, not #1415: #1415 is CLOSED. Comments on a closed issue - # still post, so this looked like it worked — but a closed issue cannot - # be pinned, drops out of every default issue view, and notifies nobody - # who is not already subscribed. Drift has been reporting into a drawer. - gh issue comment "$WATCHDOG_ISSUE" --repo tracebloc/backend --body-file body.md - - - name: Fail the run unless the audit came back clean - # A green run must mean "the inventory matches reality", never merely "the - # workflow executed". Anything other than a literal 0 fails, and an absent - # value fails hardest: it means the audit step was skipped or died, which is - # exactly the failure-that-reports-success this whole guard is about. - if: always() - env: - CODE: ${{ steps.audit.outputs.exit_code }} - REMEDIATED: ${{ steps.audit.outputs.remediated }} - FINDINGS_TOTAL: ${{ steps.audit.outputs.findings_total }} - run: | - set -euo pipefail - code="${CODE:-}" - case "$code" in - 0) - # Exit 0 has TWO meanings after backend#1965: a clean match, or a - # fully remediated run (every finding got a PR). "every entry - # matched" is only true for the first -- printing it for a - # remediated run is the same false-green headline this change - # strips from the watchdog body, so the log branches on the same - # REMEDIATED signal the verdict does. - if [ "${REMEDIATED:-0}" != "0" ]; then - echo "Every active repo read; ${FINDINGS_TOTAL:-?} drift finding(s) remediated, every one with a PR open. The fleet is fixed-pending-merge, NOT yet conformant." - else - echo "Every active repo read; every inventory entry matched." - fi - ;; - 1) - echo "::error::Caller-inventory drift detected. See the run summary." >&2 - exit 1 - ;; - 2) - echo "::error::The caller inventory could not be evaluated or remediated — bad" \ - "inventory, failed org enumeration, an unreadable repo, or remediation PRs" \ - "that could not be opened. This is NOT an all-clear." >&2 - exit 1 - ;; - *) - echo "::error::The audit step reported no exit code ('$code'). It was skipped or" \ - "it crashed; caller state across the org is UNKNOWN. Failing closed." >&2 - exit 1 - ;; - esac diff --git a/.github/workflows/code-quality-caller.yml b/.github/workflows/code-quality-caller.yml deleted file mode 100644 index 1d106dc..0000000 --- a/.github/workflows/code-quality-caller.yml +++ /dev/null @@ -1,81 +0,0 @@ -name: Code quality - -# This repo publishes `code-quality.yml` to the whole org and, until now, was -# the one repo that never ran it (backend#1603). The gap was invisible because -# `actionlint.yml` looks like coverage: it type-checks expressions, validates -# needs/uses wiring and shellchecks every `run:` block -- but it does not look -# at action REFS at all. So the pinning rule this repo defines was the one rule -# this repo was exempt from, and on 2026-08-06 a PR here reverted -# `actions/checkout` from its commit SHA to the mutable `@v4` tag with nothing -# in CI to catch it (.github#168, caught in human review). -# -# `actionlint.yml` stays as a SEPARATE hard gate rather than folding in here. -# The two have different postures on purpose: actionlint has been blocking -# since day one against a tree that was cleaned in the same change, while -# code-quality ships `soft-fail: true` by default for repos still clearing a -# backlog. Merging them would force one posture onto both. -# -# No `paths:` filter, deliberately. A path-filtered job never reports on a PR -# that touches nothing it watches, and a required check that never reports -# leaves that PR waiting forever -- the trap `actionlint.yml`'s own header -# warns about before requiring it. - -on: - pull_request: - types: [opened, reopened, synchronize, ready_for_review] - # Manual whole-tree scan (gitleaks baseline etc.) -- runs every enabled - # job in all-files mode instead of a PR diff. - workflow_dispatch: - inputs: - all-files: - description: "Scan the whole repo, not a diff" - type: boolean - default: true - -concurrency: - group: code-quality-${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true - -permissions: - contents: read - -# No `secrets: inherit` -- RFC-BACKEND-1405 Q5. This caller sits in the same -# repo as the callee, which makes inheriting look harmless; it is not. The -# rule is about what a `secrets.X` reference added later would silently gain, -# and it applies here for the same reason it applies everywhere else. -jobs: - quality: - uses: tracebloc/.github/.github/workflows/code-quality.yml@main - with: - python: true # scripts/caller-drift.py + its selftest - shell: true # scripts/house-rules.sh - - # Armed from day one rather than soft-failed into a backlog. Measured on - # develop before writing this file: shellcheck --severity=error CLEAN over - # both shell scripts, house-rules.sh CLEAN over the same, and 24 `uses:` - # refs across 23 workflows with ZERO pin violations. The two checks I - # could not run locally -- ruff and gitleaks -- get their first honest - # look on this PR; anything they surface is fixed or baselined here - # rather than the gate being softened, which is the same bargain - # e2e-test-agent's caller records. - soft-fail: false - - # action-pins armed 2026-08-06 (backend#1603 step 2). It could not be - # armed when this caller landed: the job and its inputs were added in #159 - # and were on `develop` only, and a caller must reference @main (Q3) -- - # passing an input the @main callee does not declare kills the whole call - # with a startup_failure, not just the one job (measured: run - # 31086491251). #159 has since promoted, so `main` now carries the - # six-job version and both inputs. Verified before flipping this. - # - # Hard-armed rather than left to inherit soft-fail, because this is the - # check whose absence let .github#168 revert actions/checkout from a - # pinned SHA to the mutable @v4 tag with nothing in CI to object -- in - # advance-deploy-env.yml, the most-consumed reusable in the org, running - # with PROJECTS_KANBAN_TOKEN in scope. Every other repo consumes these - # workflows at @main and inherits whatever refs they pin, so this repo - # has a specific duty to enforce the rule it publishes. - action-pins: true - action-pins-soft-fail: false - - all-files: ${{ inputs.all-files || false }} diff --git a/.github/workflows/code-quality.yml b/.github/workflows/code-quality.yml deleted file mode 100644 index 6c3f4d2..0000000 --- a/.github/workflows/code-quality.yml +++ /dev/null @@ -1,1178 +0,0 @@ -name: Code quality - -# Reusable workflow. The org's first *code* quality gate — every other reusable -# workflow in this repo automates process (kanban, FR gate, WIP), and none -# of them looks at what the code does. -# -# WHY (measured over 60 days of automated code review) -# The automated reviewer's false-positive rate is 3% — so what it reports is -# real, and it arrives at the most expensive possible moment. Of the findings -# we hand-classified, 20% were expressible as a lint or grep rule, and 14% -# were rules the team had already agreed on, being re-enforced one PR at a -# time by a reviewer instead of once by CI. This workflow moves that share to -# the left, where it costs seconds instead of a review round-trip. -# -# JOBS — each is a SEPARATE job on purpose -# ruff Python lint (opt in with `python: true`) -# shellcheck shell lint (opt in with `shell: true`) -# gitleaks credential scanning (on by default) -# house-rules the org's own grep-level rules (on by default) -# early-close the pipefail SIGPIPE gate (on by default, whole-tree) -# Separate jobs run independently, so a ruff failure never hides what -# shellcheck found. Steps inside one job would short-circuit; jobs do not. -# -# BLOCKING BEHAVIOUR — read this before adopting -# `soft-fail` defaults to TRUE: every finding is annotated on the diff and -# written to the job summary, and the job still exits 0. That is deliberate. -# A linter switched on as a required check against an unlinted backlog gets -# the check removed, not the backlog fixed. The intended path is: -# -# 1. Add the caller with the defaults below. Findings appear; nothing blocks. -# 2. Run it once with `all-files: true` to size the whole backlog. -# 3. Clear the backlog (or record a gitleaks baseline / add ignore pragmas -# for the deliberate exceptions). -# 4. Flip `soft-fail: false` in the caller. -# 5. Mark `Code quality / ` as a required status check in branch -# protection. Only then is it a gate. -# -# Step 4 is the point of the exercise. `soft-fail: true` is a migration -# setting, not a destination: a linter that only ever warns changes nothing. -# If a repo is still on the default months from now, that is the finding. -# -# ADOPTION — drop this in as `.github/workflows/code-quality-caller.yml` -# -# name: Code quality -# -# on: -# pull_request: -# types: [opened, reopened, synchronize, ready_for_review] -# -# # Supersede the previous run when a branch is pushed again. Measured: -# # workflows missing this stack ~10-minute duplicate runs per push. -# concurrency: -# group: code-quality-${{ github.workflow }}-${{ github.ref }} -# cancel-in-progress: true -# -# permissions: -# contents: read -# -# jobs: -# quality: -# uses: tracebloc/.github/.github/workflows/code-quality.yml@main -# with: -# python: true # repos with Python -# shell: true # repos with shell scripts -# # soft-fail: false # flip once the backlog is clear -# -# This workflow uses NO secrets and asks only for `contents: read`, so it runs -# under a minimal caller and the org default workflow-token permission of "read". -# Callers pass NO `secrets:` line at all — decided under RFC-BACKEND-1405 Q5 -# (backend#1420, unwound in backend#1526). `inherit` was briefly the convention -# ("harmless no-op, nothing is referenced"), but the no-op is conditional on this -# file staying secretless, and callers pin `@main`: the day this workflow gains a -# `${{ secrets.X }}` step, every inheriting caller hands it that repo's ENTIRE -# secret set (`toJSON(secrets)` dumps it). If a secret is ever genuinely needed -# here, callers switch to explicit per-secret passing (`secrets: { name: ... }`), -# never `inherit` — the need stays visible in every caller diff. Cursor Bugbot's -# recurring nag about the "missing" line is suppressed per-repo in -# `.cursor/BUGBOT.md` (the backend#1304 flywheel); the inverse — a caller that -# ADDS `secrets: inherit` — is what deserves a finding. -# -# SUPPLY CHAIN -# gitleaks is installed from its release tarball pinned by version AND -# verified against a pinned SHA-256, rather than via gitleaks-action. Two -# reasons: gitleaks-action requires a GITLEAKS_LICENSE for organization-owned -# repos (it would simply fail here), and a version+digest pin on the artefact -# we actually execute is a stronger guarantee than a commit pin on a wrapper -# that downloads it for us. `actions/checkout` is pinned to a commit SHA. -# -# EXTENDING house-rules -# Rules live in `scripts/house-rules.sh` in this repo. A repo adds its own -# without touching the shared workflow by committing `.house-rules.conf`: -# -# exclude: third_party/* -# timeout-wrapper: guard # a wrapper that already bounds time -# disable: curl-timeout -# rule: no-print | *.py | ^[[:space:]]*print\( | use client_logger, not print() -# -# Run it locally exactly as CI does: ./house-rules.sh --all -# (`--help` documents every rule, every exclusion, and the ignore pragma.) - -on: - workflow_call: - inputs: - soft-fail: - description: "Report findings but exit 0. Flip to false to make the jobs a real gate." - type: boolean - default: true - all-files: - description: "Scan the whole repo instead of only the files this PR changed." - type: boolean - default: false - python: - description: "Run the ruff job." - type: boolean - default: false - format: - description: >- - Run the black --check job. Opt-in per repo, because a formatter is - only enforceable where the tree already matches it -- see - `black-version` for why the version is the whole story. - type: boolean - default: false - format-soft-fail: - description: >- - Report formatting findings but exit 0, even when `soft-fail` is false. - Lets a repo adopt the format job advisory-first (its lint/credential - gates stay blocking) and drop this flag once its tree is swept. - type: boolean - default: false - black-version: - description: >- - Pinned black version. black's stable style changes between releases, - so a version that does not match what the tree was formatted with - reports the whole repo as unformatted. Pin it per caller and keep - it in step with the repo's pre-commit hook, or CI and local disagree. - - The default is 26.3.1: it is the version backend -- the only repo - already gating black -- is clean under, and what the pre-commit - hooks pin. It also runs on current Python, which 23.1.0 does not: - 23.1.0 calls `ast.Str`, removed in 3.12, so it errors on every file - while printing zero "would reformat" lines. Measured on 3.11, no - repo is 23.1.0-clean. - type: string - default: "26.3.1" - shell: - description: "Run the shellcheck job." - type: boolean - default: false - credential-scan: - description: "Run the gitleaks job." - type: boolean - default: true - house-rules: - description: "Run the org house-rules checker." - type: boolean - default: true - early-close: - description: >- - Run the pipefail early-close gate. ON by default and whole-tree: it - costs one awk pass, and a repo with no shell files exits clean. - type: boolean - default: true - yaml-run-blocks: - description: >- - Also scan `run:` blocks in workflow and composite-action YAML - (backend#2967). A `run:` block IS shell -- GitHub executes it with - `bash -eo pipefail` whenever the step says `shell: bash` -- but the - gate classified files by extension-else-shebang, so YAML was out of - scope and the gate reported success on a live `printf | head -1`. - type: boolean - default: true - yaml-run-blocks-soft-fail: - description: >- - Report YAML `run:` findings but exit 0, even when `soft-fail` is - false. Defaults TRUE for the same reason as `action-pins-soft-fail`: - bringing a new file class into scope surfaces a pre-existing backlog - in every repo at once, and a gate that reddens on arrival gets - removed rather than cleared (backend#1729 rule 4). Flip it per - caller once that repo's `run:` blocks are converted. - type: boolean - default: true - action-pins: - description: "Run the whole-tree action-pin gate (backend#1492, D10)." - type: boolean - default: true - action-pins-soft-fail: - description: >- - Report pin findings but exit 0, even when `soft-fail` is false. - Defaults TRUE so arming code-quality does not instantly redden every - repo that still carries the pre-#1491 unpinned backlog — same - migration shape as format-soft-fail. Flip to false per caller (or - change this default) once the #1491 sweep has merged fleet-wide. - type: boolean - default: true - ruff-version: - description: "Pinned ruff version." - type: string - default: "0.15.20" - ruff-select: - description: >- - Rule selection used ONLY when the repo has no ruff config of its own. - The default is ruff's own default set: pyflakes plus the pycodestyle - errors that indicate a real bug. Deliberately excludes every - formatting-opinion family (E1/E2/E3 whitespace, W, I import order, - D docstrings, ANN annotations) — those would report thousands of - findings on legacy code and get the job switched off. - type: string - default: "E4,E7,E9,F" - ruff-paths: - description: "Paths ruff scans in all-files mode." - type: string - default: "." - shellcheck-severity: - description: >- - Minimum severity: error | warning | info | style. Defaults to `error` - so a repo can adopt the job on day one; `warning` is the recommended - target once the backlog is clear (it is where SC2086 and friends live). - type: string - default: "error" - house-rules-config: - description: "Path to the repo's house-rules config." - type: string - default: ".house-rules.conf" - house-rules-exclude: - description: "Extra path glob for house-rules to skip (one glob)." - type: string - default: "" - dead-weight: - description: >- - Run the dead-weight checker (RFC-0087 D3) in the house-rules job: - declared-vs-imported dependencies, full python base images, CUDA torch - installed on CPU targets. Same required context as house-rules, so it - is a gate wherever house-rules is one. - type: boolean - default: true - dead-weight-soft-fail: - description: >- - Report dead-weight findings but exit 0, even when `soft-fail` is - false. Defaults TRUE, unlike the other soft-fail inputs: the checker - arrives on `main` for all 16 callers at once, and a repo flips this - to false in the same PR that writes its `indirect-use` entries and - shows zero findings (arm while green -- backend#3527). NOT covered by - this flag: scan-integrity findings (`cannot-read`, `cannot-parse`, - `config-error`) exit 1 in every mode -- an unparseable tracked .py, a - computed setup.py install_requires or a defaultless FROM ARG reddens - the required house-rules job even while advisory. The dry run of - 2026-09-09 showed every caller clean of those; confirm it on the - repo's develop before flipping this to false. - type: boolean - default: true - gitleaks-baseline: - description: "Path to a gitleaks baseline report; known findings in it are not re-reported." - type: string - default: "" - quality-ref: - description: >- - Ref of tracebloc/.github the house-rules script is taken from. Callers - pin this workflow at `@main`, so `main` is the matching version and is - the default. Override it only when calling this workflow from a branch - — e.g. to test a change to the checker before it merges. - type: string - default: "main" - -# Only `contents: read`. Requesting more would exceed a minimal caller's grant -# and fail the run at startup with no jobs. -permissions: - contents: read - -jobs: - # ---------------------------------------------------------------- Python ---- - ruff: - name: ruff - if: ${{ inputs.python }} - runs-on: ubuntu-latest - timeout-minutes: 10 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - ref: ${{ github.event.pull_request.head.sha || github.sha }} - - - name: Resolve the file set - id: files - env: - ALL_FILES: ${{ inputs.all-files }} - BASE_SHA: ${{ github.event.pull_request.base.sha }} - PATHS: ${{ inputs.ruff-paths }} - run: | - set -euo pipefail - if [ "$ALL_FILES" = "true" ] || [ -z "${BASE_SHA:-}" ]; then - echo "mode=all" >> "$GITHUB_OUTPUT" - echo "Scanning: $PATHS" - exit 0 - fi - # Three-dot: what this branch changed, not what the base moved on to. - # If the diff itself fails (shallow clone, BASE_SHA not fetched), fall - # back to scanning everything — never silently lint nothing. shellcheck - # and house-rules already do this; ruff must not be the one gate that - # no-ops on a bad diff. - if ! git diff --name-only --diff-filter=ACMR "$BASE_SHA...HEAD" > /tmp/py-diff.txt 2>/dev/null; then - echo "mode=all" >> "$GITHUB_OUTPUT" - echo "::warning::git diff against $BASE_SHA failed — scanning all files instead of skipping." - echo "Scanning: $PATHS" - exit 0 - fi - grep -E '\.pyi?$' /tmp/py-diff.txt > /tmp/py-files.txt || true - COUNT=$(wc -l < /tmp/py-files.txt | tr -d ' ') - echo "mode=diff" >> "$GITHUB_OUTPUT" - echo "count=$COUNT" >> "$GITHUB_OUTPUT" - echo "Changed Python files: $COUNT" - - - name: Install ruff - if: steps.files.outputs.mode == 'all' || steps.files.outputs.count != '0' - env: - RUFF_VERSION: ${{ inputs.ruff-version }} - run: | - set -euo pipefail - # pipx is preinstalled on GitHub runners and sidesteps the - # externally-managed-environment refusal that plain pip hits. - pipx install "ruff==$RUFF_VERSION" - ruff --version - - - name: ruff check - if: steps.files.outputs.mode == 'all' || steps.files.outputs.count != '0' - env: - MODE: ${{ steps.files.outputs.mode }} - SELECT: ${{ inputs.ruff-select }} - PATHS: ${{ inputs.ruff-paths }} - SOFT_FAIL: ${{ inputs.soft-fail }} - run: | - set -uo pipefail - - # Never override a repo that has made its own choices. - CONFIGURED=0 - if [ -f ruff.toml ] || [ -f .ruff.toml ]; then CONFIGURED=1; fi - if [ -f pyproject.toml ] && grep -q '^\[tool\.ruff' pyproject.toml; then CONFIGURED=1; fi - - set -- --output-format=concise --no-cache - if [ "$CONFIGURED" = "1" ]; then - echo "Using this repo's own ruff configuration." - else - echo "No ruff config found — using the shared default selection: $SELECT" - set -- "$@" --isolated --select "$SELECT" - fi - - RC=0 - if [ "$MODE" = "all" ]; then - # shellcheck disable=SC2086 # PATHS is a deliberate word-split list - ruff check "$@" $PATHS > /tmp/ruff.out 2>&1 || RC=$? - else - # NUL-delimited so a path with a space stays one argument; -r so an - # empty list never falls through to ruff's default of scanning - # everything. - tr '\n' '\0' < /tmp/py-files.txt \ - | xargs -0 -r ruff check "$@" > /tmp/ruff.out 2>&1 || RC=$? - fi - - cat /tmp/ruff.out - - if [ "$SOFT_FAIL" = "true" ]; then LEVEL=warning; else LEVEL=error; fi - # concise format: path:line:col: CODE message - awk -v lvl="$LEVEL" ' - /^[^ ]+:[0-9]+:[0-9]+: / { - split($0, p, ":") - msg=$0; sub(/^[^:]*:[0-9]+:[0-9]+: /, "", msg) - printf "::%s file=%s,line=%s,col=%s,title=ruff::%s\n", lvl, p[1], p[2], p[3], msg - }' /tmp/ruff.out - - COUNT=$(grep -cE '^[^ ]+:[0-9]+:[0-9]+: ' /tmp/ruff.out || true) - { - echo "### ruff" - echo "" - if [ "$COUNT" = "0" ]; then - echo "No findings." - else - echo "**$COUNT finding(s).**" - echo "" - echo '```' - head -200 /tmp/ruff.out - echo '```' - fi - echo "" - } >> "$GITHUB_STEP_SUMMARY" - - if [ "$RC" != "0" ] && [ "$SOFT_FAIL" = "true" ]; then - echo "soft-fail is on — reporting only, not failing this job." - exit 0 - fi - exit "$RC" - - # ---------------------------------------------------------------- format ---- - # black --check on the files this PR changes (all files in all-files mode). - # Diff-scoped on purpose: it enforces "leave it formatted" without ever - # demanding a repo-wide reformat, so adopting the job costs no churn. - format: - name: format - if: ${{ inputs.format }} - runs-on: ubuntu-latest - timeout-minutes: 10 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - ref: ${{ github.event.pull_request.head.sha || github.sha }} - - - name: Resolve the file set - id: files - env: - ALL_FILES: ${{ inputs.all-files }} - BASE_SHA: ${{ github.event.pull_request.base.sha }} - run: | - set -euo pipefail - if [ "$ALL_FILES" = "true" ] || [ -z "${BASE_SHA:-}" ]; then - echo "mode=all" >> "$GITHUB_OUTPUT" - exit 0 - fi - # Same three-dot diff + fail-open-to-all-files contract as the ruff - # job: never silently check nothing. - if ! git diff --name-only --diff-filter=ACMR "$BASE_SHA...HEAD" > /tmp/fmt-diff.txt 2>/dev/null; then - echo "mode=all" >> "$GITHUB_OUTPUT" - echo "::warning::git diff against $BASE_SHA failed - checking all files instead of skipping." - exit 0 - fi - grep -E '\.pyi?$' /tmp/fmt-diff.txt > /tmp/fmt-files.txt || true - COUNT=$(wc -l < /tmp/fmt-files.txt | tr -d ' ') - echo "mode=diff" >> "$GITHUB_OUTPUT" - echo "count=$COUNT" >> "$GITHUB_OUTPUT" - echo "Changed Python files: $COUNT" - - - name: Install black - if: steps.files.outputs.mode == 'all' || steps.files.outputs.count != '0' - env: - BLACK_VERSION: ${{ inputs.black-version }} - run: | - set -euo pipefail - pipx install "black==$BLACK_VERSION" - black --version - - - name: black --check - if: steps.files.outputs.mode == 'all' || steps.files.outputs.count != '0' - env: - MODE: ${{ steps.files.outputs.mode }} - # Additive soft-fail, per this input's contract ("exit 0 even when - # soft-fail is false"): format is advisory when EITHER the global - # soft-fail OR the format-specific override is set. Unlike action-pins - # below, format-soft-fail defaults to false and no caller arms it, and - # there's no need to force format HARD during a soft migration -- so it - # stays an OR, not an authoritative override. Dropping the OR flipped - # every `soft-fail: true` caller to a hard format gate (Bugbot, #1681). - SOFT_FAIL: ${{ (inputs.soft-fail || inputs.format-soft-fail) && 'true' || 'false' }} - run: | - set -uo pipefail - # No --isolated: black must read the repo's own [tool.black] - # (line-length, target-version, exclude) or its verdict would differ - # from what contributors run locally. - RC=0 - if [ "$MODE" = "all" ]; then - black --check . > /tmp/black.out 2>&1 || RC=$? - else - # NUL-delimited: paths with spaces must survive. - tr '\n' '\0' < /tmp/fmt-files.txt | xargs -0 --no-run-if-empty \ - black --check > /tmp/black.out 2>&1 || RC=$? - fi - - COUNT=$(grep -c '^would reformat' /tmp/black.out || true) - # An ERROR is a different thing from a formatting verdict and must - # never be reported as one: a black that cannot parse the tree prints - # these while the reformat count stays 0. (This exact conflation - # produced a false "0 files need formatting" during the #1303 - # sizing -- black 23.1.0 cannot run on Python 3.12+.) - ERRS=$(grep -cE '^error:|would fail to reformat' /tmp/black.out || true) - - { - echo "### format (black ${{ inputs.black-version }})" - echo "" - if [ "${ERRS:-0}" -gt 0 ]; then - echo "black could not process ${ERRS} file(s) -- this is an error, not a formatting verdict." - elif [ "${COUNT:-0}" -gt 0 ]; then - echo "${COUNT} file(s) need reformatting. Run \`black \` (or \`pre-commit run black\`)." - else - echo "No formatting changes needed." - fi - if [ "$RC" != "0" ]; then - echo "" - echo '```' - head -50 /tmp/black.out - echo '```' - fi - echo "" - } >> "$GITHUB_STEP_SUMMARY" - - # Decide from black's OUTPUT, not from its exit code. In diff mode - # black runs under xargs, and GNU xargs remaps ANY child status in - # 1-125 to its own 123 -- so a plain "would reformat" (black's 1) - # arrives as 123 and is indistinguishable from a real internal error. - # BSD xargs propagates 1 instead, which is why local validation on - # macOS passed while ubuntu CI would have hard-failed every advisory - # adopter. Reported by Bugbot + @aptracebloc on tracebloc/.github#115. - # - # Checked errors-first so a genuine internal error is never masked by - # a reformat finding in the same run. - if [ "${ERRS:-0}" -gt 0 ]; then - echo "::error::black could not process ${ERRS} file(s) - failing closed (not a formatting verdict)." - cat /tmp/black.out - exit 1 - fi - if [ "${COUNT:-0}" -gt 0 ]; then - if [ "$SOFT_FAIL" = "true" ]; then - echo "soft-fail is on - reporting ${COUNT} unformatted file(s), not failing this job." - exit 0 - fi - exit 1 - fi - if [ "$RC" != "0" ]; then - echo "::error::black exited $RC but reported no findings - failing closed." - cat /tmp/black.out - exit 1 - fi - exit 0 - - # ----------------------------------------------------------------- shell ---- - shellcheck: - name: shellcheck - if: ${{ inputs.shell }} - runs-on: ubuntu-latest - timeout-minutes: 10 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - ref: ${{ github.event.pull_request.head.sha || github.sha }} - - - name: shellcheck - env: - ALL_FILES: ${{ inputs.all-files }} - BASE_SHA: ${{ github.event.pull_request.base.sha }} - SEVERITY: ${{ inputs.shellcheck-severity }} - SOFT_FAIL: ${{ inputs.soft-fail }} - run: | - set -uo pipefail - # Preinstalled on ubuntu-latest — no download, no third-party action. - # (A comment here must not begin with the tool's name: that is the - # syntax for an inline directive and shellcheck fails to parse it.) - shellcheck --version | head -2 - - # Collect shell files by extension or shebang, from the diff or all. - : > /tmp/sh-files.txt - if [ "$ALL_FILES" = "true" ] || [ -z "${BASE_SHA:-}" ]; then - git ls-files > /tmp/cand.txt - else - git diff --name-only --diff-filter=ACMR "$BASE_SHA...HEAD" > /tmp/cand.txt \ - || git ls-files > /tmp/cand.txt - fi - while IFS= read -r f; do - [ -f "$f" ] || continue - case "$f" in - *.sh|*.bash|*.ksh) printf '%s\n' "$f" >> /tmp/sh-files.txt ;; - *.bats|*.ps1|*.psm1|*.zsh) ;; - *) head -n 1 "$f" 2>/dev/null \ - | grep -Eq '^#![[:space:]]*[^[:space:]]*(/|[[:space:]])(ba|da|k)?sh([[:space:]]|$)' \ - && printf '%s\n' "$f" >> /tmp/sh-files.txt ;; - esac - done < /tmp/cand.txt - - COUNT_FILES=$(wc -l < /tmp/sh-files.txt | tr -d ' ') - echo "Shell files to check: $COUNT_FILES" - if [ "$COUNT_FILES" = "0" ]; then - { - echo "### shellcheck" - echo "" - echo "No shell files in scope." - echo "" - } >> "$GITHUB_STEP_SUMMARY" - exit 0 - fi - - # SC1091 = "not following sourced file"; unavoidable and uninteresting - # when a library is sourced through a variable path. - RC=0 - tr '\n' '\0' < /tmp/sh-files.txt \ - | xargs -0 -r shellcheck \ - --severity="$SEVERITY" --format=gcc --exclude=SC1091 \ - > /tmp/shellcheck.out 2>&1 || RC=$? - - cat /tmp/shellcheck.out - - if [ "$SOFT_FAIL" = "true" ]; then LEVEL=warning; else LEVEL=error; fi - # gcc format: path:line:col: severity: message [SCxxxx] - awk -v lvl="$LEVEL" ' - /^[^ ]+:[0-9]+:[0-9]+: / { - split($0, p, ":") - msg=$0; sub(/^[^:]*:[0-9]+:[0-9]+: /, "", msg) - printf "::%s file=%s,line=%s,col=%s,title=shellcheck::%s\n", lvl, p[1], p[2], p[3], msg - }' /tmp/shellcheck.out - - COUNT=$(grep -cE '^[^ ]+:[0-9]+:[0-9]+: ' /tmp/shellcheck.out || true) - { - echo "### shellcheck (severity: $SEVERITY)" - echo "" - if [ "$COUNT" = "0" ]; then - echo "No findings across $COUNT_FILES file(s)." - else - echo "**$COUNT finding(s)** across $COUNT_FILES file(s)." - echo "" - echo '```' - head -200 /tmp/shellcheck.out - echo '```' - fi - echo "" - } >> "$GITHUB_STEP_SUMMARY" - - if [ "$RC" != "0" ] && [ "$SOFT_FAIL" = "true" ]; then - echo "soft-fail is on — reporting only, not failing this job." - exit 0 - fi - exit "$RC" - - # ----------------------------------------------------- pipefail early-close --- - # `producer | head -n N` (or `| grep -q`, `| grep -m N`) under `set -e` AND - # `set -o pipefail` aborts its own caller once the producer outgrows the ~64KB - # pipe buffer: the reader closes, the producer takes SIGPIPE, the pipeline - # returns 141, errexit kills the script. SIZE-dependent, so it survives review - # — measured, 50 lines exit 0 and 20k exit 141. Two incidents in `client` - # (client#656, client#678) before the rule was encoded there, and arming it - # fleet-wide (backend#2264) meant converting 19 instances across six repos. - # - # WHOLE-TREE, NEVER DIFF-SCOPED, and here that is CORRECTNESS rather than the - # policy argument action-pins makes. Whether a line is hazardous depends on - # whether its FILE runs under both options — and a library that sets neither - # inherits them from whatever sources it. Scanning only changed files would - # resolve inheritance against a partial tree: edit `lib/foo.sh` without - # touching its sourcer and the gate would call it safe. The scan is one awk - # pass over the tracked shell files, so whole-tree costs nothing worth saving. - # - # NOT A LINE-GRAMMAR CHECK. Shell options are POSITIONAL (`set +e` opens a - # best-effort region and must stand the rule down), the long and short - # spellings both count, and one-line function bodies must be scanned rather - # than skipped. That is a state machine, which is why this is a script in - # `scripts/` with its own selftest and mutation harness rather than a grep - # rule inside house-rules.sh. - # - # The opt-out is `# pipefail-guard: allow` on the offending line, and the - # convention is to state WHY inline: `client`'s worked example is a - # `df -h | head -20` that runs under `set +e` and streams, where converting it - # would trade a non-existent abort for a real hang on a stalled NFS mount. - early-close: - name: pipefail early-close - if: ${{ inputs.early-close }} - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ github.event.pull_request.head.sha || github.sha }} - - - name: Check out the shared checker - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: tracebloc/.github - ref: ${{ inputs.quality-ref }} - path: .quality-tools - persist-credentials: false - - # PyYAML IS PROVISIONED, NEVER ASSUMED. The YAML phase imports it, and a - # missing import is (correctly) exit 2 -- "cannot tell" is a finding. But - # an rc 2 is a HARD failure regardless of soft-fail, so relying on - # whatever the runner image happens to ship would turn the first run of - # this coverage into a red gate across the fleet. Same pin and same - # invocation as selftests.yml and caller-drift.yml. - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - if: ${{ inputs.yaml-run-blocks }} - with: - python-version: '3.12' - - name: Install PyYAML for the YAML run-block phase - if: ${{ inputs.yaml-run-blocks }} - # RETRIED, BECAUSE THIS SITS ON A GATE IN 18 REPOS (@saadqbal). A single - # PyPI hiccup reddened `code-quality` fleet-wide with a message about pip - # rather than about the code -- and a gate that fails for an unrelated - # reason is one people learn to re-run without reading. - # - # Three attempts with a widening pause. Deliberately NOT `continue-on- - # error`: without PyYAML the extractor exits 2 by design ("cannot tell" - # is a finding), so swallowing this would trade a loud unrelated failure - # for a quiet real one. - run: | - set -euo pipefail - for attempt in 1 2 3; do - if python -m pip install --quiet --disable-pip-version-check \ - 'pyyaml==6.0.2'; then - exit 0 - fi - echo "::warning::pip install pyyaml failed (attempt ${attempt}/3)" - # `if` rather than `[ "$attempt" -lt 3 ] && sleep ...`, for - # READABILITY, not for correctness. I first wrote this comment - # claiming the `&&` form would trip `set -e` on the last attempt and - # skip the diagnostic below. It does not: POSIX exempts every - # command of an AND-OR list except the last, so a false `[` there - # never triggers errexit. Verified both forms reach the end at rc 0 - # before saying so, since being wrong in the other direction is how - # a comment starts teaching the bug. - if [ "$attempt" -lt 3 ]; then - sleep $(( attempt * 5 )) - fi - done - echo "::error::could not install PyYAML after 3 attempts -- the YAML" - echo "::error::run-block phase cannot run, and its extractor treats a" - echo "::error::missing PyYAML as 'cannot tell' rather than as clean." - exit 1 - - - name: pipefail early-close - env: - SOFT_FAIL: ${{ inputs.soft-fail }} - YAML_RUN_BLOCKS: ${{ inputs.yaml-run-blocks }} - YAML_SOFT_FAIL: ${{ inputs.yaml-run-blocks-soft-fail }} - run: | - set -uo pipefail - GATE=".quality-tools/scripts/pipefail-early-close.sh" - chmod +x "$GATE" - - # TWO SCOPES, ONE RULE (backend#2967). `shell` is the tree's shell - # files -- the verdict this job has always produced, under whatever - # `soft-fail` the caller chose. `yaml` is the `run:` blocks of - # workflows and composite actions, which were never in scope at all: - # the classifier is extension-else-shebang and YAML is neither, so - # the gate reported SUCCESS on `e2e-test-agent@f4d6fec`'s live - # `printf | head -1`. Both halves are judged by the SAME awk; only - # the file list differs. - # - # THEY RUN AS SEPARATE INVOCATIONS, not one `all` pass, because the - # two halves need different blocking behaviour during the migration - # and a single exit code cannot say which half produced it. - OVERALL=0 - for scope in shell yaml; do - if [ "$scope" = yaml ]; then - [ "$YAML_RUN_BLOCKS" = "true" ] || continue - SOFT="$YAML_SOFT_FAIL" - TITLE="pipefail early-close (YAML run blocks)" - else - SOFT="$SOFT_FAIL" - TITLE="pipefail early-close" - fi - OUT="/tmp/early-close-$scope.out" - ERR="/tmp/early-close-$scope.err" - - # PIPEFAIL_ROOT is the CALLER's checkout. The `.quality-tools` tree - # sits inside the workspace but outside the caller's git index, so - # `git ls-files` cannot reach it and the checker never lints itself. - RC=0 - PIPEFAIL_SCOPE="$scope" PIPEFAIL_ROOT="$GITHUB_WORKSPACE" \ - bash "$GATE" > "$OUT" 2>"$ERR" || RC=$? - cat "$OUT" - cat "$ERR" >&2 || true - - # ONLY 0 AND 1 ARE VERDICTS. 0 = clean, 1 = findings; anything else is - # the gate failing to run -- rc 2 (cannot tell), 126/127 (not - # executable / not found), a signal death. Those are ALWAYS fatal, - # soft-fail or not: a gate that could not check has not passed, and - # letting soft-fail swallow it is how a gate becomes decoration - # (backend#1729 rule 3). - # - # Whitelisting the verdicts rather than blacklisting rc=2 is the - # load-bearing part. The first version tested `[ "$RC" = 2 ]`, so a - # missing or non-executable script exited 127, fell through to the - # soft-fail branch, and reported green (Bugbot, .github#300). - if [ "$RC" != 0 ] && [ "$RC" != 1 ]; then - echo "::error title=$TITLE::the gate exited $RC — it did not run to a verdict, so this is not a pass" - { - echo "### $TITLE" - echo "" - echo "**The gate exited \`$RC\`** — not a verdict (0 = clean, 1 = findings)." - echo "Hard failure regardless of \`soft-fail\`: a gate that could not check has not passed." - echo "" - echo '```' - head -20 "$ERR" - echo '```' - } >> "$GITHUB_STEP_SUMMARY" - exit "$RC" - fi - - if [ "$SOFT" = "true" ]; then LEVEL=warning; else LEVEL=error; fi - # Output is `path:line: code`. - awk -v lvl="$LEVEL" -v title="$TITLE" -F: ' - /^[^:]+:[0-9]+: / { - msg=$0; sub(/^[^:]*:[0-9]+: /, "", msg) - printf "::%s file=%s,line=%s,title=%s::pipes into an early-closing reader under errexit+pipefail; use a here-string or capture-then-slice: %s\n", lvl, $1, $2, title, msg - }' "$OUT" - - COUNT=$(grep -cE "^[^:]+:[0-9]+: " "$OUT" || true) - { - echo "### $TITLE" - echo "" - if [ "$COUNT" = "0" ]; then - echo "No findings." - else - echo "**$COUNT finding(s).** Each pipes into a reader that closes before EOF" - echo "(\`head\`, \`grep -q\`, \`grep -m N\`, \`sed q\`, \`read\`) where errexit + pipefail are both live." - echo "Use a here-string (\`head -25 <<<\"\$out\"\`) or capture-then-slice." - echo "If an instance is genuinely safe, mark the line \`# pipefail-guard: allow\` and say why." - echo "" - echo '```' - head -100 "$OUT" - echo '```' - fi - echo "" - } >> "$GITHUB_STEP_SUMMARY" - - if [ "$RC" != "0" ] && [ "$SOFT" = "true" ]; then - echo "$TITLE: soft-fail is on — reporting only, not failing this job." - elif [ "$RC" != "0" ]; then - OVERALL=1 - fi - done - exit "$OVERALL" - - # --------------------------------------------------------------- gitleaks --- - gitleaks: - name: gitleaks - if: ${{ inputs.credential-scan }} - runs-on: ubuntu-latest - timeout-minutes: 15 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - # Full history: the scan walks the PR's commit range, so the base - # commit has to exist locally. A shallow clone silently scans nothing. - fetch-depth: 0 - ref: ${{ github.event.pull_request.head.sha || github.sha }} - - - name: Install gitleaks (version + SHA-256 pinned) - env: - GITLEAKS_VERSION: "8.30.1" - GITLEAKS_SHA256: "551f6fc83ea457d62a0d98237cbad105af8d557003051f41f3e7ca7b3f2470eb" - run: | - set -euo pipefail - URL="https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VERSION}/gitleaks_${GITLEAKS_VERSION}_linux_x64.tar.gz" - curl -fsSL --tlsv1.2 --connect-timeout 30 --max-time 300 "$URL" -o /tmp/gitleaks.tar.gz - echo "${GITLEAKS_SHA256} /tmp/gitleaks.tar.gz" | sha256sum -c - - tar -xzf /tmp/gitleaks.tar.gz -C /tmp gitleaks - chmod +x /tmp/gitleaks - /tmp/gitleaks version - - - name: Scan for leaked credentials - env: - ALL_FILES: ${{ inputs.all-files }} - BASE_SHA: ${{ github.event.pull_request.base.sha }} - HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }} - BASELINE: ${{ inputs.gitleaks-baseline }} - SOFT_FAIL: ${{ inputs.soft-fail }} - run: | - set -uo pipefail - - set -- --no-banner --redact --report-format json --report-path /tmp/gitleaks.json - if [ -n "${BASELINE:-}" ] && [ -f "$BASELINE" ]; then - echo "Applying baseline: $BASELINE" - set -- "$@" --baseline-path "$BASELINE" - fi - # A .gitleaks.toml at the repo root is picked up automatically. - - # EVERY invocation below is `gitleaks git` — commit-scoped — and the - # per-repo baselines DEPEND on that: baseline fingerprints embed the - # commit SHA that introduced each finding, so they only match findings - # produced by a git-mode scan. Switching any branch here to `gitleaks - # dir`/`detect` (path-scoped fingerprints) silently invalidates every - # baseline entry fleet-wide at once — 277 suppressions un-suppress in - # a single run (backend#1404 §4). Permanent fixtures live in per-repo - # `.gitleaks.toml` allowlists instead (auto-loaded from the checkout - # root, commit-INDEPENDENT); only genuine historical exposure stays - # baselined, tracked for rotation in backend#1355. - RC=0 - if [ "$ALL_FILES" = "true" ] || [ -z "${BASE_SHA:-}" ]; then - echo "Scanning the full history." - /tmp/gitleaks git "$@" . || RC=$? - else - # Exactly the commits this PR adds — every version of every line it - # introduced, so a value added and then deleted again is still found. - echo "Scanning commit range ${BASE_SHA}..${HEAD_SHA}" - /tmp/gitleaks git "$@" --log-opts="--no-merges ${BASE_SHA}..${HEAD_SHA}" . || RC=$? - # gitleaks uses 0 = clean, 1 = leaks found; any other code is an - # operational error (missing base.sha, bad log-opts). Don't let a - # broken credential scan look clean — fall back to a full-history - # scan, the same as ruff/shellcheck/house-rules (Bugbot #65). - if [ "$RC" != "0" ] && [ "$RC" != "1" ]; then - echo "::warning::gitleaks range scan errored (exit $RC) — falling back to a full-history scan." - RC=0 - /tmp/gitleaks git "$@" . || RC=$? - fi - fi - - COUNT=0 - if [ -s /tmp/gitleaks.json ]; then - COUNT=$(jq 'length' /tmp/gitleaks.json 2>/dev/null || echo 0) - fi - - # Captured whole HERE, then sliced with a here-string below -- NOT - # `jq ... | head -50`. `head` closes the pipe after row 50; with a big - # enough finding set jq is still buffering, so it takes SIGPIPE, and - # under pipefail the PIPELINE returns 141. Errexit then aborts the step - # mid-summary -- and -e IS on here: Actions runs `run:` as `bash -e {0}`, - # so the `set -uo pipefail` above leaves it ON while reading as though it - # were off. The damage is not just a short table: the ::error::/::warning:: - # annotations below never emit, and the soft-fail `exit 0` is skipped, so - # a repo configured `soft-fail: true` HARD-fails instead of reporting. - # Latent, not live -- it needs ~700+ findings to fill a 64K pipe buffer - # (measured: tips at ~250 findings / ~24K, nondeterministically at the - # boundary, because it is a race). A here-string is one command, so jq - # always runs to completion and the status is its own. - # (backend#1778 -- same class as .github#173 and the PII gate's #1409.) - # - # `|| true` keeps a parse failure non-fatal, which is what the existing - # `2>/dev/null` already intended; it is near-unreachable anyway, since - # COUNT != 0 means jq already parsed this file once. - ROWS="" - if [ "$COUNT" != "0" ]; then - ROWS=$(jq -r '.[] | "| \(.RuleID) | `\(.File)` | \(.StartLine) | \(.Commit[0:8]) |"' \ - /tmp/gitleaks.json 2>/dev/null || true) - fi - - { - echo "### gitleaks" - echo "" - if [ "$COUNT" = "0" ]; then - echo "Nothing detected." - else - echo "**$COUNT finding(s).** Values are redacted here and in the log." - echo "" - echo "| Rule | File | Line | Commit |" - echo "|---|---|---:|---|" - # Guarded: `head <<<""` would emit one blank line and break the - # table, where the old pipeline printed nothing. - if [ -n "$ROWS" ]; then - head -50 <<<"$ROWS" - fi - echo "" - echo "Treat anything detected here as compromised: **rotate it first**, then" - echo "remove it from the code. Rewriting history is not remediation — the" - echo "value was already pushed. A deliberate false positive belongs in" - echo "\`.gitleaks.toml\` (allowlist) or in a committed baseline report." - fi - echo "" - } >> "$GITHUB_STEP_SUMMARY" - - if [ "$COUNT" != "0" ]; then - if [ "$SOFT_FAIL" = "true" ]; then LEVEL=warning; else LEVEL=error; fi - jq -r --arg lvl "$LEVEL" '.[] | "::" + $lvl + " file=" + .File + ",line=" + (.StartLine|tostring) + ",title=gitleaks: " + .RuleID + "::Detected by rule " + .RuleID + ". Rotate the value, then remove it from the code."' \ - /tmp/gitleaks.json 2>/dev/null || true - fi - - if [ "$RC" != "0" ] && [ "$SOFT_FAIL" = "true" ]; then - echo "soft-fail is on — reporting only, not failing this job." - echo "Note: this is the one job worth flipping soft-fail off for first." - exit 0 - fi - exit "$RC" - - # ------------------------------------------------------------ house rules --- - house-rules: - name: house-rules - if: ${{ inputs.house-rules }} - runs-on: ubuntu-latest - timeout-minutes: 10 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - ref: ${{ github.event.pull_request.head.sha || github.sha }} - - - name: Check out the shared checker - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: tracebloc/.github - # Same repo this workflow lives in, so `main` is the version that - # matches a `@main` caller. `quality-ref` overrides it for testing. - ref: ${{ inputs.quality-ref }} - path: .quality-tools - persist-credentials: false - - - name: Run the house-rules checker - env: - ALL_FILES: ${{ inputs.all-files }} - BASE_SHA: ${{ github.event.pull_request.base.sha }} - CONFIG: ${{ inputs.house-rules-config }} - EXCLUDE: ${{ inputs.house-rules-exclude }} - SOFT_FAIL: ${{ inputs.soft-fail }} - run: | - set -uo pipefail - CHECKER=".quality-tools/scripts/house-rules.sh" - chmod +x "$CHECKER" - - set -- --github --summary "$GITHUB_STEP_SUMMARY" - if [ -n "${CONFIG:-}" ] && [ -f "$CONFIG" ]; then - set -- "$@" --config "$CONFIG" - fi - if [ -n "${EXCLUDE:-}" ]; then - set -- "$@" --exclude "$EXCLUDE" - fi - # The checker's own tree must not be linted as if it were the repo's. - set -- "$@" --exclude '.quality-tools/*' - if [ "$SOFT_FAIL" = "true" ]; then - set -- "$@" --soft-fail - fi - if [ "$ALL_FILES" = "true" ] || [ -z "${BASE_SHA:-}" ]; then - set -- "$@" --all - else - set -- "$@" --base "$BASE_SHA" - fi - - "$CHECKER" "$@" - - # RFC-0087 D3 (backend#3523): what a repo declares, ships and installs must - # be reachable from what it runs. Rides THIS job on purpose -- `quality / - # house-rules` is a required status check on every train repo's develop - # (measured 2026-09-09), so the checker is a gate the day it lands without - # 16 branch-protection edits; a new job name would be advice (rule 2) - # until each protection was hand-updated. - # - # WHOLE-TREE, never diff-scoped: removing an import elsewhere is what makes - # a pin dead, and that hunk is never in the PR that added the pin. - # - # Advisory by default (`dead-weight-soft-fail: true`) and hard per caller, - # so the fleet does not go red on promotion day. Python >= 3.11 for - # tomllib; ubuntu-latest ships 3.12. The version guard is a real refusal, - # not a skip: a checker that silently ran nothing would report clean. - - name: Run the dead-weight checker - if: ${{ inputs.dead-weight }} - env: - CONFIG: ${{ inputs.house-rules-config }} - EXCLUDE: ${{ inputs.house-rules-exclude }} - SOFT_FAIL: ${{ inputs.dead-weight-soft-fail }} - run: | - set -uo pipefail - CHECKER=".quality-tools/scripts/dead-weight.py" - python3 -c 'import sys; sys.exit(0 if sys.version_info >= (3, 11) else 1)' || { - echo "::error::dead-weight needs python3 >= 3.11 (tomllib); the runner has $(python3 --version)" - exit 1 - } - - set -- --root . --github --summary "$GITHUB_STEP_SUMMARY" - if [ -n "${CONFIG:-}" ]; then - set -- "$@" --config "$CONFIG" - fi - if [ -n "${EXCLUDE:-}" ]; then - set -- "$@" --exclude "$EXCLUDE" - fi - set -- "$@" --exclude '.quality-tools/*' - if [ "$SOFT_FAIL" = "true" ]; then - set -- "$@" --soft-fail - fi - - python3 "$CHECKER" "$@" - - # Whole-tree action-pin gate (backend#1492, D10). Hand-pinning does not hold: - # while one PR pinned jlumbroso/free-disk-space, a second PR added a NEW - # unpinned call site of the same action in a non-overlapping hunk of the same - # file — both merged cleanly, no conflict, caught only by a human reading the - # diff (#1446/#1449). This job makes that silent recurrence impossible. - # - # WHOLE-TREE, NEVER DIFF-SCOPED — deliberately. A guard that inspects only - # what a PR adds reports clean forever over the existing backlog (the exact - # failure mode of PR-mode gitleaks on frontend-app). Every run scans every - # workflow file in the checkout. - # - # THE GRAMMAR, and why not a YAML parse: pin-checking needs the `uses:` REF - # STRINGS, not the workflow's semantic structure. A strict line grammar - # (comment lines excluded; quoted refs unwrapped) covers every real workflow - # in the fleet, and anything it cannot parse — e.g. `uses: ${{ ... }}` - # expressions — is REPORTED as a finding rather than skipped: a guard that - # cannot verify must refuse to claim it did (RFC-1405 property 2; with - # soft-fail off, that refusal is red). - # - # ALLOWED forms (everything else is a finding): - # ./local/path repo-local actions - # tracebloc/<...>@main org reusables float on @main BY DECISION - # (RFC-1405 Q3; any other tracebloc ref is - # drift, same rule as caller-drift's) - # owner/action[/path]@<40-hex sha> D10 pin (trailing "# vX.Y.Z" comment is - # convention but not enforced here) - # docker://image@sha256: digest-pinned images only - action-pins: - name: action-pins - if: ${{ inputs.action-pins }} - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - ref: ${{ github.event.pull_request.head.sha || github.sha }} - - - name: Scan every workflow for unpinned action refs - env: - # The pin-specific override alone decides this job's posture, so a repo - # can arm the supply-chain check while the rest of its suite is still - # advisory during a migration. That is the whole reason this input is - # separate from `soft-fail`. - # - # It used to read `(inputs.soft-fail || inputs.action-pins-soft-fail)`. - # Because `soft-fail` defaults to true, that OR made the override - # incapable of EVER arming the job -- it could only ever weaken it -- - # while 16 of 16 callers pass `action-pins-soft-fail: false` expecting - # the opposite, and four of them carry a comment claiming the check is - # "Armed ... independent of soft-fail above". It was not (backend#1681). - SOFT_FAIL: ${{ inputs.action-pins-soft-fail && 'true' || 'false' }} - run: | - set -euo pipefail - python3 - <<'PY' - import glob, os, re, sys - - USES = re.compile(r"^\s*-?\s*uses:\s*(.+?)\s*$") - SHA_PIN = re.compile(r"^[\w.-]+/[\w.-]+(/[\w./-]+)?@[0-9a-f]{40}$") - TB_MAIN = re.compile(r"^tracebloc/[\w.-]+(/[\w./-]+)*@main$") - DOCKER_DIGEST = re.compile(r"^docker://\S+@sha256:[0-9a-f]{64}$") - - # Built from parts so this SOURCE never contains a literal GitHub - # expression opener - actionlint validates expressions inside run: - # blocks and rejects the bare sequence even inside a Python string. - EXPR_MARKER = "$" + "{{" - - findings = [] - files = sorted(glob.glob(".github/workflows/*.yml") + glob.glob(".github/workflows/*.yaml")) - for f in files: - for lineno, raw in enumerate(open(f, encoding="utf-8"), 1): - stripped = raw.lstrip() - if stripped.startswith("#"): - continue - # drop a trailing comment BEFORE matching, so "@sha # v4" parses - code = re.split(r"\s#", raw, 1)[0] - m = USES.match(code) - if not m: - continue - ref = m.group(1).strip().strip("'\"") - if ref.startswith("./"): - continue - # tracebloc/* is judged FIRST: the org convention is @main and - # nothing else, so a tracebloc ref frozen on a SHA is drift, - # not a pin (Bugbot, .github#159) - SHA_PIN must never see it. - if ref.startswith("tracebloc/"): - if TB_MAIN.match(ref): - continue - why = "tracebloc/* must be @main (Q3) - any other ref is drift" - elif SHA_PIN.match(ref) or DOCKER_DIGEST.match(ref): - continue - elif EXPR_MARKER in ref: - why = "expression ref - cannot be verified, so it is refused (property 2)" - else: - why = "not pinned to a 40-char commit SHA (D10)" - findings.append((f, lineno, ref, why)) - - if not files: - # Any repo CALLING this reusable necessarily has at least one - # workflow file (its own caller), so an empty glob means the - # checkout or working directory is wrong - a malfunction, not a - # clean tree. Malfunctions fail even under soft-fail: that flag - # governs FINDINGS, never the scan's own integrity (Bugbot, - # .github#159; the watchdog's findings-vs-malfunction split). - print("::error::action-pins scanned ZERO workflow files - the checkout or cwd is wrong; refusing to report a pass (backend#1492)") - sys.exit(2) - - level = "warning" if os.environ.get("SOFT_FAIL") == "true" else "error" - for f, lineno, ref, why in findings: - print(f"::{level} file={f},line={lineno}::unpinned action ref '{ref}' - {why} (backend#1492)") - - # The summary carries the FULL list, like ruff/format/gitleaks do: - # annotations cap at ten per step, so a backlog-carrying repo would - # otherwise show ten warnings and a bare number (Bugbot, .github#159). - summary = os.environ.get("GITHUB_STEP_SUMMARY") - if summary: - with open(summary, "a", encoding="utf-8") as out: - out.write(f"## action-pins\n\nscanned {len(files)} workflow file(s), {len(findings)} finding(s)\n\n") - if findings: - out.write("| file | line | ref | why |\n|---|---|---|---|\n") - for f, lineno, ref, why in findings: - out.write(f"| {f} | {lineno} | `{ref}` | {why} |\n") - - print(f"action-pins: {len(files)} file(s) scanned, {len(findings)} finding(s)") - if findings and os.environ.get("SOFT_FAIL") != "true": - sys.exit(1) - if findings: - print("soft-fail is on - reporting only, not failing this job.") - PY diff --git a/.github/workflows/conflict-gate.yml b/.github/workflows/conflict-gate.yml deleted file mode 100644 index 4e5761d..0000000 --- a/.github/workflows/conflict-gate.yml +++ /dev/null @@ -1,161 +0,0 @@ -name: Conflict gate - -# A MERGE-CONFLICTED PR RUNS NONE OF ITS `pull_request` WORKFLOWS -# (tracebloc/backend#2637). -# -# GitHub cannot compute a merge ref for a PR whose base has moved incompatibly, -# and a `pull_request` run is keyed on that merge ref. So a conflicted PR -# dispatches NO `pull_request` jobs at all -- not the drift guards, not the -# installer tests, not `Source-of-truth drift`, not `make drift`, not -# `gen-manifest.sh --check`. All of them are silently inactive, and the PR's -# rollup shows nothing red, because nothing ran. -# -# THIS IS NOT FIXABLE BY ADDING A TRIGGER. It is platform behaviour, and there is -# no merge commit for those jobs to check out. The only signal that can reach a -# conflicted PR is one written onto its head sha FROM OUTSIDE, which is what this -# does: a check run, via the Checks API. -# -# MEASURED, on `model-zoo#206`, 2026-08-27, while it was DIRTY against develop: -# * `actions/runs?head_sha=e7465ea` -> total_count 0. Nothing dispatched. -# * its rollup carried ONE entry: `Cursor Bugbot`, SUCCESS. Bugbot reviews the -# diff rather than via a `pull_request` trigger, so it is the one voice left, -# and it says green. -# * `model-zoo/develop` requires SEVEN contexts. None was present. -# * PRs #204 and #205, opened either side of it, each got their full matrix. -# The conflict was the only difference. -# -# THE SECOND SHAPE IS WORSE, and it is why `bricked-prs.py` is not already the -# answer. `backend#2257`, measured the same day and also CONFLICTING, had EIGHT -# workflow runs on its head sha and ALL ELEVEN of backend/develop's required -# contexts present and SUCCESS -- every one of them computed against a merge base -# that no longer exists. It reads unanimously green while conflicted. That -# watcher reasons from a required context being ABSENT, so with nothing missing it -# cannot see this PR at all. Only asking about mergeability directly does. -# -# WHY A CRON AND NOT `push`. A conflict is almost always created by a push to the -# BASE branch -- someone else merging -- which would be the causally exact -# trigger. But this workflow lives in `tracebloc/.github` and a `push` here fires -# only for this repo's own pushes; reacting to all 17 repos' pushes would need a -# caller in each. So the org-wide sweep is scheduled, like `bricked-prs.yml` and -# `kanban-reconcile.yml`, and a per-repo `push`-triggered caller is available -# later as a latency improvement rather than a correctness one. -# -# WHY NOT `pull_request_target`, which needs no merge ref either: NOTHING in this -# org uses it. Measured 2026-08-27, `actions/runs?event=pull_request_target` -# returns total_count 0 fleet-wide, so its behaviour on a conflicted PR is -# unverified here. Building a fail-closed guard on an unmeasured platform claim is -# CLAUDE.md rule 8's mistake, so this uses a trigger already proven by the other -# org-wide crons in this repo. -# -# THIS CONTEXT IS NOT REQUIRED ANYWHERE YET, DELIBERATELY (arm while green). -# It is landing as VISIBILITY: a red row on the PR where a reviewer is looking, -# instead of a finding in a cron log in another repo. Requiring it needs one more -# thing first -- a trigger that gives EVERY PR a status promptly. On a 30-minute -# sweep a PR opened at minute one would sit at "Expected -- waiting for status" -# until the next run, which is precisely the brick `bricked-prs.py` exists to -# hunt. Arming it is Lukas's call, and the follow-up is the per-repo caller above. -# -# Runs in tracebloc/.github only, like the other org-wide crons. - -on: - # THE SCHEDULE IS PAUSED, NOT REMOVED (2026-09-09, backend#3468). Every - # scheduled run since the first one on 2026-08-27 -- 534 runs, 0 successes, - # measured on the runs API the day this was written -- failed at "Mint an - # installation token" and never reached the Sweep step. Until backend#3242 the - # mint asked `permission-statuses: write`, a scope the release-train App's - # installation does not grant, so GitHub answered HTTP 422 ("The level of - # access for permissions requested are not granted to this installation") on - # every run. #446 moved the mint to `permission-checks: write`, and THE 422 DID - # NOT MOVE: scheduled run 34333719757 (09:16Z the same day, head 7ccaf30, so - # with #446 in) requested exactly `permission-pull-requests: read` + - # `permission-checks: write`; steps 1-4 succeeded, the mint failed with the - # same 422, Sweep was skipped. So the installation grants NEITHER `statuses` - # NOR `checks`. ~48 red runs a day that mark nothing is a gate nobody reads, - # and a cron that cannot start is not visibility -- so the trigger is off - # until the permission exists. - # - # TO RE-ARM: an org admin grants the tracebloc-release-train App the - # **Checks: Read and write** repository permission (GitHub App settings -> - # Permissions & events; the org installation then has to ACCEPT the new - # permission, or the mint keeps answering 422), dispatches ONE - # `workflow_dispatch` run and confirms it reaches the Sweep step -- not merely - # past the mint -- and only then uncomments the two `schedule:` lines below. - # Granting Commit statuses does nothing: the mint no longer asks for it. - # Nothing else changes: the mint and its permission request are deliberately - # left as they are, because they state what the job needs, and the fix is on - # the App, not here. The selftest pins this state -- `schedule` absent from the - # parsed triggers, the commented block still present -- so re-arming means - # updating `scripts/tests/conflict-gate-selftest.py` in the same PR. - # - # Every 30 minutes, once armed. The window this closes is measured in hours -- - # client#847 sat conflicted for ~3 of them with a hard failure invisible the - # whole time -- and a conflict is worth knowing about inside the review cycle, - # not after it. The sweep is one `gh pr list` per repo, so this is cheap. - # schedule: - # - cron: "*/30 * * * *" - workflow_dispatch: {} - -permissions: - contents: read - -concurrency: - group: conflict-gate - # NOT cancel-in-progress. A cancelled sweep leaves the check runs it had not yet - # written stale -- including `success` rows it was about to clear -- so a half - # sweep is worse than a late one. - cancel-in-progress: false - -jobs: - sweep: - name: Conflicted PRs - runs-on: ubuntu-latest - timeout-minutes: 20 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # Without this the `pip install` below hits PEP 668 on ubuntu-latest - # (24.04): the runner's Python is externally managed, pip refuses, the step - # fails and THE SWEEP NEVER RUNS. A scheduled guard that cannot start is - # worse than none -- nothing reports, and silence reads as "no conflicts". - # Same shape as bricked-prs.yml's own note. - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: '3.12' - - name: Install PyYAML - # conflict-gate.py imports caller-drift.py for its `gh` wrappers and the - # inventory loader, and that module hard-fails without PyYAML by design. - run: pip install --quiet pyyaml - # Authenticates as the tracebloc-release-train App (backend#2036) instead of - # one human's PAT. `owner:` yields an ORG-scoped installation token, which is - # what an org-wide sweep needs. No fallback to a PAT: a fallback would let a - # broken App path keep working silently. - # - # TWO SCOPES, AND NOTHING ELSE. This job does not read branch protection -- - # unlike bricked-prs.py it does not care what a base REQUIRES, only whether - # the PR conflicts -- so it needs no `administration`, and it lists PRs - # without `--base` so it needs no branch list either. - # - # `checks: write` AND NOT `statuses: write` (backend#3242). The signal is a - # check run, not a commit status. The tracebloc-release-train App's - # installation does not grant `statuses` at all, so the mint for - # `permission-statuses: write` was REFUSED before the sweep could run and - # this gate never executed. `checks` is the scope this signal actually - # needs, so the same red-row-on-the-PR signal is written with it -- but as - # of 2026-09-09 the installation does not grant `checks` either (scheduled - # run 34333719757 422'd on exactly this mint; see the `on:` block and - # backend#3468). The request stays as written: it names the grant to make. - # - # pull-requests: read the open-PR list and its two mergeability fields - # checks: write the whole point: writing the check run onto the head - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - permission-pull-requests: read - permission-checks: write - - name: Sweep - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: python3 scripts/conflict-gate.py diff --git a/.github/workflows/conformance-gate.yml b/.github/workflows/conformance-gate.yml deleted file mode 100644 index 7e0558e..0000000 --- a/.github/workflows/conformance-gate.yml +++ /dev/null @@ -1,189 +0,0 @@ -name: Conformance gate - -# The required check that gives caller-drift.yml teeth (backend#1608). -# -# THE HOLE THIS FILLS -# caller-drift.yml already runs on PRs that touch repo-inventory.yml, and it -# already fails when the inventory disagrees with reality. But it is not a -# required status check, so a PR that adds a repo to the train with drift goes -# red and can still be merged. The gate existed in trigger form with no teeth. -# -# WHY NOT JUST REQUIRE caller-drift ITSELF -# It carries a `paths:` filter. A path-filtered job never reports on a PR that -# misses the filter, and a required check that never reports leaves that PR -# pending forever -- the trap actionlint.yml's header documents. Dropping the -# filter instead would run a 20-repo API audit on every PR in this repo, which -# is why the filter is there. -# -# So: this job always runs and always reports. On a PR that does not touch the -# contract it is green in seconds with zero audit API calls. On a PR that does, -# it requires the audit's verdict for THIS head sha. -# -# WHY IT KEYS ON THE WORKFLOW FILE, NOT THE CHECK NAME -# Check-run names are not unique across workflows -- caller-drift.yml and -# standards-sync.yml both expose a job called `selftest`, so polling by name -# would happily accept the wrong workflow's verdict. Querying -# actions/workflows/caller-drift.yml/runs?head_sha=... is unambiguous. -# -# FAIL CLOSED (RFC-1405 property 2) -# Every branch that is not "the audit ran on this sha and succeeded" is RED, -# with a named reason: no run found before the timeout, a non-success -# conclusion, or an unreadable API. "Could not verify" and "verified" must -# never look alike -- an inventory change is exactly where a green that -# checked nothing is most expensive. - -on: - pull_request: - types: [opened, reopened, synchronize, ready_for_review] - -permissions: - contents: read - actions: read # read caller-drift's run conclusion - pull-requests: read # list the PR's changed files - -concurrency: - group: conformance-gate-${{ github.event.pull_request.number }} - cancel-in-progress: true - -jobs: - gate: - runs-on: ubuntu-latest - # Above the 40-minute poll budget below, so the budget's own error message - # is what a caller sees rather than an opaque job timeout. - timeout-minutes: 45 - steps: - - name: Require the conformance audit when the contract changes - env: - GH_TOKEN: ${{ github.token }} - REPO: ${{ github.repository }} - PR_NUMBER: ${{ github.event.pull_request.number }} - HEAD_SHA: ${{ github.event.pull_request.head.sha }} - # Changing any of these changes what the org's repos are contractually - # required to have, so the audit must have passed on this exact head. - # - # THIS FILE IS IN ITS OWN LIST, and that is not decoration. Without it, - # a PR touching ONLY conformance-gate.yml took the "does not touch the - # repo contract" branch below, printed that line and exited 0 -- so the - # gate could be defanged (drop repo-inventory.yml from GUARDED, or - # replace the poll with `exit 0`) on a green check, and a FOLLOW-UP PR - # then edited the contract against a gate that no longer guarded it. - # Two PRs, both green, no audit. (backend#1681.) - # - # What this does NOT fix, stated so nobody reads more into it than is - # there: `pull_request` workflows run from the PR's merge ref, so the - # gate evaluating a PR is that PR's version of the gate, and - # caller-drift.py is likewise supplied by the head. The integrity of - # this family still rests on the required human review. This closes the - # silent path; it does not make the gate self-hosting. - GUARDED: | - repo-inventory.yml - scripts/caller-drift.py - scripts/tests/caller-drift-selftest.py - .github/workflows/caller-drift.yml - .github/workflows/conformance-gate.yml - AUDIT_WORKFLOW: caller-drift.yml - # The budget MUST exceed the worst case of what it waits for, or - # fail-closed degrades into fail-annoying and people learn to bypass - # the gate. caller-drift is selftest (timeout 5m) + audit (timeout - # 30m) = 35m worst case; the old 40x30s = 20m budget would have gone - # red on a legitimately slow audit and blocked the PR pending a manual - # re-run. 80x30s = 40m, with the job timeout above it (Bugbot, - # .github#173). - POLL_SECONDS: 30 - POLL_TRIES: 80 - run: | - set -euo pipefail - - # --- which files does this PR touch? ------------------------------- - # --paginate, because a PR over one page would otherwise look like it - # touches only the first 30 files -- a truncated read that says "not - # guarded" is the silent pass this whole file exists to prevent. - # BOTH .filename and .previous_filename. A RENAME reports the new path in - # .filename only, so reading that alone lets `git mv repo-inventory.yml - # elsewhere.yml` sail through as "not guarded" -- the contract file - # moves out from under the audit and the gate waves it past. This is the - # same reason promote-repo.sh's publishable_delta matches both fields - # (a file renamed OUT of a published tree is still a publish-path - # change). Bugbot, .github#173. - if ! FILES=$(gh api --paginate "repos/${REPO}/pulls/${PR_NUMBER}/files" \ - --jq '.[] | .filename, (.previous_filename // empty)'); then - echo "::error::could not list this PR's changed files - refusing to guess whether the contract changed" - exit 1 - fi - # A PR that genuinely changes nothing is not a thing GitHub creates, so - # an empty list means the read failed in a way that did not set a - # non-zero exit. Treat it as unreadable, not as "touches nothing". - if [ -z "${FILES}" ]; then - echo "::error::the changed-file list came back empty - unreadable, not 'no files'" - exit 1 - fi - - TOUCHED="" - while IFS= read -r guarded; do - [ -z "${guarded}" ] && continue - # A here-string, NOT `printf ... | grep -q`. Under pipefail, grep -q - # closes the pipe on its first match; if the file list is large enough - # to still be buffering, printf takes SIGPIPE, the PIPELINE returns - # 141, and `if` reads a real hit as a miss -- so the gate reports "not - # guarded" and passes GREEN. That is exactly the silent pass this file - # exists to prevent, and it gets MORE likely the bigger the PR is. - # Measured: 20k filenames with the guarded path first -> rc=141, gate - # green. Same defect class as the PII gate's (backend#1409). - # A here-string is one command, so the status is grep's own. - if grep -qxF "${guarded}" <<< "${FILES}"; then - TOUCHED="${TOUCHED}${guarded} " - fi - done <<< "${GUARDED}" - - if [ -z "${TOUCHED}" ]; then - echo "This PR does not touch the repo contract - nothing to verify." - { - echo "### Conformance gate" - echo - echo "Not guarded - this PR touches none of the contract files." - } >> "$GITHUB_STEP_SUMMARY" - exit 0 - fi - - echo "Contract files touched: ${TOUCHED}" - echo "Requiring ${AUDIT_WORKFLOW} to have PASSED on ${HEAD_SHA}." - - # --- poll for the audit's verdict on THIS sha ---------------------- - # The audit is triggered by the same event, so it may not have been - # created yet when this job starts. Absence is never success: if no run - # for this sha appears within the budget, that is RED. - i=0 - CONCLUSION="" - STATUS="" - while [ "$i" -lt "${POLL_TRIES}" ]; do - if ! RESP=$(gh api "repos/${REPO}/actions/workflows/${AUDIT_WORKFLOW}/runs?head_sha=${HEAD_SHA}&per_page=100"); then - echo "::error::could not read ${AUDIT_WORKFLOW} runs for ${HEAD_SHA} - failing closed" - exit 1 - fi - STATUS=$(printf '%s' "$RESP" | jq -r '[.workflow_runs[]?] | sort_by(.run_started_at) | last | .status // ""') - CONCLUSION=$(printf '%s' "$RESP" | jq -r '[.workflow_runs[]?] | sort_by(.run_started_at) | last | .conclusion // ""') - if [ "${STATUS}" = "completed" ]; then - break - fi - i=$((i + 1)) - echo " audit status='${STATUS:-}' - waiting (${i}/${POLL_TRIES})" - sleep "${POLL_SECONDS}" - done - - if [ "${STATUS}" != "completed" ]; then - echo "::error::${AUDIT_WORKFLOW} did not complete for ${HEAD_SHA} within the poll budget (status='${STATUS:-none}'). Refusing to pass a contract change whose audit never reported." - exit 1 - fi - if [ "${CONCLUSION}" != "success" ]; then - echo "::error::${AUDIT_WORKFLOW} concluded '${CONCLUSION}' for ${HEAD_SHA}. A contract change must not merge on a failed or skipped audit." - exit 1 - fi - - echo "Audit passed on ${HEAD_SHA}." - { - echo "### Conformance gate" - echo - echo "Contract files touched: ${TOUCHED}" - echo - echo "${AUDIT_WORKFLOW} concluded success on ${HEAD_SHA}." - } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/customer-priority-bump-caller.yml b/.github/workflows/customer-priority-bump-caller.yml deleted file mode 100644 index 3c378c2..0000000 --- a/.github/workflows/customer-priority-bump-caller.yml +++ /dev/null @@ -1,48 +0,0 @@ -name: Customer priority bump - -# THIN CALLER (backend#2396). This repo is the last of the three that were never -# wired -- the others are `release-train` and `rfcs`, which now have callers too. -# All three are wired at the CALLER level. Caller presence is not the whole story: -# `design-system-v2`, `release-train` and `rfcs` still cannot fire, because they -# have no `work-type:*` labels for the trigger to match (backend#2598). -# -# NAMED `-caller.yml`, NOT `customer-priority-bump.yml`. In this repo that path -# already holds the REUSABLE, so a caller under the same name would overwrite the -# thing it calls. This repo pairs `.yml` with `-caller.yml` throughout: -# stale-backlog, set-pr-status, fr-gate, advance-deploy-env, code-quality, -# fr-pass-comment, kanban-closure. -# -# THIS FILE SHIPS AHEAD OF THE `exempt` -> `required` FLIP, and that ordering is -# forced rather than preferred -- the same deadlock #307 documented for the stale -# sweep. The audit reads caller PRESENCE from a repo's audited branch via the API -# (`--source-dir` covers only copies and reusables), so for `.github` itself the -# caller does not exist on `develop` until this merges. Bundling the flip here would -# fail `MISSING required caller` on its own PR, forever, because `gate` is required. -# -# KNOWN TRANSIENT: `exempt` plus a caller on the audited branch is ITSELF a finding -# (caller-drift.py:2260), so this repo's audit goes red between this merge and the -# flip. That window blocks every other GUARDED-file PR while it is open, so the flip -# should follow immediately -- and anything already queued on a green audit (e.g. -# .github#306) is better landed first. -# -# WHY NOW: .github#313 (backend#2348) added a second job to the reusable, -# `bug-to-ready`, which moves a `work-type:bug` card from `Backlog` to `Ready`. -# Without a caller, a bug filed in this repo still lands in `Backlog` -- the -# refinement queue nobody pulls from. The exemption's stated blocker (backend#1408 -# P3: "three dead inputs and hard-codes the `priority` label") has expired: #1408 is -# CLOSED COMPLETED, all four inputs are consumed, and the hard-coded `priority` -# label is RFC-BACKEND-0008 D5 working as decided. -# -# NO INPUTS AND NO `permissions:` BLOCK, matching all 16 working callers and the -# callee itself, which declares neither. Every input is defaulted -# (`trigger-label`, `bug-label`, `project-number: 2`, `org: tracebloc`), and a -# narrower grant than the callee needs fails the run at startup with no jobs. - -on: - issues: - types: [labeled] - -jobs: - bump: - uses: tracebloc/.github/.github/workflows/customer-priority-bump.yml@main - secrets: inherit diff --git a/.github/workflows/customer-priority-bump.yml b/.github/workflows/customer-priority-bump.yml deleted file mode 100644 index f3c865d..0000000 --- a/.github/workflows/customer-priority-bump.yml +++ /dev/null @@ -1,558 +0,0 @@ -name: Label-driven issue triage - -# Reusable workflow, called by a thin `customer-priority-bump.yml` caller in -# each wired repo on issues.types=labeled (callers pass `secrets: inherit` and -# no inputs). TWO label rules live here, both keyed on the label that was just -# added: -# -# from:customer -> add the binary `priority` label to the issue (D5). -# Writes no board field -- the Priority single-select was -# removed from the board under D5. -# work-type:bug -> move the issue's kanban card from `Backlog` to `Ready` -# (backend#2348). Defects skip refinement. -# -# WHY BOTH LIVE IN ONE FILE, AND WHY THE FILENAME NO LONGER MATCHES -# ---------------------------------------------------------------- -# The bug rule needs exactly one thing: to run on `issues: labeled` in every -# repo. This reusable is the ONLY place in the org that already does, and 16 of -# the 19 repos already call it. Adding a second reusable instead would need a -# `repo-inventory.yml` row for all 19 repos plus a caller rollout -- and -# `repo-inventory.yml` is guarded by `conformance-gate.yml`, so that change -# cannot merge until an org audit passes on its exact head sha, which every other -# merge invalidates. A rule that has been unimplemented since it was written does -# not need to wait behind that; it needs to run. -# -# The cost is honest and stated: the file name and the 16 per-repo caller names -# still say "customer priority bump", so a bug-label run shows up in each repo's -# Actions tab under that name. The workflow's own `name:` above is the half that -# could be fixed without touching 16 repos, so it was. Renaming the file is a -# caller rollout (BUGBOT.md property 1: land the callee first) and is still open -# work -- it no longer waits on anything, since the wiring below is complete. -# -# WIRED EVERYWHERE NOW. Every repo has a caller (`.github`'s landed under -# backend#2396) and `customer_priority_bump_caller_missing` is GONE from -# `repo-inventory.yml` -- no repo carries that key any more, so any text sending a -# reader to that lookup is describing a key that cannot answer. (The STRING still -# appears elsewhere in the repo, in `scripts/reason-citations.py` and in this -# comment; it is the INVENTORY that no longer records it. @aptracebloc on #350.) -# -# THE LABEL GAP IS CLOSED, AND NOW ASSERTED (backend#2598). This block used to -# record a live defect: repos carrying this caller with no `work-type:*` labels at -# all. GitHub silently DROPS a template label the target repo lacks -- no error, no -# annotation, no run -- so the issue was filed unlabelled, `bug-to-ready` never saw -# its label, and the card sat in `Backlog`, the refinement queue nobody pulls from. -# The exact miss backend#2348 was filed to close, surviving in the repos a -# caller-presence check calls done. -# -# Measured 2026-08-27 before the fix: `design-system-v2`, `release-train` and `rfcs` -# had ZERO of the seven triage labels, and `e2e-test-agent` had TWO of seven. All -# twenty now have all seven, and `.github/workflows/triage-labels.yml` runs daily to -# keep it that way -- it derives the label domain from the two producers (this file's -# `*-label` input defaults plus its `--add-label` writes, and every -# `.github/ISSUE_TEMPLATE/*.yml` `labels:` entry) and asserts each label exists in -# every repo the inventory declares. So a label deleted in a repo's Settings UI is a -# red run rather than a silent stop. -# -# TWO LESSONS WORTH MORE THAN THE FIX, both about the DOMAIN and not the answer: -# -# * `e2e-test-agent` was invisible to backend#2598 as filed, because that ticket -# derived over the `work-type:*` PREFIX. That repo had `work-type:bug` and -# `priority` and none of the other five, so a prefix sweep called it covered -# while four labels its templates apply were being dropped. Deriving from a -# prefix rather than from the producers is CLAUDE.md rule 6's vocabulary gap -- -# committed by the ticket that was written to close it. -# * An earlier version of this comment named TWO repos, because it checked the -# repos the previous prose happened to name instead of the domain the inventory -# declares (@aptracebloc, #350). -# -# Both are why the check parses the producers and the inventory and holds no list of -# its own. Do not reintroduce one here. - -on: - workflow_call: - inputs: - trigger-label: - description: "The label that means 'a customer asked for this'" - type: string - default: "from:customer" - bug-label: - description: "The label that means 'this is a defect' (skips refinement)" - type: string - default: "work-type:bug" - project-number: - description: "GitHub Projects v2 number (default: 2 = engineer kanban)" - type: number - default: 2 - org: - description: "GitHub org owning the project" - type: string - default: tracebloc - -jobs: - bump: - if: github.event.label.name == inputs.trigger-label - runs-on: ubuntu-latest - steps: - # Authenticates as the tracebloc-release-train App (backend#2036) instead of - # one human's PAT. This one needs `issues: write` -- it labels an issue in the - # calling repo -- which the App gained alongside `organization_projects: write`. - # `owner:` makes the token ORG-scoped so the same mint works for every caller. - # No fallback to the old PAT: a fallback would let a broken App path look like - # a working migration. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # LEAST PRIVILEGE, DERIVED FROM THIS JOB'S ONE CALL (backend#2157). The - # step below runs `gh issue edit --add-label priority` and nothing else: - # it resolves the issue and the repo's label set (Issues read) and writes - # the label (Issues write). `issues: write` subsumes both. - # - # Nothing here touches a project board, a pull request, or repository - # content -- so `organization-projects`, `pull-requests` and - # `contents:write` all drop, along with the administration/actions/checks - # reads the unscoped mint was carrying. - # - # `repositories:` is deliberately NOT narrowed, matching the reasoning on - # the `bug-to-ready` mint below: repo narrowing is measured in this org - # for a READ (add-to-kanban.yml, backend#2181) and not for a WRITE, and an - # unmeasured narrowing on a workflow that fires from every repo in the - # fleet fails red on every customer-labelled issue. - # - # NOT PROVEN BY READING: an under-scoped token fails at the call, not at - # the mint. The first `from:customer` label after this lands is the test. - permission-issues: write - - - name: Label the customer issue as priority - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - ISSUE_NUMBER: ${{ github.event.issue.number }} - REPO_FULL: ${{ github.repository }} - run: | - set -euo pipefail - # D5: priority is a binary "priority" label, not a project field. The - # label is visible on the issue itself and filterable everywhere; the - # Priority single-select field has been removed from the board. - # - # THE TRIGGER LABEL'S OWN DESCRIPTION NOW SAYS SO TOO, and for months it - # said the opposite (backend#2743). All twenty enrolled repos advertised - # `from:customer` as "auto-bumps Priority to P1" -- an outcome D5 deleted - # and this step stopped producing -- so the label that fires the rule - # promised a board value nobody would ever see. It was uniformly wrong - # because .github#364 created the labels four repos lacked by faithfully - # copying names, colours and descriptions off the repos that had them. - # Swept across all twenty on 2026-08-27, byte-identical, to: - # - # Filed from a customer request — automation adds the binary `priority` label - # - # Derived, not chosen: the sweep refused any text that did not name the - # label the `gh issue edit` line below actually writes, and refused the - # removed vocabulary outright. Change one, change the other. - # - # NOT UNDER A CHECK YET, said plainly rather than left to be assumed: - # `triage-labels-check.py` (.github#364) asserts a triage label EXISTS in - # every enrolled repo, not that its description still describes this step - # -- which is exactly how the stale text survived D5 with a green fleet. - # backend#2744 closes that, on top of #364 rather than beside it, because - # the file it extends is not on `develop` yet. - gh issue edit "$ISSUE_NUMBER" --repo "$REPO_FULL" --add-label priority - echo "-> Issue #$ISSUE_NUMBER labelled 'priority'" - - # --------------------------------------------------------------------------- - # A bug-labelled issue lands in `Ready`, not `Backlog` (backend#2348) - # --------------------------------------------------------------------------- - # THE RULE, quoted from `org-standards.md`: "label them `work-type:bug` (the - # Bug template does it) and the board moves the card straight into `Ready`". - # - # Nothing implemented it. `add-to-kanban.yml` adds every new issue at - # `Backlog` and no workflow read the label afterwards, so the rule was carried - # by whoever remembered. Measured 2026-08-22: seven bug-labelled tickets filed - # in one day (backend#2324, #2327, #2329, #2340, #2341, #2344, - # frontend-app#871) all landed in `Backlog` and all seven needed a hand-run - # mutation. A 100% miss rate is the tell that nothing does it at all -- a rule - # people mostly follow produces a mixed record. - # - # `Ready` is the queue engineers pull from; `Backlog` is the refinement queue - # nobody pulls from. That split is the whole point, so a defect filed correctly - # and labelled correctly was invisible to the people meant to pick it up. - bug-to-ready: - # A COST GATE, not the decision. It exists so that an unrelated label -- and - # every repo in the fleet fires this workflow on every `labeled` event -- does - # not mint an App token. The decision is `label_gate` in the step below, where - # it can be extracted, run and mutated by - # `scripts/tests/bug-to-ready-selftest.py`. `==` here is exact string - # equality, so this can only ever be STRICTER than that gate: it may skip work - # the gate would decline, never admit work the gate refuses. - if: github.event_name == 'issues' && github.event.label.name == inputs.bug-label - runs-on: ubuntu-latest - timeout-minutes: 10 - # Nothing here uses the caller's GITHUB_TOKEN: every call is made with the - # App installation token minted below. - permissions: {} - steps: - # LEAST PRIVILEGE, DERIVED FROM WHAT THIS RUNS (backend#2157), unlike the - # `bump` job above which still carries the App's full installation grant - # (`mint-scope.py`'s EXEMPT row for this file is about that job, not this - # one). This step reads one issue's project items and writes one - # single-select field on an org project -- so `issues: read` plus - # `organization-projects: write` is the whole requirement. - # - # `repositories:` is deliberately NOT narrowed here. The board write needs - # the ORG-level grant that `owner:` yields, and the interaction between repo - # narrowing and an org ProjectV2 write is measured for a READ - # (kanban-columns.yml, backend#2181) and not for a write. An unmeasured - # narrowing on a workflow that fires for every bug in the fleet fails red on - # every defect filed, and this job's whole point is that a defect should not - # need a human to notice it. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - permission-issues: read - permission-organization-projects: write - - - name: Promote the card to Ready, but only from Backlog - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - ORG: ${{ inputs.org }} - PROJECT_NUMBER: ${{ inputs.project-number }} - BUG_LABEL: ${{ inputs.bug-label }} - # THE TWO ANCHORS, QUOTED SO THE BOARD CHECK CAN SEE THEM. - # `scripts/kanban-columns-check.py` collects every board column name a - # WRITERS workflow quotes on a code line and asserts it exists on the - # live board -- which is what stops a rename in the Projects UI turning - # this job into a silent no-op. Unquoted YAML values are invisible to it. - SOURCE_COLUMN: "Backlog" - TARGET_COLUMN: "Ready" - EVENT_NAME: ${{ github.event_name }} - LABEL_ADDED: ${{ github.event.label.name }} - # A PR is not an issue. `issues` events never fire for pull requests, so - # this is belt and braces for a caller wired to `pull_request: labeled` - # -- both payload shapes are checked because they carry the PR in - # different places. - HAS_PR_PAYLOAD: ${{ github.event.pull_request != null || github.event.issue.pull_request != null }} - NUMBER: ${{ github.event.issue.number }} - REPO_FULL: ${{ github.repository }} - run: | - set -euo pipefail - REPO_NAME="${REPO_FULL#*/}" - - # ---- 1. is this an event this job may act on? -------------------- - # selftest:label-gate-start - # THE AUTHORITATIVE EVENT/LABEL GATE. The job-level `if:` above is a - # cost gate; this is the decision, written where a test can run it. - # - # EVERY REFUSAL NAMES ITSELF. A gate with five refusal paths and one - # bare failure cannot tell a test which path it took, so a case goes on - # passing while exercising a different refusal than its name claims - # (CLAUDE.md rule 10). - label_gate() { # $1=event $2=label added $3=label we act on $4=PR payload? - if [ "${1:-}" != "issues" ]; then echo "refuse:not-an-issues-event"; return; fi - if [ "${4:-}" = "true" ]; then echo "refuse:pull-request-payload"; return; fi - if [ -z "${2:-}" ]; then echo "refuse:unreadable-label"; return; fi - if [ -z "${3:-}" ]; then echo "refuse:no-configured-label"; return; fi - if [ "$2" != "$3" ]; then echo "refuse:other-label"; return; fi - echo proceed - } - GATE=$(label_gate "${EVENT_NAME:-}" "${LABEL_ADDED:-}" "${BUG_LABEL:-}" "${HAS_PR_PAYLOAD:-}") - case "$GATE" in - proceed) - echo "'${LABEL_ADDED}' added to issue #${NUMBER} - evaluating its card" ;; - refuse:other-label) - # THE ONLY GREEN REFUSAL, and the only one that is a normal event: - # this workflow runs on every `labeled` event in every wired repo, - # so most runs land here. - echo "::notice::'${LABEL_ADDED}' is not '${BUG_LABEL}' - nothing to do" - exit 0 ;; - *) - # Every other refusal is a payload this decision was never written - # for. Unreachable through the `if:` above, which is exactly why it - # is LOUD: if it ever fires, a caller has been wired to an event - # this job cannot judge, and a quiet exit 0 would hide that for as - # long as nobody reads the run log. - echo "::error::${GATE}: this job promotes a bug-labelled ISSUE and cannot judge this payload" >&2 - exit 1 ;; - esac - # selftest:label-gate-end - - # Every GraphQL call in this job -- both reads AND the write -- goes through - # here. `gh api graphql` exits 0 on an HTTP 200 that carries a GraphQL - # `errors[]` payload, so an exit code alone cannot tell a completed - # operation from a refused one. ONE function rather than a check per call - # site: the two reads each carried their own inline copy and the write - # carried none, which is exactly how the write came to be fail-open - # (Bugbot, .github#313). Rule 1 (derive, never restate) and rule 9 -- the - # selftest extracts THIS function by name, so the assertion and the - # mutation drive the same code the job runs. - reject_graphql_errors() { # $1=raw payload $2=the message to fail with - # UNPARSEABLE IS ITS OWN ARM, and it has to come first. `jq -e - # 'has("errors")'` exits non-zero BOTH when the key is absent and when - # the input is not JSON at all, so a single check would read a truncated - # or empty body as "no errors" -- fail-open on precisely the input that - # means "cannot tell". - if ! jq -e . >/dev/null 2>&1 <<< "$1"; then - echo "::error::$2 (the response was not readable JSON, so it cannot be" \ - "shown to be error-free)" >&2 - return 1 - fi - if jq -e 'has("errors")' <<< "$1" >/dev/null 2>&1; then - echo "::error::$2" >&2 - return 1 - fi - return 0 - } - - - # ---- 2. the board, read ONCE ------------------------------------ - # One query for the project id, the Status field id, the target option - # id AND the option ORDER. The order is what the monotonic gate is - # derived from, so reading it in the same response as the ids means the - # decision and the write cannot be made against two different boards. - # - # FAIL CLOSED on the read (backend#1729 rule 3): an unreadable board is - # not evidence that the card may move. - # shellcheck disable=SC2016 # the $names are GraphQL variables, not shell - keep literal - if ! PROJ=$(gh api graphql -f query=' - query($org: String!, $num: Int!) { - organization(login: $org) { - projectV2(number: $num) { - id - fields(first: 50) { - totalCount - nodes { - ... on ProjectV2SingleSelectField { id name options { id name } } - } - } - } - } - }' -F org="$ORG" -F num="$PROJECT_NUMBER"); then - echo "::error::could not read project #${PROJECT_NUMBER} - refusing to guess where this card sits" >&2 - exit 1 - fi - # A GraphQL `errors[]` payload AT EXIT 0 is a partial read, and a partial - # read of the option list is exactly the input that makes a position - # comparison meaningless (the shape `bugbot-gate.py` pins a mutation for). - reject_graphql_errors "$PROJ" \ - "the project read came back with GraphQL errors - a partial board is not a board" || exit 1 - - FIELD_TOTAL=$(jq -r '.data.organization.projectV2.fields.totalCount // -1' <<< "$PROJ") - # A field list longer than the page read means `Status` may be on a page - # nobody looked at, and "absent from the page I read" is not "absent". - if [ "$FIELD_TOTAL" -lt 0 ] || [ "$FIELD_TOTAL" -gt 50 ]; then - echo "::error::project #${PROJECT_NUMBER} reported ${FIELD_TOTAL} fields against a page of 50 -" \ - "the Status field may be unread. Paginate rather than treating a truncated read as complete." >&2 - exit 1 - fi - - PROJECT_ID=$(jq -r '.data.organization.projectV2.id // ""' <<< "$PROJ") - STATUS_FIELD=$(jq -r '.data.organization.projectV2.fields.nodes[]? - | select(.name=="Status") | .id' <<< "$PROJ") - # THE OPTION ID IS DERIVED, NEVER HELD. A stored `Ready` option id would - # keep writing after the board changed under it, and writing the WRONG - # column is strictly worse than writing nothing (backend#2348). - TARGET_OPT=$(jq -r --arg s "$TARGET_COLUMN" '.data.organization.projectV2.fields.nodes[]? - | select(.name=="Status") | .options[] | select(.name==$s) | .id' <<< "$PROJ") - if [ -z "$PROJECT_ID" ] || [ "$PROJECT_ID" = "null" ] \ - || [ -z "$STATUS_FIELD" ] || [ "$STATUS_FIELD" = "null" ] \ - || [ -z "$TARGET_OPT" ] || [ "$TARGET_OPT" = "null" ]; then - echo "::error::could not resolve the Status field or its '${TARGET_COLUMN}' option in" \ - "project #${PROJECT_NUMBER}. NOTHING WAS WRITTEN - this runs before any mutation." >&2 - exit 1 - fi - - # ---- 3. where is the card now? ---------------------------------- - # THE ISSUE'S OWN projectItems, not a scan of the project. Project #2 - # carries ~700 items, so a project-side scan is a pagination bug waiting - # to happen; the issue knows which cards it has. `totalCount` is read so - # a card beyond the page cannot be reported as "not on the board". - # - # The retry is for the `opened`+template case: the Bug template applies - # the label at creation, so this can race `add-to-kanban.yml`. - ITEM_ID=""; CURRENT_COL=""; ARCHIVED="" - for attempt in 1 2 3 4 5; do - # shellcheck disable=SC2016 # the $names are GraphQL variables, not shell - keep literal - if ! IRESP=$(gh api graphql -f query=' - query($org: String!, $repo: String!, $num: Int!) { - repository(owner: $org, name: $repo) { - issue(number: $num) { - projectItems(first: 20) { - totalCount - nodes { - id isArchived project { number } - status: fieldValueByName(name: "Status") { - ... on ProjectV2ItemFieldSingleSelectValue { name } - } - } - } - } - } - }' -F org="$ORG" -F repo="$REPO_NAME" -F num="$NUMBER"); then - echo "::error::could not read issue #${NUMBER}'s project items - an unreadable card is not" \ - "a card that may be left where it is" >&2 - exit 1 - fi - reject_graphql_errors "$IRESP" \ - "the project-items read came back with GraphQL errors - unreadable, not 'no card'" || exit 1 - PI_TOTAL=$(jq -r '.data.repository.issue.projectItems.totalCount // -1' <<< "$IRESP") - # A MISSING connection is not an EMPTY one. `totalCount` is absent when - # the issue itself did not resolve, and retrying that four more times - # then reporting "not on the board" would name the wrong cause. - if [ "${PI_TOTAL:--1}" -lt 0 ]; then - echo "::error::the read did not describe issue #${NUMBER}'s project items at all" \ - "(no totalCount) - an unreadable response is not an empty one" >&2 - exit 1 - fi - NODE=$(jq -c --arg n "$PROJECT_NUMBER" 'first(.data.repository.issue.projectItems.nodes[]? - | select(.project.number == ($n | tonumber))) // {}' <<< "$IRESP") - ITEM_ID=$(jq -r '.id // ""' <<< "$NODE") - if [ -n "$ITEM_ID" ] && [ "$ITEM_ID" != "null" ]; then - CURRENT_COL=$(jq -r '.status.name // ""' <<< "$NODE") - ARCHIVED=$(jq -r 'if .isArchived == true then "true" else "false" end' <<< "$NODE") - break - fi - if [ "${PI_TOTAL:-0}" -gt 20 ]; then - echo "::error::issue #${NUMBER} is on ${PI_TOTAL} projects and the kanban card is not among" \ - "the 20 read - paginate rather than reporting a truncated read as 'not on the board'" >&2 - exit 1 - fi - echo "not on project #${PROJECT_NUMBER} yet (attempt ${attempt}/5) - add-to-kanban may still be running" - if [ "$attempt" -lt 5 ]; then sleep 5; fi - done - - # FAIL CLOSED, LOUDLY, AND THE DIRECTION IS THE DECISION HERE. - # - # The three sibling consumers of the board chose their failure paths as a - # SET (backend#2243), and this one resembles `advance-deploy-env.yml`: no - # fallback, abort. `kanban-closure-router.yml` exits 0 on a card it cannot - # find because declining to write is its conservative end state -- it is - # protecting shipped state from being overwritten. Declining here is not - # conservative: it reproduces the exact defect this job exists to fix, a - # defect parked in `Backlog` that nobody reads. And `kanban-reconcile.yml` - # may skip because it runs weekly and gets another go; a `labeled` event - # fires ONCE, so a green no-op is the last anyone hears of it. - if [ -z "$ITEM_ID" ] || [ "$ITEM_ID" = "null" ]; then - echo "::error::issue #${NUMBER} carries '${BUG_LABEL}' but is not on project" \ - "#${PROJECT_NUMBER} after 5 tries. It needs '${TARGET_COLUMN}' and this run could not" \ - "put it there - check this repo's add-to-kanban caller, then set the column by hand." >&2 - exit 1 - fi - - # ---- 4. may the card move? -------------------------------------- - # Byte-identical to `kanban-closure-router.yml`'s, and asserted so by - # the selftest: both read the same `$PROJ` shape, so a divergence would - # be a defect rather than a difference. - col_index() { - echo "$PROJ" | jq -r --arg s "$1" \ - '[.data.organization.projectV2.fields.nodes[] - | select(.name=="Status") | .options[].name] | index($s) // -1' - } - # selftest:monotonic-start - # MONOTONIC. Automation in this org never moves a card backward, and the - # direction is asked of the BOARD rather than restated here: a rank table - # in this file would agree with itself while disagreeing with reality - # (backend#1729 rule 1). - # - # promote the card is at $SOURCE_COLUMN, or is on the board with no - # Status at all. Those are the only two states from which - # $TARGET_COLUMN is forward. An unplaced card is not "past - # Ready" -- leaving it unplaced keeps it invisible, which is - # the complaint. - # hold anywhere else: `In progress`, `Code review`, `On dev`, - # `FR on staging`, `Ready for prod`, `Prod`, `Done`, - # `Cancelled`, `North Stars`, already `Ready`, or archived. - # The label routinely arrives AFTER triage moved the card, and - # demoting a shipped card would un-ship it on the board. - # unknown a column the board does not report. Nothing can be said - # about "forward" from a position that cannot be placed. - # noboard an anchor is missing, or $TARGET_COLUMN does not sit - # strictly AFTER $SOURCE_COLUMN. The second half is the - # monotonicity assertion itself: position is load-bearing, so - # one drag of the Status options in the UI can make this - # "promotion" a demotion. Checked FIRST, before the - # no-Status shortcut, because a board that cannot be trusted - # to be in pipeline order cannot be trusted for any card - # (backend#1994 is the same hole one file over: existence was - # checked and ORDER was not). - promote_decision() { # $1=the card's current column $2=isArchived - _s=$(col_index "${SOURCE_COLUMN}"); _t=$(col_index "${TARGET_COLUMN}") - if [ "$_s" -lt 0 ] || [ "$_t" -lt 0 ] || [ "$_s" -ge "$_t" ]; then echo noboard; return; fi - if [ "${2:-}" = "true" ]; then echo hold; return; fi - case "${1:-}" in - ""|"No status") echo promote; return ;; - esac - _c=$(col_index "$1") - if [ "$_c" -lt 0 ]; then echo unknown; return; fi - if [ "$_c" -eq "$_s" ]; then echo promote; else echo hold; fi - } - # selftest:monotonic-end - - # selftest:policy-start - _d=$(promote_decision "${CURRENT_COL:-}" "${ARCHIVED:-}") - case "$_d" in - promote) - _write=yes ;; - hold) - _write=no - echo "::notice::#${NUMBER} sits in '${CURRENT_COL:-}' (archived=${ARCHIVED:-false})," \ - "not '${SOURCE_COLUMN}' - leaving it alone, automation never moves a card backward" ;; - unknown) - # NOT a quiet decline. The router's `unknown` arm exits 0 because - # there, declining is the safe end state; here it means a card that - # should be in the pull queue is somewhere this job cannot place, - # and nobody would ever hear about it. - echo "::error::#${NUMBER} sits in '${CURRENT_COL:-}', which project #${PROJECT_NUMBER}" \ - "does not report as a Status option - refusing to guess whether '${TARGET_COLUMN}' is forward" >&2 - exit 1 ;; - noboard) - echo "::error::project #${PROJECT_NUMBER} does not place '${SOURCE_COLUMN}' strictly before" \ - "'${TARGET_COLUMN}' in its Status options. Promoting would be a DEMOTION, so nothing was" \ - "written. Check the column order and names on the board." >&2 - exit 1 ;; - *) - # No fall-through. An unrecognised verdict is a code defect, and the - # one thing it must not do is reach the write. - echo "::error::unrecognised promotion verdict '${_d}' - refusing to write" >&2 - exit 1 ;; - esac - # selftest:policy-end - - if [ "$_write" != "yes" ]; then - exit 0 - fi - - # shellcheck disable=SC2016 # the $names are GraphQL variables, not shell - keep literal - if ! WRESP=$(gh api graphql -f query=' - mutation($p: ID!, $i: ID!, $f: ID!, $o: String!) { - updateProjectV2ItemFieldValue(input: { - projectId: $p, itemId: $i, fieldId: $f, - value: {singleSelectOptionId: $o} - }) { projectV2Item { id } } - }' -F p="$PROJECT_ID" -F i="$ITEM_ID" -F f="$STATUS_FIELD" -f o="$TARGET_OPT"); then - echo "::error::the Status write failed for issue #${NUMBER} - the card is still in" \ - "'${CURRENT_COL:-}' and needs '${TARGET_COLUMN}' by hand" >&2 - exit 1 - fi - # The write goes through the SAME rejection as the two reads. This event - # fires exactly ONCE, so a false success here is permanent: the step logs a - # Backlog -> Ready move, stays green, and the card never moved. - reject_graphql_errors "$WRESP" \ - "the Status write for issue #${NUMBER} came back with GraphQL errors at exit 0 - the card is still in '${CURRENT_COL:-}' and needs '${TARGET_COLUMN}' by hand" || exit 1 - # Absence of errors is not presence of the write. Confirm the mutation - # returned the item it claims to have moved -- the same "did it actually - # land" discipline merge-confirm.sh applies to a merge. - if [ -z "$(jq -r '.data.updateProjectV2ItemFieldValue.projectV2Item.id // empty' <<< "$WRESP")" ]; then - echo "::error::the Status write for issue #${NUMBER} returned no item id, so the move is UNCONFIRMED - the card needs '${TARGET_COLUMN}' by hand" >&2 - exit 1 - fi - echo "-> issue #${NUMBER}: Status '${CURRENT_COL:-}' -> '${TARGET_COLUMN}'" - { - echo "### Defect skipped refinement" - echo - echo "\`${BUG_LABEL}\` on issue #${NUMBER}: \`${CURRENT_COL:-}\` -> \`${TARGET_COLUMN}\`" - } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/fr-gate-caller.yml b/.github/workflows/fr-gate-caller.yml deleted file mode 100644 index a07247c..0000000 --- a/.github/workflows/fr-gate-caller.yml +++ /dev/null @@ -1,50 +0,0 @@ -name: FR gate - -# Per-repo caller. Blocks merges to staging/main unless every contained kanban item -# is in "Ready for staging" or "Ready for prod" respectively. All logic lives in -# tracebloc/.github/.github/workflows/fr-gate.yml. -# -# This repo HOSTS that reusable, so this caller is a self-reference — and it is -# pinned `@main` like every other caller, deliberately (RFC-BACKEND-1405 open -# question 3, answered: branch promotion, one standard process for every repo). -# -# The consequence is worth naming rather than discovering: a change to fr-gate.yml -# on `develop` is NOT gating its own PR. The gate that runs is main's. So this repo -# cannot self-test a gate change before promoting it, which is precisely why the -# develop -> staging -> main path matters here more than anywhere else: `staging` -# is the only place a new gate runs against real board state before ten other -# repos start consuming it at `@main`. -# -# NO `branches:` FILTER, DELIBERATELY (backend#2840). The obvious shape here is -# `branches: [staging, main]` — run the gate only where it can block. It is WRONG, -# and the way it is wrong is invisible: `branches:` filters on the PR's base AT -# EVENT TIME, so a PR retargeted OUT of the set (main -> develop) fires `edited` -# with base=develop, the filter rejects it, and the gate never re-runs. The last -# run stays FAILURE, welded to the head sha, and `gate` is a REQUIRED check — so -# the PR is blocked forever with nothing able to clear it (`synchronize` needs a -# push; a retarget has none). Measured on .github#388: three stale FAILUREs, and -# only an unrelated commit to mint a fresh sha cleared it. -# -# So the trigger is unfiltered and the DECISION moves into the job, where it CAN -# see the base: fr-gate.yml maps a non-promotion base to an empty `required`, and -# every gating step is `if: steps.target.outputs.required != ''`, so on develop the -# job does two echoes and reports `gate` SUCCESS — which is exactly what supersedes -# the stale FAILURE on a retarget. "Derive, never restate": the base is read once, -# in the one place that runs on every event. Costs a runner-second per develop PR; -# removes the class. -# -# `edited` is still load-bearing for the half that DID work — main <-> staging -# retargets, where the new base still gates (.github#237, backend#1945) — and is -# now also the event that re-runs the gate green on a retarget out to develop. -# -# `master` never appears here: this repo has never had one, and the base filter -# that would have listed it is gone regardless (backend#1428, backend#2840). - -on: - pull_request: - types: [opened, reopened, synchronize, ready_for_review, labeled, unlabeled, edited] - -jobs: - gate: - uses: tracebloc/.github/.github/workflows/fr-gate.yml@main - secrets: inherit diff --git a/.github/workflows/fr-gate.yml b/.github/workflows/fr-gate.yml deleted file mode 100644 index 3f66b48..0000000 --- a/.github/workflows/fr-gate.yml +++ /dev/null @@ -1,212 +0,0 @@ -name: FR gate - -# Reusable workflow. Called from each active repo on pull_request events -# targeting staging, main, or master. Blocks the merge unless every item -# included in the promotion is at or beyond the correct "Ready for X" column: -# -# target = staging → all items must be at "On dev" or later (automatic — D6) -# target = main/master → all items must be at "Ready for prod" or later -# -# "or later" means an item already further down the pipeline (e.g. "Prod") -# satisfies an earlier gate ("On dev") instead of being falsely -# blocked. See the rank() helper below for the canonical stage ordering. -# -# This enforces the "FR must pass before promotion" rule. It runs as a -# required status check (configured via branch protection) so the merge -# button stays grey until the gate passes. -# -# Override: add the "skip-fr-gate" label to bypass the check (for hotfixes -# or emergency releases). The label is a deliberate, visible action so we -# can audit overrides after the fact. -# -# Item discovery: authoritative commit->PR attribution via the -# commits/{sha}/pulls API between base and head (see the discovery step). -# Release-train promotion PRs (head release-train/*) are transparent -# plumbing: they never appear as gated items and never vouch for their -# range -- attribution passes through to the cargo PRs they carry. If -# discovery yields no PRs at all, falls back to checking the promotion -# PR's own Status. - -on: - workflow_call: - inputs: - project-number: - type: number - default: 2 - org: - type: string - default: tracebloc - quality-ref: - description: >- - Ref of tracebloc/.github the walk script (scripts/fr-gate-walk.sh) is - taken from. Callers pin this workflow at `@main`, so `main` is the - matching version. Override only to test a change to the walk before - it merges. The release train pins the same file by COMMIT SHA. - type: string - default: "main" - -jobs: - gate: - runs-on: ubuntu-latest - steps: - - name: Determine required Status from target branch - id: target - env: - BASE: ${{ github.base_ref }} - run: | - case "$BASE" in - staging) echo "required=On dev" >> "$GITHUB_OUTPUT" ;; - main|master) echo "required=Ready for prod" >> "$GITHUB_OUTPUT" ;; - *) echo "required=" >> "$GITHUB_OUTPUT" ;; - esac - - - name: Skip if not promoting to staging/main/master - if: steps.target.outputs.required == '' - run: echo "Target branch '${{ github.base_ref }}' is not gated — nothing to enforce." - - - name: Check for skip-fr-gate label - id: skip - if: steps.target.outputs.required != '' - env: - LABELS: ${{ toJSON(github.event.pull_request.labels.*.name) }} - run: | - if echo "$LABELS" | grep -q '"skip-fr-gate"'; then - echo "skip=true" >> "$GITHUB_OUTPUT" - echo "::warning::FR gate bypassed via 'skip-fr-gate' label." - # (promotion-shape guard below is also bypassed by this label) - else - echo "skip=false" >> "$GITHUB_OUTPUT" - fi - - - name: Promotion-shape guard — only the train promotes - if: steps.target.outputs.required != '' && steps.skip.outputs.skip != 'true' - env: - HEAD_REF: ${{ github.event.pull_request.head.ref }} - LABELS: ${{ toJSON(github.event.pull_request.labels.*.name) }} - run: | - # Manual promotion PRs bypass the release train's Bugbot soft-gate, - # its tagging, and its run records — and indirect-merge the train's - # own PRs (backend#1336 / averaging-service, 2026-07-29). Promotions - # into staging/main/master are the train's job. Sanctioned lanes: - # release-train/* the train's own mirror PRs - # hotfix-backmerge/* hotfix.yml's automated back-merges - # 'hotfix' label single-repo emergency fix (D14a, audited) - # 'skip-fr-gate' label full gate override (handled above, audited) - case "$HEAD_REF" in - release-train/*|hotfix-backmerge/*) exit 0 ;; - esac - if echo "$LABELS" | grep -q '"hotfix"'; then - exit 0 - fi - echo "::error::Manual promotion PRs are retired — promotions into this branch go through the release train (tracebloc/release-train → Actions → 'Release train'). For a single-repo emergency prod fix use the 'hotfix' label; 'skip-fr-gate' remains the audited full override." - exit 1 - - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - if: steps.target.outputs.required != '' && steps.skip.outputs.skip != 'true' - with: - fetch-depth: 0 - ref: ${{ github.event.pull_request.head.sha }} - - # THE WALK LIVES HERE, NOT IN THE CALLING REPO (backend#3323, RFC-0075 D6). - # The two steps below used to carry it inline; the release train needs the - # same walk to cut the prod hop at the FR frontier, and a second copy in - # release-train is the drift rule 9 forbids. So the script is checked out - # of tracebloc/.github -- same shape as bugbot-gate.yml and code-quality.yml - # -- into a path beside the caller's checkout. `.github` is public, so this - # needs no token; `persist-credentials: false` because the walk reads with - # the App token below, never with the checkout's. - - name: Check out the shared walk script - if: steps.target.outputs.required != '' && steps.skip.outputs.skip != 'true' - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: tracebloc/.github - ref: ${{ inputs.quality-ref }} - path: .fr-gate-tools - persist-credentials: false - - # Authenticates as the tracebloc-release-train App (backend#2036) instead of a - # human's PAT. `owner:` yields an ORG-scoped installation token, which this gate - # needs: it reads the org PROJECT, and a repo-scoped token cannot. - # - # READ-ONLY, and that is the whole reason this gate could move. It reads - # `commits/{sha}/pulls`, `compare/{base}...{head}` and the ProjectV2 graphql -- - # every one covered by permissions the App already holds. It never writes a - # card, a label or a comment; the gate's only output is its own conclusion. - # - # Gated behind the same `if:` as the steps that use it, so a PR this gate skips - # (not promoting, or `skip-fr-gate`) mints nothing. - # - # NO FALLBACK TO THE PAT: a fallback would let a broken App path keep working - # silently, and this gate blocking wrongly is far better than it passing wrongly. - - name: Mint an installation token - id: app-token - if: steps.target.outputs.required != '' && steps.skip.outputs.skip != 'true' - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # THREE READS AND NOTHING ELSE (backend#2157). Without these it minted - # the App's whole set -- contents:write, issues:write, - # organization_projects:write, administration:read -- across every repo - # the installation covers, to run a gate that writes nothing at all. - # A pure reader holding org-wide write is the sharpest mismatch of the - # four this ticket found. - # - # NOT DERIVED FROM A TEMPLATE -- from the three calls the comment above - # already names, which is why it could be checked rather than guessed: - # commits/{sha}/pulls -> contents: read + pull-requests: read - # compare/{base}...{head} -> contents: read - # ProjectV2 graphql -> organization-projects: read - # - # `actions/checkout` above is NOT on this token -- it takes the job's - # default GITHUB_TOKEN, so nothing here needs contents:write for the - # checkout either. - # - # IF THIS SCOPE IS SHORT the gate fails closed and blocks promotions, - # which is loud, immediate and recoverable with the `skip-fr-gate` - # label. That is the failure direction this gate already chose for - # itself: "blocking wrongly is far better than passing wrongly". - permission-contents: read - permission-pull-requests: read - permission-organization-projects: read - - - name: Discover items in this promotion - id: items - if: steps.target.outputs.required != '' && steps.skip.outputs.skip != 'true' - env: - BASE_REF: ${{ github.base_ref }} - HEAD_SHA: ${{ github.event.pull_request.head.sha }} - REPO_FULL: ${{ github.repository }} - GH_TOKEN: ${{ steps.app-token.outputs.token }} - REPO: ${{ github.repository }} - run: | - set -euo pipefail - # Make the base branch tip available locally for the diff. - git fetch origin "$BASE_REF":"refs/remotes/origin/$BASE_REF" --depth=200 2>/dev/null || true - BASE_SHA=$(git rev-parse "origin/$BASE_REF") - export BASE_SHA - # Commit->PR attribution, the promotion-PR transparency, the two-pass - # merge handling (Bugbot #72/#73) and the three-dot file count all live - # in the script's header and body -- one implementation, shared with the - # train's frontier resolver. It writes prs= / unattributed= / - # changed_files= to $GITHUB_OUTPUT exactly as this step always did. - bash .fr-gate-tools/scripts/fr-gate-walk.sh discover - - - name: Verify each item is in required Status - if: steps.target.outputs.required != '' && steps.skip.outputs.skip != 'true' - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - ORG: ${{ inputs.org }} - PROJECT_NUMBER: ${{ inputs.project-number }} - REPO_FULL: ${{ github.repository }} - REQUIRED: ${{ steps.target.outputs.required }} - PRS: ${{ steps.items.outputs.prs }} - UNATTRIB: ${{ steps.items.outputs.unattributed }} - PROMOTION_PR: ${{ github.event.pull_request.number }} - CHANGED_FILES: ${{ steps.items.outputs.changed_files }} - run: | - # The rank() ordering, resolve_status's race-aware retries and the - # fail-closed verdict (blocked / missing / unreadable / unattributable - # all block) are the script's `verify` mode. Same text, same exit code. - bash .fr-gate-tools/scripts/fr-gate-walk.sh verify diff --git a/.github/workflows/fr-pass-comment-caller.yml b/.github/workflows/fr-pass-comment-caller.yml deleted file mode 100644 index afc886f..0000000 --- a/.github/workflows/fr-pass-comment-caller.yml +++ /dev/null @@ -1,20 +0,0 @@ -name: FR pass comment - -# Template for each active repo. Copy this file into a repo's -# .github/workflows/ directory to enable the /fr-pass comment shortcut, which -# advances a kanban item from "FR on staging" → "Ready for prod" — the single -# functional-review gate. -# -# RFC-BACKEND-1405 D6 retired the dev-side review: "On dev" is set automatically -# when the release train merges to develop, so there is no "FR on dev" column. -# This file is the source the per-repo callers are copied from, so a stale -# transition named here propagates on the next copy. - -on: - issue_comment: - types: [created] - -jobs: - advance: - uses: tracebloc/.github/.github/workflows/fr-pass-comment.yml@main - secrets: inherit diff --git a/.github/workflows/fr-pass-comment.yml b/.github/workflows/fr-pass-comment.yml deleted file mode 100644 index 5b74b69..0000000 --- a/.github/workflows/fr-pass-comment.yml +++ /dev/null @@ -1,608 +0,0 @@ -name: FR pass comment handler - -# Reusable workflow. Called from each active repo on issue_comment created. -# Listens for "/fr-pass" comments on PRs/issues that are currently in -# "FR on staging" and advances them one column: -# FR on staging → Ready for prod -# (D6: the dev-side review is gone — "On dev" is automatic — so /fr-pass only -# applies at staging, the single functional-review gate.) -# -# NEVER NO-OP IN SILENCE (backend#1319) -# Every terminating path below leaves a signal on the thread: 👍 when the card -# moved, 👎 plus a one-paragraph comment saying why when it did not. The single -# deliberate exception is a comment that merely *mentions* the string without -# issuing the command ("/fr-passport", "you can /fr-pass it later") — reacting -# to prose would turn the bot into noise. -# -# The rule that follows from that: no decision may live in the job-level `if`. -# A false job `if` skips the job with zero steps, so there is nothing left to -# react or comment with — the run just reports `skipped` and the reviewer -# believes the card moved. The job `if` is therefore only a cheap "is this even -# about /fr-pass" filter; authorisation, self-signoff, board membership and the -# column check all happen in steps that can report. -# -# A REFUSAL IS NOT A SUCCESS (backend#1413) -# Exactly three outcomes exit 0 — `advanced`, `already-advanced` and -# `not-a-command`. EVERY other terminating path exits 1, including -# `not-authorised`, `self-signoff`, `not-on-project`, `ambiguous-item`, -# `no-status` and `wrong-column`, where the handler worked perfectly and -# correctly declined to move the card. -# -# Failing a deliberate refusal reads oddly, and it is the point. `gh run list` -# is the only place anyone reads many threads at once, and green there means -# "the card moved". While those six exited 0, 26 green runs concealed three -# cards that had not moved; the refusal was diagnosed as a bug in the advance -# logic and the cards were then moved by hand through the project API — -# bypassing the control rather than satisfying it. The 👎 and its note stay the -# real explanation; a red run is just the cheapest signal that one exists. -# -# WHY A COMMENT GATE -# It leaves a record on the thread, so reviewers can see who passed FR and -# when. Dragging the card on the kanban works too — this is the shortcut. - -on: - workflow_call: - inputs: - project-number: - type: number - default: 2 - org: - type: string - default: tracebloc - allow-self-signoff: - description: >- - Allow the PR/issue author to sign off their own promotion on ANY item. - Default TRUE as of 2026-08-01 (RFC-BACKEND-1405 D6). D30's original - two-pairs-of-eyes rule is retired: code review already puts a second - human on the diff, and most implementation is now AI-assisted, so - requiring a *different* human for the functional review was ceremony - rather than a control. Functional review still requires a human to - assert it — it just no longer has to be a different one. Set this to - false on a caller to restore the stricter rule for one repo. Changing - this default changes the policy org-wide, since the per-repo callers - pass no inputs. - type: boolean - default: true - -# GITHUB_TOKEN stays read-only and is NOT used for any write here. Two reasons it -# cannot be: the org default is `default_workflow_permissions: read`, and a called -# workflow can only narrow the caller job's token, never widen it — so a -# `permissions: issues: write` block on this workflow would be silently -# ineffective while looking like it worked. That is exactly how the 👍/👎 reaction -# came to fail with "Resource not accessible by integration (HTTP 403)" on every -# run, including the ones that advanced their card successfully. All writes -# therefore go through PROJECTS_KANBAN_TOKEN, which already has cross-repo write -# (wip-limit-check.yml posts PR comments with it). The block below pins GITHUB_TOKEN -# to contents:read and nothing else, which is also why there is no GITHUB_TOKEN -# fallback on the writes: it would be dead code that logs a misleading retry. -permissions: - contents: read - -jobs: - advance: - # The job-level conditions are a cheap "is this even about /fr-pass" filter - # and nothing more; no decision lives here (see NEVER NO-OP IN SILENCE). - # - # The first is deliberately loose — GitHub's contains() is a case-insensitive - # substring test, so "/FR-PASS", leading whitespace and trailing text all get - # through. Loose is the point: a comment that gets in but turns out not to be - # the command exits silently from the first step, whereas a genuine command - # that never gets in can never be reported. The precise match lives in the - # step. - # - # The other two stop this handler answering itself (backend#1413). Every - # refusal note below contains the literal string `/fr-pass`, so posting one - # re-fired this workflow — 6 of the 26 runs on 2026-08-01 were the bot reading - # its own comment. That only ever terminated because the step's anchored grep - # rejects a line starting with an emoji, which is luck, not a design: reflow a - # note so `/fr-pass` lands first on its line and it loops until the concurrency - # limits bite. - # - # * Not a bot account. This is the guard the loop *should* have needed, and - # today it catches nothing — the writes use PROJECTS_KANBAN_TOKEN, a PAT - # belonging to a human, so the handler's own comments arrive under that - # person's login, indistinguishable from their real reviews. Keyed on the - # account type rather than a login for exactly that reason: hard-coding - # the login would lock out the busiest reviewer, and moving the token to a - # GitHub App later makes this clause the whole fix on its own. - # * Not one of our own notes. This is what actually breaks the loop, against - # a marker the reporting step prepends to every note it posts, so it holds - # however the wording is later reflowed. startsWith, not contains, and the - # marker goes first for that reason: GitHub's "Quote reply" prefixes "> ", - # so a human who quotes a refusal and issues the command underneath is - # still heard. Skipping a verbatim note of our own costs nothing — there is - # nobody to report to, and 👎-ing our own comment is precisely the noise the - # not-a-command exception exists to avoid. - if: >- - contains(github.event.comment.body, '/fr-pass') - && github.event.comment.user.type != 'Bot' - && !startsWith(github.event.comment.body, '') - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - # Authenticates as the tracebloc-release-train App (backend#2036) instead of - # one human's PAT. Needs BOTH grants: `organization_projects: write` to advance - # the card and `issues: write` to post the outcome comment and its reaction. - # `owner:` makes the token ORG-scoped -- a repo-scoped one cannot reach an org - # ProjectV2. No fallback to the old PAT: a fallback would let a broken App path - # look like a working migration. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # LEAST PRIVILEGE, DERIVED FROM THIS JOB'S OWN CALLS (backend#2157). - # Two steps share this token and between them make six calls; this is the - # union, with the permission each one documents: - # - # GET repos/{r}/collaborators/{u}/permission Metadata read (implicit) - # GET repos/{r}/pulls/{n} PRs read - # organization().projectV2 + item Status read projects read - # updateProjectV2ItemFieldValue projects WRITE - # POST issues/comments/{id}/reactions Issues WRITE <- only option - # POST issues/{n}/comments Issues write (or PRs write) - # - # `issues: write` is what forces a write grant here, and it is not - # substitutable: GitHub documents the issue-comment REACTION endpoint as - # Issues write ALONE, while the comment endpoint accepts either. Both - # endpoints serve PR threads too -- a PR's conversation IS an issue -- so - # `pull-requests` stays at read, needed only for the promotion-PR ref - # lookup in step 3a. - # - # `Metadata` is granted implicitly alongside any repository permission, - # which is why the collaborator-permission read needs no row of its own. - # Note what happens if that call fails anyway: the step warns and falls - # back to `author_association`, a LOOSER authorisation test. So a scope - # error here degrades security rather than reddening -- if the warning - # "Could not read repository permission" starts appearing in these runs, - # treat it as this change's regression, not as noise. - # - # Nothing reads or writes repository content, so contents drops. - permission-issues: write - permission-pull-requests: read - permission-organization-projects: write - - - name: Decide the outcome and advance the card - id: assess - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - ORG: ${{ inputs.org }} - PROJECT_NUMBER: ${{ inputs.project-number }} - ALLOW_SELF_SIGNOFF: ${{ inputs.allow-self-signoff }} - REPO_FULL: ${{ github.repository }} - NUMBER: ${{ github.event.issue.number }} - IS_PR: ${{ github.event.issue.pull_request != null }} - ITEM_AUTHOR: ${{ github.event.issue.user.login }} - ACTOR: ${{ github.event.comment.user.login }} - ASSOCIATION: ${{ github.event.comment.author_association }} - # Via the environment, never interpolated into the script: a comment - # body is attacker-controlled text and must not reach the shell parser. - COMMENT_BODY: ${{ github.event.comment.body }} - RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - BOARD_URL: ${{ github.server_url }}/orgs/${{ inputs.org }}/projects/${{ inputs.project-number }} - run: | - set -euo pipefail - - RESULT_FILE="$RUNNER_TEMP/fr-pass.result" - NOTE_FILE="$RUNNER_TEMP/fr-pass.note.md" - : > "$RESULT_FILE" - : > "$NOTE_FILE" - - # A literal ``` fence, kept in a variable so no backtick ever has to be - # escaped inside a command substitution below. - FENCE='```' - - # Publish an outcome on EVERY exit, including an unplanned one. The - # reporting step keys off this one output, so a crash halfway through - # (API error, jq error, a `set -e` trip) still produces a 👎 and an - # explanation rather than the silence reported in backend#1319. - publish() { - if [ ! -s "$RESULT_FILE" ]; then - printf 'internal-error' > "$RESULT_FILE" - printf '%s\n' \ - "⚠️ \`/fr-pass\` failed before it could decide anything, so the card was **not** moved." \ - "" \ - "This is a bug in the handler, not something you did. Run log: $RUN_URL" > "$NOTE_FILE" - fi - echo "result=$(cat "$RESULT_FILE")" >> "$GITHUB_OUTPUT" - } - trap publish EXIT - - # decide [note-line...] — record the outcome, plus the comment - # to post with it. No note lines = react only, no thread comment. - decide() { - printf '%s' "$1" > "$RESULT_FILE" - shift - if [ "$#" -gt 0 ]; then - printf '%s\n' "$@" > "$NOTE_FILE" - fi - } - - # ------------------------------------------------ 1. is it a command? - # /fr-pass must be the first token on some line of the comment. Leading - # whitespace and trailing text are fine; a quoted reply ("> /fr-pass") - # or an inline mention is not the command and must not re-trigger it. - BODY_FILE="$RUNNER_TEMP/fr-pass.body" - printf '%s' "$COMMENT_BODY" | tr -d '\r' > "$BODY_FILE" - if ! grep -qiE '^[[:space:]]*/fr-pass([[:space:]].*)?$' "$BODY_FILE"; then - echo "Body mentions /fr-pass but does not issue it as a command — staying silent by design." - decide not-a-command - exit 0 - fi - - # --------------------------------------------- 2. may they sign off? - # author_association is not a permission check: a member whose org - # membership is private reads as CONTRIBUTOR (or NONE) on repos they - # were not added to individually, so the old MEMBER/OWNER test would - # silently reject a real reviewer. Ask for the actual repository - # permission, which also states D30's intent — "not a triage/read - # outside collaborator" — directly instead of approximating it. - PERM_ERR="$RUNNER_TEMP/fr-pass.perm.err" - PERM_JSON=$(gh api "repos/$REPO_FULL/collaborators/$ACTOR/permission" 2> "$PERM_ERR") || PERM_JSON="" - PERM=$(printf '%s' "$PERM_JSON" | jq -r '.permission // empty' 2> /dev/null || true) - ROLE=$(printf '%s' "$PERM_JSON" | jq -r '.role_name // empty' 2> /dev/null || true) - - AUTHORISED=false - case "$PERM" in admin | maintain | write) AUTHORISED=true ;; esac - case "$ROLE" in admin | maintain | write) AUTHORISED=true ;; esac - - if [ -z "$PERM_JSON" ]; then - # API unreachable (scope, rate limit, outage). Fall back to the - # association rather than reject a legitimate reviewer — and say so. - echo "::warning::Could not read repository permission for @$ACTOR — falling back to author_association=$ASSOCIATION" - sed -n '1,3p' "$PERM_ERR" - case "$ASSOCIATION" in MEMBER | OWNER | COLLABORATOR) AUTHORISED=true ;; esac - fi - - if [ "$AUTHORISED" != "true" ]; then - decide not-authorised \ - "🚫 \`/fr-pass\` was **not** applied: @$ACTOR does not have write access to \`$REPO_FULL\`." \ - "" \ - "Functional sign-off is limited to people who can write to the repository (seen: permission \`${PERM:-unknown}\`, association \`$ASSOCIATION\`). Ask someone on the team to sign off, or move the card by hand on the [engineering kanban]($BOARD_URL)." - exit 1 - fi - - # -------------------------- 3a. is this a release-train promotion? - # Promotion PRs (`release-train/*`, and hotfix back-merges) are - # plumbing: the FR gate on them evaluates the CONTAINED items, so the - # promotion PR itself has nothing for a second reviewer to assess and - # D30 would only block the train on its own author. The comment event - # carries no branch refs, so look them up; FAIL CLOSED — an - # unreadable PR is treated as normal work and keeps the D30 rule. - IS_PROMOTION=false - if [ "$IS_PR" = "true" ]; then - if REFS=$(gh api "repos/$REPO_FULL/pulls/$NUMBER" \ - --jq '[.head.ref, .base.ref] | @tsv' 2>/dev/null); then - HEAD_REF=$(printf '%s' "$REFS" | cut -f1) - BASE_REF=$(printf '%s' "$REFS" | cut -f2) - case "$HEAD_REF" in - release-train/* | hotfix-backmerge/*) - # Belt and braces: those heads only ever target an - # integration branch, so require that too. - case "$BASE_REF" in - staging | main | master) IS_PROMOTION=true ;; - esac - ;; - esac - else - echo "Could not read the PR refs for #$NUMBER; treating it as normal work (D30 stays in force)." - fi - fi - - # ----------------------------------------- 3. self sign-off (D30) - # THE bug from backend#1319 lived here, as a clause in the job-level - # `if`: when the reviewer was also the PR author the whole job was - # skipped, so no step was left to react or explain. 8 of the 13 - # /fr-pass comments on 2026-07-29 died exactly this way. The rule is - # unchanged; what changed is that it now says so out loud. - if [ "$ACTOR" = "$ITEM_AUTHOR" ] \ - && [ "$ALLOW_SELF_SIGNOFF" != "true" ] \ - && [ "$IS_PROMOTION" != "true" ]; then - decide self-signoff \ - "👀 \`/fr-pass\` was **not** applied: functional review needs a second pair of eyes, and @$ACTOR opened this one (D30)." \ - "" \ - "Ask another team member to comment \`/fr-pass\`, or move the card to **Ready for prod** on the [engineering kanban]($BOARD_URL) if you are deliberately overriding the rule." \ - "" \ - "_Release-train promotion PRs are exempt from this rule automatically. To allow self sign-off on everything org-wide, set \`allow-self-signoff: true\` on \`fr-pass-comment.yml\`._" - exit 1 - fi - - # ----------------------------------------- 4. resolve the board item - REPO_NAME="${REPO_FULL#*/}" - - # One query, two roots: project metadata + this item's Status. - # $itemNum names the issue/PR number so it stays distinct from $num. - if [ "$IS_PR" = "true" ]; then - # shellcheck disable=SC2016 # $itemNum is a GraphQL variable - must not expand in shell - CONTENT_QUERY='pullRequest(number: $itemNum)' - else - # shellcheck disable=SC2016 # $itemNum is a GraphQL variable - must not expand in shell - CONTENT_QUERY='issue(number: $itemNum)' - fi - - GQL_ERR="$RUNNER_TEMP/fr-pass.gql.err" - # No `2>/dev/null || PROJ='{}'` here. Swallowing the error made an - # expired token, a rate limit and a genuinely unlisted card all produce - # the same "not on project" 👎 — undiagnosable from the thread. - if ! PROJ=$(gh api graphql -f query=" - query(\$org: String!, \$num: Int!, \$repo: String!, \$itemNum: Int!) { - organization(login: \$org) { - projectV2(number: \$num) { - id - field(name: \"Status\") { - ... on ProjectV2SingleSelectField { id options { id name } } - } - } - } - repository(owner: \$org, name: \$repo) { - $CONTENT_QUERY { - projectItems(first: 20) { - totalCount - nodes { - id - project { number } - fieldValueByName(name: \"Status\") { - ... on ProjectV2ItemFieldSingleSelectValue { name } - } - } - } - } - } - }" -F org="$ORG" -F num="$PROJECT_NUMBER" -F repo="$REPO_NAME" -F itemNum="$NUMBER" 2> "$GQL_ERR"); then - GQL_HEAD=$(sed -n '1,5p' "$GQL_ERR" | tr -d "$FENCE") - decide api-error \ - "⚠️ \`/fr-pass\` could **not** be applied: the kanban API call failed, so the card was left alone." \ - "" \ - "$FENCE" \ - "$GQL_HEAD" \ - "$FENCE" \ - "" \ - "Usually an expired \`PROJECTS_KANBAN_TOKEN\` or a transient GitHub error. Re-comment \`/fr-pass\` to retry. Run log: $RUN_URL" - exit 1 - fi - - GQL_ERRORS=$(printf '%s' "$PROJ" | jq -r '(.errors // []) | map(.message) | join("; ")') - PROJECT_ID=$(printf '%s' "$PROJ" | jq -r '.data.organization.projectV2.id // empty') - STATUS_FIELD=$(printf '%s' "$PROJ" | jq -r '.data.organization.projectV2.field.id // empty') - - if [ -n "$GQL_ERRORS" ] || [ -z "$PROJECT_ID" ] || [ -z "$STATUS_FIELD" ]; then - decide api-error \ - "⚠️ \`/fr-pass\` could **not** be applied: project #$PROJECT_NUMBER or its \`Status\` field did not resolve, so the card was left alone." \ - "" \ - "GraphQL said: \`${GQL_ERRORS:-no error message}\`" \ - "" \ - "Run log: $RUN_URL" - exit 1 - fi - - # Take whichever content root the query returned, then keep only the - # items that belong to OUR project. - NODES=$(printf '%s' "$PROJ" | jq -c --arg n "$PROJECT_NUMBER" ' - [ ((.data.repository // {}) | (.issue // .pullRequest // {}) | .projectItems.nodes // [])[] - | select(.project.number == ($n | tonumber)) ]') - TOTAL_ITEMS=$(printf '%s' "$PROJ" | jq -r ' - ((.data.repository // {}) | (.issue // .pullRequest // {}) | .projectItems.totalCount) // 0') - ITEM_COUNT=$(printf '%s' "$NODES" | jq -r 'length') - - if [ "$ITEM_COUNT" = "0" ]; then - # A truncated page is not proof of absence: say which case this is. - if [ "$TOTAL_ITEMS" -gt 20 ]; then - decide api-error \ - "⚠️ \`/fr-pass\` could **not** be applied: this item is on $TOTAL_ITEMS projects and the handler reads only the first 20, so project #$PROJECT_NUMBER may have been cut off." \ - "" \ - "Raise the \`projectItems(first: 20)\` page size in \`fr-pass-comment.yml\`. Run log: $RUN_URL" - exit 1 - fi - decide not-on-project \ - "🔍 \`/fr-pass\` did nothing: this item is not on the [engineering kanban]($BOARD_URL) (project #$PROJECT_NUMBER), so there is no card to advance." \ - "" \ - "\`add-to-kanban.yml\` normally adds every new issue and PR. If it was missed, add the card, set **FR on staging**, then comment \`/fr-pass\` again. If the tracked work lives on a linked issue rather than this PR, sign off there instead." - exit 1 - fi - - if [ "$ITEM_COUNT" != "1" ]; then - # A single piece of content can only sit on a project once, so this - # means the board data is odd. Do not guess which card was meant. - decide ambiguous-item \ - "⚠️ \`/fr-pass\` did nothing: this item resolves to $ITEM_COUNT separate cards on project #$PROJECT_NUMBER, so the handler will not guess which one to advance." \ - "" \ - "Clean up the duplicate on the [engineering kanban]($BOARD_URL) and comment \`/fr-pass\` again." - exit 1 - fi - - ITEM_ID=$(printf '%s' "$NODES" | jq -r '.[0].id') - CURRENT=$(printf '%s' "$NODES" | jq -r '.[0].fieldValueByName.name // ""') - - # The two column names below are hard-coded, and this board has been - # renamed before (RFC-BACKEND-0008 collapsed the dev-side FR columns). - # Check they still exist, so the next rename fails loudly here instead - # of reporting "wrong column" for every card on the board. - # Rename window (backend#1592): the staging-review column is "Staging - # (human review)" after the UI rename and "FR on staging" before it, and - # the rename is a single instant with no overlap. So resolve it to - # whichever exists and keep the loud failure for the case where NEITHER - # does -- which is the check this block was written for. - if printf '%s' "$PROJ" | jq -e '.data.organization.projectV2.field.options[] - | select(.name == "Staging (human review)")' > /dev/null; then - STAGING_REVIEW="Staging (human review)" - else - STAGING_REVIEW="FR on staging" - fi - for WANT in "$STAGING_REVIEW" "Ready for prod"; do - if ! printf '%s' "$PROJ" \ - | jq -e --arg s "$WANT" '.data.organization.projectV2.field.options[] | select(.name == $s)' > /dev/null; then - decide missing-option \ - "⚠️ \`/fr-pass\` could **not** be applied: project #$PROJECT_NUMBER has no \`Status\` option named **$WANT**, so the card was left alone." \ - "" \ - "The board's columns were probably renamed. Update the column names in \`.github/.github/workflows/fr-pass-comment.yml\`. Run log: $RUN_URL" - exit 1 - fi - done - - # ------------------------------------------------------ 5. the column - case "$CURRENT" in - "Staging (human review)"|"FR on staging") - NEXT="Ready for prod" - ;; - "Ready for prod" | "Prod") - decide already-advanced \ - "✅ Nothing to do: this card is already in **$CURRENT**, past the staging functional review." \ - "" \ - "No change made — you can ignore this." - # Stays 0: the gate is already satisfied, so the caller got the - # state they asked for. Nothing is left for a human to do, which is - # the only thing a red run is here to say. - exit 0 - ;; - "") - decide no-status \ - "🔍 \`/fr-pass\` did nothing: this card is on the board but its \`Status\` is empty, so there is no column to advance from." \ - "" \ - "Set it to **FR on staging** on the [engineering kanban]($BOARD_URL) and comment \`/fr-pass\` again." - exit 1 - ;; - *) - decide wrong-column \ - "🔍 \`/fr-pass\` did nothing: this card is in **$CURRENT**, and \`/fr-pass\` only advances **FR on staging → Ready for prod** (the one functional-review gate)." \ - "" \ - "Promote it to **FR on staging** first — that happens automatically when the release train pushes to \`staging\`." - exit 1 - ;; - esac - - NEXT_OPT=$(printf '%s' "$PROJ" | jq -r --arg s "$NEXT" \ - '.data.organization.projectV2.field.options[] | select(.name == $s) | .id') - - # Pass the option ID with -f (raw string), NOT -F: ProjectV2 option IDs can - # be all-numeric, and -F coerces all-digit values to an integer, which the - # $o: String! variable rejects. -f forces a string. (Some option IDs contain - # letters today, but don't rely on that — IDs regenerate if recreated.) - MUT_ERR="$RUNNER_TEMP/fr-pass.mutate.err" - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - if ! gh api graphql -f query=' - mutation($p: ID!, $i: ID!, $f: ID!, $o: String!) { - updateProjectV2ItemFieldValue(input: { - projectId: $p, itemId: $i, fieldId: $f, - value: {singleSelectOptionId: $o} - }) { projectV2Item { id } } - }' -F p="$PROJECT_ID" -F i="$ITEM_ID" -F f="$STATUS_FIELD" -f o="$NEXT_OPT" > /dev/null 2> "$MUT_ERR"; then - MUT_HEAD=$(sed -n '1,5p' "$MUT_ERR" | tr -d "$FENCE") - decide mutation-failed \ - "⚠️ \`/fr-pass\` could **not** be applied: the card is still in **$CURRENT** — the write to project #$PROJECT_NUMBER failed." \ - "" \ - "$FENCE" \ - "$MUT_HEAD" \ - "$FENCE" \ - "" \ - "Re-comment \`/fr-pass\` to retry. Run log: $RUN_URL" - exit 1 - fi - - echo "→ #$NUMBER: $CURRENT → $NEXT" - decide advanced - { - echo "from=$CURRENT" - echo "to=$NEXT" - } >> "$GITHUB_OUTPUT" - - - name: Report the outcome (👍 / 👎 + why) - # always(), with no `steps.*` condition: if the step above died before it - # could publish anything, THAT is the case most in need of reporting. - if: always() - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - REPO_FULL: ${{ github.repository }} - NUMBER: ${{ github.event.issue.number }} - COMMENT_ID: ${{ github.event.comment.id }} - RESULT: ${{ steps.assess.outputs.result }} - FROM: ${{ steps.assess.outputs.from }} - TO: ${{ steps.assess.outputs.to }} - RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} - run: | - set -uo pipefail - - NOTE_FILE="$RUNNER_TEMP/fr-pass.note.md" - OUTCOME="${RESULT:-}" - if [ -z "$OUTCOME" ]; then - # The step above produced no output at all — it was cancelled, or it - # died before its EXIT trap could run. Still report something. - OUTCOME="internal-error" - if [ ! -s "$NOTE_FILE" ]; then - printf '%s\n' \ - "⚠️ The \`/fr-pass\` handler stopped before deciding anything, so the card was **not** moved." \ - "" \ - "Run log: $RUN_URL" > "$NOTE_FILE" - fi - fi - - # The one intentionally silent outcome: the comment mentioned /fr-pass - # but never issued it. Reacting to prose would make the bot noise. - if [ "$OUTCOME" = "not-a-command" ]; then - echo "Not a /fr-pass command — no reaction, by design." - echo "### /fr-pass: not a command (no action taken)" >> "$GITHUB_STEP_SUMMARY" - exit 0 - fi - - if [ "$OUTCOME" = "advanced" ]; then - REACTION="+1" - SUMMARY="advanced #$NUMBER: $FROM → $TO" - else - REACTION="-1" - SUMMARY="no-op on #$NUMBER ($OUTCOME)" - fi - - FAILED="" - - # Reaction. The old version used GITHUB_TOKEN and hid the outcome behind - # `|| true`, so every attempt died on HTTP 403 in silence — even the runs - # that DID advance the card. PROJECTS_KANBAN_TOKEN is the only token here - # that can write at all (see the permissions note at the top of the - # file), so there is no second token to fall back to — a failure is - # reported and fails the run instead of being swallowed. - REACT_ERR="$RUNNER_TEMP/fr-pass.react.err" - if ! gh api -X POST "/repos/$REPO_FULL/issues/comments/$COMMENT_ID/reactions" \ - -f content="$REACTION" > /dev/null 2> "$REACT_ERR"; then - echo "::error::Could not react on comment $COMMENT_ID." - sed -n '1,6p' "$REACT_ERR" - FAILED="reaction" - fi - - # Explanation. Only outcomes that are a real problem carry a note; a - # clean advance stays a bare 👍 so the happy path keeps the thread quiet. - if [ -s "$NOTE_FILE" ]; then - # Prepend the marker the job `if` tests for, so this handler can - # recognise its own notes and not answer them (backend#1413). Added - # here, at the single place a note is posted, rather than in each - # `decide` call above — a note that forgets it would resurrect the - # loop. First line, because the job `if` uses startsWith. HTML - # comments are stripped when GitHub renders markdown but kept in the - # stored body the event carries, so the thread reads as it did before. - BODY=$(printf '%s\n\n%s\n' '' "$(cat "$NOTE_FILE")") - COMMENT_ERR="$RUNNER_TEMP/fr-pass.comment.err" - # REST, not `gh pr/issue comment`: a PR's conversation IS an issue, so - # this one endpoint covers both without branching on the item type. - if ! gh api -X POST "/repos/$REPO_FULL/issues/$NUMBER/comments" \ - -f body="$BODY" > /dev/null 2> "$COMMENT_ERR"; then - echo "::error::Could not post the explanation on #$NUMBER." - sed -n '1,6p' "$COMMENT_ERR" - FAILED="$FAILED comment" - fi - fi - - { - echo "### /fr-pass: $SUMMARY" - echo "" - echo "- outcome: \`$OUTCOME\`" - echo "- reaction: $REACTION on comment $COMMENT_ID" - } >> "$GITHUB_STEP_SUMMARY" - - # A write that failed must not leave a green run behind it — a red run - # is the last remaining signal once the thread could not be reached. - if [ -n "$FAILED" ]; then - echo "::error::/fr-pass could not report its outcome ($FAILED). Outcome was: $OUTCOME" - exit 1 - fi - - echo "/fr-pass → $OUTCOME (reacted $REACTION)" diff --git a/.github/workflows/git-reap-selftest.yml b/.github/workflows/git-reap-selftest.yml deleted file mode 100644 index 64757db..0000000 --- a/.github/workflows/git-reap-selftest.yml +++ /dev/null @@ -1,46 +0,0 @@ -name: git-reap selftest - -# git-reap deletes branches, and its whole safety argument is one sentence in -# its own header: "I could not tell" is never treated as "it merged". Two of the -# three defects found in review were the code contradicting that exact sentence -# -- a failed `gh pr list` and a silently truncated one both read as "no merged -# PRs", so a branch nobody had verified was reported as having no merge -# evidence. A claim like that belongs in a machine check, not a comment. -# -# Offline: the suite builds throwaway git repos and stubs `gh` on PATH, so it -# needs no token and reaches no network. Path-filtered like the other selftests -# -- it only has to run when the thing it tests changes. - -on: - pull_request: - paths: - - scripts/git-reap - - scripts/tests/git-reap-selftest.sh - - .github/workflows/git-reap-selftest.yml - push: - branches: [main, develop, staging] - paths: - - scripts/git-reap - - scripts/tests/git-reap-selftest.sh - - .github/workflows/git-reap-selftest.yml - -permissions: - contents: read - -concurrency: - group: git-reap-selftest-${{ github.ref }} - cancel-in-progress: true - -jobs: - selftest: - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # The suite creates real branches and real merges; a committer identity is - # not configured on a fresh runner. - - run: | - git config --global user.email "selftest@tracebloc.io" - git config --global user.name "git-reap selftest" - git config --global init.defaultBranch develop - - run: bash scripts/tests/git-reap-selftest.sh diff --git a/.github/workflows/kanban-archive.yml b/.github/workflows/kanban-archive.yml deleted file mode 100644 index b65f5df..0000000 --- a/.github/workflows/kanban-archive.yml +++ /dev/null @@ -1,881 +0,0 @@ -name: Kanban archive - -# Continuous board hygiene (RFC-BACKEND-0008 D3): archive every item that has -# reached a terminal column (Prod / Cancelled) so the board shows only live work. -# -# Archived items are RETAINED and searchable - this hides done work, it does not -# delete it. Archiving is what keeps the board fast: at the current merge rate -# ~800 items/month land in Prod, so terminal work has to leave the board on entry, -# not accumulate. -# -# Archives by COLUMN MEMBERSHIP, never by age. Age is unusable on this board (the -# retired auto-classify rewrote updatedAt every 10 min), and a just-shipped item -# should archive promptly, not after N idle days. Runs daily, just after the -# reconcile pass (which moves any drifted items into Prod first). - -on: - schedule: - - cron: '0 5 * * *' # 05:00 UTC daily (after kanban-reconcile at 04:00) - workflow_dispatch: - inputs: - dry-run: - description: "Log what would be archived without archiving" - type: boolean - default: false - -permissions: - contents: read - # Lets the completeness check below download the PREVIOUS run's recorded board - # size. That number is the only one in this job that THIS run's credential did - # not produce, which is what makes it able to answer a question no - # single-read check can (backend#2802). - actions: read - -concurrency: - group: kanban-archive - cancel-in-progress: false - -jobs: - archive: - runs-on: ubuntu-latest - env: - ORG: tracebloc - PROJECT_NUMBER: 2 - DRY_RUN: ${{ github.event.inputs.dry-run || 'false' }} - steps: - # Board writes authenticate as the tracebloc-release-train App (backend#2036), - # not as a human's PAT. `owner:` makes it an ORG-scoped installation token -- - # a repo-scoped one cannot touch an org ProjectV2. No fallback to the old PAT: - # a fallback would make a broken App path look like a working migration. - # - # GH_TOKEN moved from job-level env to per-step because a job-level env cannot - # read a step output. That is an improvement, not a workaround -- it now says - # exactly which steps hold a credential. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # LEAST PRIVILEGE, DERIVED FROM WHAT THIS RUNS (backend#2157) -- and the - # WRITE half is still exactly that: `archiveProjectV2Item` is the only - # mutation, so `organization-projects: write` remains the whole write - # requirement. No contents, no metadata, nothing else. - # - # THE READ HALF WAS WRONG, AND SILENTLY (backend#2623). Projects v2 filters - # `items.nodes` by whether the credential can read each item's CONTENT, so a - # token that cannot read issues and pull requests does not get a permission - # error -- it gets a SHORTER LIST. The archiver reported success over a - # board it could barely see. Isolated by controlled A/B: - # - # pinned `repositories:` + no content reads ....... 102 items - # `repositories:` removed ......................... 155 items (+53) - # + issues:read and pull-requests:read ........... 1626 items (+1471) - # - # So BOTH changes are load-bearing: the org-wide scope lets it see items - # whose content lives in other repos, and the content reads stop the API - # filtering those items back out. Dropping either one re-narrows it. - # - # WHY NOT THE PAT INSTEAD. `PROJECTS_KANBAN_TOKEN` also widens the read, but - # only to ~742 items -- fewer than this -- and it is a PERSON's credential - # (the shape backend#2087 is open about). App + read scopes sees more and - # stays a machine identity, so it is better on both axes. - # - # READ-ONLY, AND THAT IS THE POINT: `issues` and `pull-requests` are `read`, - # never `write`. This widens what the archiver can SEE, not what it can - # touch. A credential that cannot see the board cannot archive it, and a - # guard that silently sees 7% of its input is the class backend#1729 named. - permission-organization-projects: write - permission-issues: read - permission-pull-requests: read - - - name: Resolve project id - id: ids - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - # shellcheck disable=SC2016 # $names are GraphQL variables, not shell - keep literal - PROJECT_ID=$(gh api graphql -f query=' - query($org: String!, $num: Int!) { - organization(login: $org) { projectV2(number: $num) { id } } - }' -F org="$ORG" -F num="$PROJECT_NUMBER" \ - --jq '.data.organization.projectV2.id') - echo "project_id=$PROJECT_ID" >> "$GITHUB_OUTPUT" - - - name: Collect un-archived terminal items (Prod / Cancelled) - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - : > terminal.ids - : > seen.tsv - cursor="null" - while :; do - if [ "$cursor" = "null" ]; then - # shellcheck disable=SC2016 # $names are GraphQL variables, not shell - keep literal - OUT=$(gh api graphql -f query=' - query($org: String!, $num: Int!) { - organization(login: $org) { projectV2(number: $num) { - items(first: 100) { - totalCount - pageInfo { hasNextPage endCursor } - nodes { id isArchived - fieldValueByName(name: "Status") { - ... on ProjectV2ItemFieldSingleSelectValue { name } } } - } } } }' -F org="$ORG" -F num="$PROJECT_NUMBER") - else - # shellcheck disable=SC2016 # $names are GraphQL variables, not shell - keep literal - OUT=$(gh api graphql -f query=' - query($org: String!, $num: Int!, $c: String!) { - organization(login: $org) { projectV2(number: $num) { - items(first: 100, after: $c) { - totalCount - pageInfo { hasNextPage endCursor } - nodes { id isArchived - fieldValueByName(name: "Status") { - ... on ProjectV2ItemFieldSingleSelectValue { name } } } - } } } }' -F org="$ORG" -F num="$PROJECT_NUMBER" -F c="$cursor") - fi - echo "$OUT" | jq -r '.data.organization.projectV2.items.nodes[] - | select(.isArchived == false) - | select(.fieldValueByName.name == "Prod" or .fieldValueByName.name == "Cancelled" or .fieldValueByName.name == "Done") - | .id' >> terminal.ids - # WHAT THIS READ ACTUALLY SAW, recorded per item (backend#2623). The counts - # below are the only thing that can tell "the board is clean" apart from - # "this credential cannot see the board", and for weeks nothing printed - # them. A null Status means the item was returned but its field value was - # not readable -- which silently drops it from the filter above. - echo "$OUT" | jq -r '.data.organization.projectV2.items.nodes[] - | [(.isArchived|tostring), (.fieldValueByName.name // "NULL_STATUS")] - | @tsv' >> seen.tsv - HAS_NEXT=$(echo "$OUT" | jq -r '.data.organization.projectV2.items.pageInfo.hasNextPage') - cursor=$(echo "$OUT" | jq -r '.data.organization.projectV2.items.pageInfo.endCursor') - [ "$HAS_NEXT" = "true" ] || break - done - total=$(wc -l < seen.tsv | tr -d " ") - nullst=$(awk -F"\t" '$2=="NULL_STATUS"' seen.tsv | wc -l | tr -d " ") - live=$(awk -F"\t" '$1=="false"' seen.tsv | wc -l | tr -d " ") - echo "Items this credential could read: $total (un-archived: $live, unreadable Status: $nullst)" - echo "Un-archived terminal items: $(wc -l < terminal.ids)" - { - echo "## What the archive credential saw" - echo "" - echo "| measure | count |" - echo "|---|---|" - echo "| items read | $total |" - echo "| un-archived | $live |" - echo "| Status unreadable (null) | $nullst |" - echo "| terminal + un-archived | $(wc -l < terminal.ids | tr -d " ") |" - } >> "$GITHUB_STEP_SUMMARY" - # A NULL Status IS A FINDING, not a shrug. The item exists, the filter cannot - # judge it, and the run would otherwise report success having skipped it. - if [ "$nullst" -gt 0 ]; then - echo "::warning::$nullst item(s) returned an unreadable Status for this credential. They cannot be judged terminal or not, so they are neither archived nor reported -- this is the silent-skip path backend#2623 exists to close." - fi - echo "$total" > seen.total - - - name: Archive them - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - PROJECT_ID="${{ steps.ids.outputs.project_id }}" - n=$(wc -l < terminal.ids | tr -d ' ') - if [ "$n" = "0" ]; then echo "Nothing to archive."; fi - if [ "$DRY_RUN" = "true" ]; then - echo "DRY RUN - would archive $n terminal item(s)." - exit 0 - fi - ok=0; fail=0 - while IFS= read -r itemId; do - [ -z "$itemId" ] && continue - # `2>&1 >/dev/null` keeps stderr (the API error) and drops the mutation - # payload, which we never read. The reverse order would discard the one - # thing needed to diagnose a failure. Same form as advance-deploy-env.yml. - # shellcheck disable=SC2016 # $names are GraphQL variables, not shell - keep literal - if err=$(gh api graphql -f query=' - mutation($p: ID!, $i: ID!) { - archiveProjectV2Item(input: {projectId: $p, itemId: $i}) { item { id } } - }' -F p="$PROJECT_ID" -F i="$itemId" 2>&1 >/dev/null); then - ok=$((ok+1)) - else - echo "::warning::failed to archive item $itemId: $err" - fail=$((fail+1)) - fi - done < terminal.ids - echo "=== Archived: $ok ok, $fail failed ===" - # Handed to the completeness check as the ONLY legitimate reason the - # board may be smaller than it was yesterday. Written before the - # fail-closed exit below so it exists whatever the archive did. - echo "$ok" > archived.count - { - echo "## Kanban archive" - echo "" - echo "Archived $ok terminal item(s), $fail failed." - } >> "$GITHUB_STEP_SUMMARY" - - # Fail closed. A failed archive leaves terminal work on the board, which - # is the exact condition this workflow exists to prevent, and nothing - # retries it beyond tomorrow's run hitting the same error. Exiting 0 here - # meant the board could degrade for weeks behind an unbroken green history - # - at ~800 items/month reaching Prod, slowly enough to look like normal - # growth. The summary is written above first so it survives this exit. - if [ "$fail" -ne 0 ]; then - echo "::error::$fail of $((ok+fail)) terminal item(s) failed to archive (see warnings above)." - exit 1 - fi - - - # THE ONE NUMBER THIS RUN DID NOT PRODUCE ITSELF (backend#2802). - # - # Every completeness check below this line compares two figures that come - # out of the SAME credential's SAME view of the board -- `reread_total` - # against `declared_total`, and the re-read against the first read. Both - # pairs shrink together, so all of them agree while the view itself is - # partial. Measured 2026-08-28: two runs two minutes apart reported - # "paginated 93, totalCount=93 (agree)" and "paginated 30, totalCount=30 - # (agree)" with ONE item archived between them, while a PAT saw 742. Each - # run was internally consistent and both were wrong. A board cannot lose - # 63 cards to a single archive, but nothing in a single run could say so, - # because the disagreement is BETWEEN runs and every number was inside one. - # - # So carry yesterday's count forward as an artifact and compare across the - # boundary. Cards leave this board exactly one way -- being archived -- so - # `previous - archived_this_run` is a hard floor on today's size, and a - # total below it is a shrunken view, not a smaller board. - - name: Recall the previous run's board size - # `!cancelled()`, NOT the implicit `success()` (Bugbot High, #383). A - # non-zero `fail` exits `Archive them` -- and that exit is deliberate, - # because a Job left un-archived is what this workflow exists to prevent. - # But the run has usually archived `ok` cards BEFORE the one that failed, - # so the board really is smaller. Skipping this step and the assert below - # meant those archives never reached `board.total`, and the next run's - # `previous - archived_now` sat above the real board: a red that clears - # only if the next run happens to archive the difference by accident. - # - # The upload step already carried `!cancelled()` for exactly this reason. - # Leaving the two steps that PRODUCE board.total on `success()` armed one - # half of the mechanism: the upload survived the failure and found - # nothing to upload. - if: ${{ !cancelled() && env.DRY_RUN != 'true' }} - env: - # The DEFAULT token, not the App token: this reads run artifacts in - # this repo, which is what `actions: read` above grants it. The App - # token is scoped for the project board and is the very credential - # under suspicion here -- reading the baseline with it would put both - # sides of the comparison back inside one view. - GH_TOKEN: ${{ github.token }} - # `gh run download` TAKES NO --repo AND THIS JOB HAS NO CHECKOUT, so it - # resolved the repository from the git remote, found no `.git`, and died - # before making any API call (backend#2802). That is why the artifacts - # LISTING above worked -- it puts the repo in the URL path -- while the - # fetch failed, and the run went red having archived successfully. - # - # Reproduced outside Actions, gh 2.98.0: - # $ cd /tmp/empty && gh run download -n board-baseline - # failed to run git: fatal: not a git repository - # $ GH_REPO=tracebloc/.github gh run download -n board-baseline - # $ cat prev/board.total -> 574 - # - # `gh` does NOT read `GITHUB_REPOSITORY` (tested), so being on a runner - # does not supply it. GH_REPO is the variable it does read, and it is - # cheaper and narrower than adding a checkout to a job that needs no - # source. - GH_REPO: ${{ github.repository }} - run: | - set -uo pipefail - : > prev.total - : > prev.error - # RETRY A TRANSIENT READ, THEN FAIL CLOSED (backend#3068). The reads - # in this step -- the artifact listing, the baseline download and the - # reconcile probe -- are cross-run / API reads subject to eventual - # consistency (an artifact is not queryable the instant it is uploaded), - # brief 5xx / secondary-rate-limit blips, and cross-run availability - # races. A SINGLE attempt failed ~60% of this cron's runs, and every one - # of those failures refuses the whole cross-run comparison below -- so a - # momentary blip became a red run that hid a green archive (backend#3068). - # - # `retry_read` runs its argument command; on a non-zero exit it waits and - # retries up to BASELINE_READ_RETRIES times, then returns the LAST - # attempt's status. Exhausting the retries therefore fails closed exactly - # as a single attempt did -- an unreadable baseline must NEVER read as - # "nothing to archive". It streams the wrapped command's own combined - # output on stdout, so `arts=$(retry_read gh api ...)` is unchanged, while - # its own progress notes go to stderr. Attempts and delay are - # env-overridable so the selftest can drive the loop without sleeping; - # the production defaults are 4 attempts, 3s apart. - retry_read() { - local attempt=1 max="${BASELINE_READ_RETRIES:-4}" delay="${BASELINE_READ_DELAY:-3}" - local out rc - while :; do - out=$("$@" 2>&1) - rc=$? - if [ "$rc" -eq 0 ]; then - printf '%s' "$out" - return 0 - fi - if [ "$attempt" -ge "$max" ]; then - printf '%s' "$out" - echo "::warning::a board-baseline read failed after $attempt attempt(s); failing closed." >&2 - return "$rc" - fi - echo "::warning::a board-baseline read failed (attempt $attempt/$max); retrying in ${delay}s." >&2 - sleep "$delay" - attempt=$((attempt + 1)) - done - } - # ASK WHETHER A BASELINE EXISTS BEFORE ASKING FOR ITS CONTENT, because - # those are the two answers this step must never conflate (Bugbot, - # #383). The first draft walked `gh run list ... || true` and let an - # empty result mean "no baseline" -- so an API failure, an expired - # token or a rate limit rendered as the benign first-run warning and - # the job still passed. That is the ticket's own defect reproduced one - # layer inside its fix: a check reporting clean having compared - # nothing, because it could not tell "absent" from "unreadable". - # - # The artifacts endpoint answers it unambiguously: it FAILS on a - # broken lookup and returns an empty list when there is genuinely - # nothing yet. It also carries `workflow_run.id`, so the run listing - # is not needed at all -- one call, and no second way to be wrong. - # PAGINATE, do not refuse on truncation (reviewer, .github#393). An earlier - # cut of this fix fetched one `per_page=100` page and refused when - # `total_count > returned` -- but `total_count` is a filtered, monotonically - # GROWING count that GitHub never brings back down, and `board-baseline` - # gains a record a day, so it crosses 100 in ~98 days and the refusal then - # fires on EVERY run forever. That is the red-nobody-can-clear this file's - # own comments cite three times, reproduced in the fix for it. The live - # baseline being on page 1 (newest-first) would make the refusal spurious - # even so -- and leaning on that ordering is itself fragile, because the - # artifacts API does not document it. - # - # `--paginate` follows every page, so the listing is read WHOLE regardless - # of how many records accumulate or what order they come in: no truncation - # to detect, and the newest un-expired artifact is found wherever it sits. - # `--jq '.artifacts[]'` streams the artifact objects across all pages, so - # `$arts` is newline-delimited JSON that the `jq -s` reads below slurp back - # into one array. A non-JSON body makes `--jq` fail and `gh` exit non-zero, - # which the branch below catches and refuses -- fail-closed, no fall-through. - if ! arts=$(retry_read gh api --paginate "repos/$GITHUB_REPOSITORY/actions/artifacts?name=board-baseline&per_page=100" --jq '.artifacts[]'); then - # selftest:unreadable-path - printf 'the artifact listing failed: %s\n' "$(printf '%s' "$arts" | tail -1)" > prev.error - echo "::warning::could not list the board-baseline artifacts; the assert step will refuse." - else - live=$(printf '%s' "$arts" | jq -s '[.[] | select(.expired == false)] | length') - if [ "$live" = "0" ]; then - echo "No unexpired board-baseline artifact exists yet (first run, or retention lapsed)." - else - rid=$(printf '%s' "$arts" \ - | jq -sr '[.[] | select(.expired == false)] | sort_by(.created_at) | last | .workflow_run.id') - # KEEP THE DOWNLOAD'S STDERR. `>/dev/null 2>&1` is the reason this - # failure was diagnosable only by reproducing it by hand: the step - # reported "could not be downloaded" and threw away the one line - # that said why ("failed to run git: fatal: not a git repository"). - # Stdout still goes to /dev/null -- it is progress noise -- but the - # error text is captured and echoed by the refusal below. - # STDERR TO A SCRATCH FILE, PROMOTED TO prev.error ONLY ON FAILURE - # (Bugbot, High -- on my own fix). Writing straight into prev.error - # couples the two: the assert step at the bottom treats ANY non-empty - # prev.error as an unreadable baseline (`if [ -s prev.error ]`), so a - # SUCCESSFUL download that printed anything at all would mark the - # baseline unreadable and keep the job red after a good archive -- - # the exact failure this PR fixes, arriving from the other side. - # - # MEASURED, and stated honestly: `gh run download` writes 0 bytes to - # stderr on success on a non-TTY (tested, gh 2.98.0), so the finding - # as described does not reproduce today. The COUPLING is real - # regardless, and one deprecation notice on stderr in a future gh is - # all it would take. Decoupled unconditionally because it costs - # nothing: only the failure branch can now populate prev.error. - # RETRIED INLINE (backend#3068). The download is a compound read -- - # fetch, then require a non-empty board.total -- with its stderr routed - # to prev.stderr for the refusal branch below, so it does not fit the - # pure-command `retry_read` wrapper. It uses the same BASELINE_READ_* - # budget. Exhausting the retries falls through to the fail-closed - # refusal, NEVER to a clean first-run read: an unreadable baseline is - # not an absent one. - dl_attempt=1 - dl_ok=0 - while :; do - if gh run download "$rid" -n board-baseline -D prev >/dev/null 2>prev.stderr && [ -s prev/board.total ]; then - dl_ok=1 - break - fi - if [ "$dl_attempt" -ge "${BASELINE_READ_RETRIES:-4}" ]; then - break - fi - echo "::warning::the board-baseline download failed (attempt $dl_attempt); retrying." >&2 - sleep "${BASELINE_READ_DELAY:-3}" - dl_attempt=$((dl_attempt + 1)) - done - if [ "$dl_ok" = 1 ]; then - cp prev/board.total prev.total - echo "Baseline: run $rid saw $(cat prev.total) item(s)." - # IS THIS JOB THE ONLY ARCHIVER SINCE? It is not, and assuming - # so was wrong (Bugbot High, .github#383). `kanban-reconcile.yml` - # archives too -- `archiveProjectV2Item` at its own apply step -- - # every Monday at 04:00, an hour before this job. Cards it - # archived are gone from the board and counted by no `ok` of - # ours, so a floor of `previous - archived_now` sits above the - # real board every Monday: a red produced by the check being - # wrong, which is the fastest way to get a tier switched off. - # - # DERIVED, not estimated: ask when reconcile last succeeded and - # compare it against the baseline's own timestamp. If it ran in - # between, the arithmetic cannot be exact and this run says so - # instead of refusing. Every other day the floor is exact. - cut=$(printf '%s' "$arts" \ - | jq -sr '[.[] | select(.expired == false)] | sort_by(.created_at) | last | .created_at') - # `completed`, NOT `success` (Bugbot, #383). Reconcile archives - # inside a counted-then-fail-closed apply loop, so a run that - # archived cards and THEN failed did archive them -- and filtering - # to successes made exactly those runs invisible here. The floor - # then read their cards as a shrunken view and `view_bad` withheld - # the baseline, which is the same red-that-cannot-clear this whole - # block exists to avoid. What matters is whether reconcile RAN - # since the baseline, not whether it finished happy. - if ! rec=$(retry_read gh api "repos/$GITHUB_REPOSITORY/actions/workflows/kanban-reconcile.yml/runs?status=completed&per_page=1"); then - # selftest:unreadable-path - printf 'could not tell whether kanban-reconcile archived since the baseline: %s\n' "$(printf '%s' "$rec" | tail -1)" > prev.error - else - last_rec=$(printf '%s' "$rec" | jq -r '.workflow_runs[0].updated_at // ""') - if [ -n "$last_rec" ] && [ "$last_rec" \> "$cut" ]; then - printf '%s\n' "$last_rec" > prev.otherarchiver - fi - fi - else - # The listing SAW a live artifact and the fetch still failed, so - # this is a broken read, not an empty history. Refusing is the - # whole point of having asked the two questions separately. - # selftest:unreadable-path - # APPEND, DO NOT CLOBBER. `gh run download`'s stderr is already in - # prev.error at this point, and it holds the only line that says - # WHY -- overwriting it with this summary is what made the last - # five red runs undiagnosable from their own logs. The summary - # goes first because it names the run; the tool's own words follow. - reason=$(tr '\n' ' ' < prev.stderr 2>/dev/null | cut -c1-300) - printf 'a live board-baseline artifact exists (run %s) but could not be downloaded. gh said: %s\n' \ - "$rid" "${reason:-}" > prev.error - echo "::warning::the baseline exists but could not be read; the assert step will refuse." - fi - fi - fi - - # A SEPARATE STEP, AND THAT IS THE FIX (Bugbot High, #339). This block used - # to sit at the bottom of "Archive them", BELOW `if [ "$n" = "0" ]; then - # ... exit 0; fi`. So the one case it exists to catch -- a first read that - # finds nothing terminal because the credential cannot see it -- left the - # step before asserting anything, and the job went green having checked - # nothing. A guard reachable only when the bug is absent is not a guard. - # As its own step it runs whatever the archive did or did not do. - # NOT IN DRY RUN. A dry run archives nothing, so terminal cards are still - # there by construction and this would fail every time -- turning the - # preview into a permanent red that teaches people to ignore it. The - # assertion is about whether a REAL run left the board clean. - - name: Assert the board is clean - # `!cancelled()` for the reason the recall step above gives: this is where - # `board.total` is written, so gating it on the archive succeeding makes a - # partial archive unrecordable and its red permanent. Running here after a - # failed archive is also the more honest verdict -- it re-reads the board - # and says what is actually left, rather than declining to look. - if: ${{ !cancelled() && env.DRY_RUN != 'true' }} - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - # ASSERT THE BOARD IS CLEAN, not that the archive did what it saw - # (backend#2623). Every guard above answers "did my writes succeed?" and - # every one of them can answer yes while the board is untouched -- which is - # what happened: on 2026-08-26 this workflow archived 63 of 63 items, - # reported success, and left 668 un-archived terminal cards (475 Prod, 153 - # Done, 40 Cancelled) that a human PAT could see and this credential could - # not. Weeks of unbroken green, and the header comment above had already - # predicted the shape ("the board could degrade for weeks behind an unbroken - # green history"). - # - # So re-read AFTER archiving and fail on anything terminal that is still - # live. This is deliberately a SECOND read rather than arithmetic on the - # first: the defect being caught is that the first read is incomplete, so - # comparing it against itself would agree and prove nothing -- the - # test-a-list-against-itself trap. - # - # It fails CLOSED on an unreadable re-read too: zero parsed items is not - # evidence of a clean board, so an empty result with a non-zero total is a - # finding, not a pass. - left=0; reread_total=0; nullst_after=0; declared_total=""; cursor="null" - # Counted so the identity below need not ASSUME whether this connection - # returns archived items -- see the comment there. - arch_seen=0 - while :; do - if [ "$cursor" = "null" ]; then - # shellcheck disable=SC2016 # $names are GraphQL variables, not shell - keep literal - OUT=$(gh api graphql -f query=' - query($org: String!, $num: Int!) { - organization(login: $org) { projectV2(number: $num) { - items(first: 100) { - totalCount - pageInfo { hasNextPage endCursor } - nodes { id isArchived - fieldValueByName(name: "Status") { - ... on ProjectV2ItemFieldSingleSelectValue { name } } } - } } } }' -F org="$ORG" -F num="$PROJECT_NUMBER") || OUT="" - else - # shellcheck disable=SC2016 # $names are GraphQL variables, not shell - keep literal - OUT=$(gh api graphql -f query=' - query($org: String!, $num: Int!, $c: String!) { - organization(login: $org) { projectV2(number: $num) { - items(first: 100, after: $c) { - totalCount - pageInfo { hasNextPage endCursor } - nodes { id isArchived - fieldValueByName(name: "Status") { - ... on ProjectV2ItemFieldSingleSelectValue { name } } } - } } } }' -F org="$ORG" -F num="$PROJECT_NUMBER" -F c="$cursor") || OUT="" - fi - if [ -z "$OUT" ]; then - echo "::error::the post-archive re-read failed, so this run cannot show the board is clean. Unverified is not verified-clean." - exit 1 - fi - # FAIL CLOSED ON AN UNREADABLE CONNECTION. `nodes|length` renders a - # missing or null path as 0, so a query that returned no items at all - # would read as "nothing left to archive" and pass (Bugbot HIGH, - # backend#2623). Check the connection resolved before counting it. - if [ "$(echo "$OUT" | jq -r 'try (.data.organization.projectV2.items.nodes|type) catch "null"')" != "array" ]; then - echo "::error::the post-archive re-read returned no items array, so this run cannot show the board is clean. Unverified is not verified-clean." - exit 1 - fi - # THE SERVER'S OWN COUNT, which is the only number in this job that does - # not come from the same credential's view of `nodes`. Comparing two - # same-credential reads catches a read that DEGRADED, never one that was - # incomplete in both -- the blindness this whole step exists to close. - page_tc=$(echo "$OUT" | jq -r '.data.organization.projectV2.items.totalCount // "null"') - case "$page_tc" in - ''|null|*[!0-9]*) - echo "::error::the post-archive re-read did not return totalCount, so the completeness of this read cannot be established." - exit 1 ;; - esac - [ -z "$declared_total" ] && declared_total="$page_tc" - n=$(echo "$OUT" | jq -r '.data.organization.projectV2.items.nodes|length') - reread_total=$((reread_total + n)) - arch_seen=$((arch_seen + $(echo "$OUT" | jq -r '[.data.organization.projectV2.items.nodes[] - | select(.isArchived == true)] | length'))) - left=$((left + $(echo "$OUT" | jq -r '[.data.organization.projectV2.items.nodes[] - | select(.isArchived == false) - | select(.fieldValueByName.name == "Prod" or .fieldValueByName.name == "Cancelled" or .fieldValueByName.name == "Done")] - | length'))) - nullst_after=$((nullst_after + $(echo "$OUT" | jq -r '[.data.organization.projectV2.items.nodes[] - | select(.isArchived == false) - | select(.fieldValueByName == null or .fieldValueByName.name == null)] - | length'))) - HAS_NEXT=$(echo "$OUT" | jq -r '.data.organization.projectV2.items.pageInfo.hasNextPage') - cursor=$(echo "$OUT" | jq -r '.data.organization.projectV2.items.pageInfo.endCursor') - [ "$HAS_NEXT" = "true" ] || break - done - echo "Post-archive re-read: $reread_total item(s), $left still terminal + un-archived." - { - echo "" - echo "Post-archive re-read: **$left** terminal item(s) still un-archived (of $reread_total read)." - } >> "$GITHUB_STEP_SUMMARY" - # NOTE: this no longer exits here. It is the primary assertion, but exiting - # on it skipped the completeness numbers below, and those are what say - # whether a non-zero `left` means "archiving missed some" or "this - # credential cannot see the board at all". Recorded into `fail` and raised - # with everything else at the end of the step. - left_bad=0 - if [ "$left" -ne 0 ]; then - echo "::error::$left terminal item(s) remain un-archived after this run. The archive succeeded on what it could see, which is NOT the same as the board being clean -- see backend#2623." - left_bad=1 - fi - # THE RE-READ IS THE SAME CREDENTIAL, SO IT SHARES THE BLIND SPOT - # (Bugbot High, #339). The comment above says a second read avoids - # comparing the first read against itself. That is true of the ITEM - # LIST and false of what actually made the incident invisible: the - # items WERE returned; their `Status` came back null for this - # credential, so they matched no terminal filter and `left` stayed 0. - # A second read with the same token reproduces that precisely, and - # then agrees with itself -- the trap the comment claims to dodge. - # - # Two things the terminal count alone cannot say are therefore findings: - # - # 1. A null Status on an UN-ARCHIVED item. It may be terminal and - # this credential cannot tell. "Cannot tell" is not "clean". The - # first read only ::warning::-ed on exactly this, which is the - # silent skip backend#2623 exists to close, one layer further in. - # 2. A re-read that saw FEWER items than the first read. Both reads - # are this credential's, so this cannot see what neither can; it - # does catch a read that degraded mid-run, which would otherwise - # lower `left` and read as the board having improved. - # EVALUATE EVERY COMPLETENESS FINDING BEFORE EXITING, and exit once at the - # end. These three answer DIFFERENT questions and the most diagnostic one - # was ordered last, so the first to fire hid it: on 2026-08-27 this step - # exited on a single null Status while `totalCount` -- the only number here - # that does not come from this credential's own view of `nodes` -- was never - # compared. That comparison is what separates "this credential sees a small - # board" from "items are being omitted from a large one", which is precisely - # the question backend#2623 needs answered, and one short-circuit meant a - # whole run produced no answer to it. A guard that masks a stronger guard - # costs a run per diagnosis. - fail=$left_bad - # SEPARATE FROM `fail`, and the distinction is the whole of the second - # Bugbot High (.github#383). `fail` asks "is the board clean?"; - # `view_bad` asks "can this run's own count be trusted?", and only the - # second decides whether the baseline advances. Tying the record to - # `fail` made the red SELF-SUSTAINING: a run that archived 50 cards and - # then failed on a leftover terminal one skipped the record, so the next - # run met a floor 50 too high, refused, skipped the record in turn, and - # nothing could ever get back under it. Rule 4 -- a red nobody can clear - # trains people to skip the tier. A leftover card or an unreadable - # Status makes the JOB fail without making the COUNT wrong. - view_bad=0 - if [ "$nullst_after" -gt 0 ]; then - echo "::error::$nullst_after un-archived item(s) returned an unreadable Status on the post-archive re-read. They cannot be judged terminal, so this run CANNOT show the board is clean -- unverified is not verified-clean (backend#2623)." - fail=1 - fi - # selftest:xrun-begin - first_total=$(cat seen.total) - # THE BARE SHRINK CHECK IS GONE, and it was RED ON EVERY PRODUCTIVE RUN - # (backend#2820, found in FR on staging). It compared the re-read - # against the PRE-archive count while both reads filter - # `isArchived == false` -- so archiving N shrinks the second read by - # exactly N and the guard fired on precisely the runs that did their - # job. Measured on develop after #380 restored the credential's sight: - # 749 read, 253 archived, 496 re-read, and it failed with "a reason - # that is not archiving" about arithmetic it never performed. - # - # Its own error text named the subtraction it was missing. The identity - # below IS that subtraction, so this is a strict replacement rather - # than a deletion: every shrink the old check could catch is a - # `reread_total != first_total - archived` too, and the identity also - # catches a shrink the old one could not see (one exactly cancelled by - # an archive count) and a GROWTH, which it ignored entirely. - # - # Leaving both was this PR's own bug: the description claimed the - # identity "replaces the weaker bound" while the code kept both, so - # #383 would have shipped the red-on-success the ticket describes. - # Rule 4 -- never land a red gate; a daily cron that fails whenever it - # works is one nobody reads, which is how the original blindness - # survived in the first place. - # `if`, not `[ -s f ] && v=$(cat f)`: the short-circuit leaves the - # absent-file case with a non-zero `$?`, which is harmless HERE and - # becomes a failed step the moment anything is appended after it or - # this block moves to the end of the step. Measured, so the comment - # does not overstate it: `set -e` exempts every command of an AND-OR - # list but the last, so the short-circuit does NOT exit mid-step - # today. The `if` is what makes that independent of position. - archived_now=0 - if [ -s archived.count ]; then archived_now=$(cat archived.count); fi - prev_total="" - if [ -s prev.total ]; then prev_total=$(cat prev.total); fi - # THE EXACT IDENTITY, and it needs no baseline and no other workflow. - # Two reads bracket this job's own archiving, so the re-read's size is - # determined -- not bounded, determined. Nothing else archives inside - # the seconds between them (reconcile runs an hour earlier), so any - # other number is a view that moved under this run. This is what - # catches the measured 93 -> 30 with no cross-run state at all. - # - # WHETHER ARCHIVING SHRINKS THE CONNECTION IS DERIVED, NOT ASSUMED - # (Bugbot High, .github#383). The obvious form, `first_total - - # archived_now`, is right only if `ProjectV2.items` omits archived - # items -- and if that ever stopped being true the identity would fire - # on EVERY productive run, a permanent red produced by the check being - # wrong. Measured on this board, it does omit them: run 33084151778 - # read `1626 (un-archived: 1626)` -- a full read with zero archived - # items present, weeks after 83+ had been archived, and nothing calls - # `deleteProjectV2Item` anywhere in this repo. - # - # But the reads themselves already carry the answer, so the arithmetic - # asks instead of trusting that measurement. `arch_seen - arch_first` - # is how many items became archived BETWEEN the two reads: - # - connection omits archived -> both are 0, expected = first - ok - # - connection keeps archived -> the difference is `ok`, expected = first - # Exact either way, and it stops depending on a fact about someone - # else's API that no test here can pin. - arch_first=$(awk -F"\t" '$1=="true"' seen.tsv | wc -l | tr -d " ") - expected=$((first_total - archived_now + arch_seen - arch_first)) - # DIRECTIONAL, NOT EXACT (backend#2833). Growth between the two reads - # is routine rather than exceptional: `add-to-kanban` puts a card on - # the board the moment anyone opens an issue or a PR, so one can - # arrive in the seconds between the reads, and `reread_total > - # expected` then says nothing at all about this credential's sight. - # Exact equality therefore failed runs for doing their job -- and it - # set `view_bad`, so no baseline was recorded, the next run's floor - # stayed at the pre-growth total, and the red could not clear. That is - # rule 4 twice over: a red on success, and a red with no way out. - # - # Shrinkage is the entire signal. Archiving is the only way a card - # leaves this board and this job knows exactly how many it archived, - # so anything BELOW `expected` is a view that moved rather than a - # board that changed. This deliberately cannot see a shrink exactly - # cancelled by an addition; the cross-run floor below is what catches - # a persistent one, and a permanent red catches nothing whatsoever. - # `expected` IS PRINTED, and the suite asserts the printed number - # against one written down per case (backend#2833). A lower bound - # cannot see an `expected` computed too LOW -- the shortfall just - # reads as growth -- so the moment the comparison stopped being exact, - # every term of this arithmetic became unfalsifiable through the - # verdict alone. Measured: dropping `+ arch_seen - arch_first` (the - # Bugbot High of .github#383, that archiving-shrinks-the-connection - # must not be assumed) went from caught to UNCAUGHT on exactly that - # change. Emitting the value is what puts it back under test. - echo "Within-run: archiving accounts for $expected item(s) ($first_total read - $archived_now archived + $arch_seen archived-now - $arch_first archived-then); the re-read returned $reread_total." - if [ "$reread_total" -lt "$expected" ]; then - echo "::error::this run read $first_total item(s), archived $archived_now, and then read only $reread_total -- at least $((expected - reread_total)) item(s) below the $expected that archiving accounts for (archived items in the two reads: $arch_first then $arch_seen). The view shrank under this run, so its counts describe no single board (backend#2802)." - fail=1 - view_bad=1 - elif [ "$reread_total" -gt "$expected" ]; then - echo "Within-run: re-read $reread_total against the $expected archiving accounts for -- $((reread_total - expected)) item(s) added between the two reads. Growth is not a shrunken view." - fi - # THE SERVER'S OWN COUNT, and NOT an equality (backend#2831). - # `totalCount` and `nodes` are not two views of one set immediately - # after a bulk archive. Measured on run 33301924089: the first read - # returned 518, this job archived 49, the re-read paginated exactly - # 469 -- a COMPLETE read, since 518 - 49 = 469 -- and the server still - # reported totalCount=475. `totalCount` had caught up with 43 of the - # 49 archives, not all of them, so the equality failed the run - # BECAUSE it archived. - # - # #383 asserted the two agree on the strength of a run that archived - # NOTHING (33084151778, "1626 (un-archived: 1626)"). That shows the - # returned nodes were un-archived; it says nothing about what - # `totalCount` counts, and conflating those two was the error. - # - # The omission this exists to catch survives, because the lag has a - # CEILING: `totalCount` can only still be counting cards this job just - # archived, so the gap cannot exceed `archived_now`. Wider than that, - # or on a run that archived nothing at all, no lag can explain it and - # items really are being counted without being returned. - # - # It is moved BELOW the identity, into the tested region, for a reason - # the old position made impossible: `archived_now` is read here, so - # above it the ceiling is not merely unknown but `set -u`-fatal. - if [ "$reread_total" -gt "$declared_total" ]; then - echo "Completeness: paginated $reread_total item(s), server totalCount=$declared_total -- $((reread_total - declared_total)) more returned than counted, so nothing is being omitted from the read." - elif [ "$reread_total" -eq "$declared_total" ]; then - echo "Completeness: paginated $reread_total item(s), server totalCount=$declared_total (agree)." - elif [ "$((declared_total - reread_total))" -le "$archived_now" ]; then - echo "::warning::the re-read paginated $reread_total item(s) while the server reports totalCount=$declared_total. The gap of $((declared_total - reread_total)) is within the $archived_now card(s) this job archived, so totalCount has not finished catching up and no omission is implied (backend#2831)." - else - echo "::error::the post-archive re-read paginated $reread_total item(s) but the server reports totalCount=$declared_total -- a gap of $((declared_total - reread_total)), wider than the $archived_now this job archived, so archive lag cannot account for it. Items are being omitted from the read, and a terminal card can exist that this run never judged (backend#2623)." - fail=1 - view_bad=1 - fi - # ACROSS THE RUN BOUNDARY, which is the only axis the three checks - # above cannot see (backend#2802). Archiving is the sole way a card - # leaves this board, so yesterday's total minus what this run archived - # THIS RUN'S BOARD SIZE: THE SMALLER OF THE TWO COUNTS IT HOLDS. - # - # ONE NUMBER, used by the floor comparison below and by the baseline - # record at the foot of this step, because using different ones is how - # the lag path went wrong (backend#2833). `declared_total` is a server - # counter and `reread_total` is what this credential actually walked; - # they disagree in both directions and for different reasons. - # - # THE SMALLER, DELIBERATELY, and the asymmetry is the argument: a - # baseline that is too HIGH is unclearable -- tomorrow's floor sits - # above the real board, the check fires, `view_bad` blocks the record, - # and every run after it meets the same wall. A baseline that is too - # LOW merely under-detects for one cycle and then self-corrects. The - # self-sustaining red is the expensive failure, so the arithmetic is - # biased away from it. - # - # On the LAG path this is what fixes the bug: `totalCount` is still - # counting cards archived seconds ago, so it is the larger and is - # discarded. On the "more returned than counted" path `totalCount` is - # the smaller and is kept, which leaves that case exactly as it was. - board_size="$declared_total" - if [ "$reread_total" -lt "$board_size" ]; then board_size="$reread_total"; fi - - # is a FLOOR on today's. Below it, the board did not shrink -- this - # credential's view of it did. - # UNREADABLE FIRST, and it is a REFUSAL rather than the warning below. - # The two look identical from here -- both leave `prev_total` empty -- - # and collapsing them is what this ticket is about. An empty history is - # a bounded cannot-tell with no remedy, so it warns; a failed read is - # actionable, and a cron job's warnings are exactly what went unread - # for weeks while this defect ran. - if [ -s prev.error ]; then - echo "::error::the previous board size could not be read ($(cat prev.error)), so this run made NO cross-run comparison. An unread baseline is not an absent one -- refusing rather than reporting the first-run warning (backend#2802)." - fail=1 - elif [ -s prev.otherarchiver ]; then - # NOT A REFUSAL, because the arithmetic genuinely cannot be exact - # here: kanban-reconcile archived after this baseline was taken and - # published no count, so a drop is expected and unquantified. Saying - # "cannot tell" is the honest answer; refusing would be a red the - # calendar produces, and passing silently would be the defect this - # whole check exists to remove. - echo "::warning::kanban-reconcile succeeded at $(cat prev.otherarchiver), after this baseline was recorded. It archives too, so the floor cannot be exact and NO cross-run comparison was made this run (backend#2802)." - elif [ -n "$prev_total" ]; then - floor=$((prev_total - archived_now)) - if [ "$board_size" -lt "$floor" ]; then - echo "::error::this run saw $board_size item(s), but the previous run saw $prev_total and this run archived only $archived_now -- so at least $((floor - board_size)) item(s) stopped being visible for a reason that is not archiving. This is a shrunken view, not a smaller board (backend#2802)." - fail=1 - view_bad=1 - else - echo "Cross-run: board size $board_size against a floor of $floor (previous $prev_total - $archived_now archived)." - fi - else - # NOT A PASS, and said out loud. With no baseline the cross-run - # comparison did not run, so this is the one shape it exists to - # catch going unchecked -- rule 3, "cannot tell" is a finding. It is - # a warning rather than a failure only because a first run, and the - # run after any retention lapse, legitimately has nothing to compare - # against; a red there would be a red nobody could ever clear. - echo "::warning::no previous board size was available, so this run could NOT rule out a shrunken view (backend#2802). Its own counts agreeing proves only that they came from the same read." - fi - # THE BASELINE IS WHAT THIS RUN PAGINATED (backend#2833). - # - # It used to record `declared_total`, and on the LAG PATH that is a - # number this job had just warned about: `totalCount` still counting - # the cards we archived seconds ago. `view_bad` stays 0 there -- - # correctly, the count is not WRONG, it is behind -- so the inflated - # figure was written as tomorrow's baseline. Tomorrow's floor is then - # inflated minus tomorrow's archives; when `totalCount` catches up the - # floor check fires, sets `view_bad`, and NO corrected baseline is - # written. Every following run meets the same too-high floor. - # - # That is the self-sustaining red the comment above `view_bad` says was - # fixed, arriving through the other door: not from tying the record to - # `fail`, but from recording a figure known to be temporarily too big. - # - # `reread_total` is what this credential actually walked, post-archive. - # It cannot lag, because it IS the read. - if [ "$view_bad" -eq 0 ]; then - echo "$board_size" > board.total - else - echo "::warning::not recording a baseline from this run: its own read was incoherent, and enshrining it would ratchet the floor down to meet the defect." - fi - # selftest:xrun-end - { - echo "" - echo "| completeness check | value |" - echo "|---|---|" - echo "| paginated (re-read) | $reread_total |" - echo "| server totalCount | $declared_total |" - echo "| first read | $first_total |" - echo "| un-archived null Status | $nullst_after |" - echo "| previous run totalCount | ${prev_total:-none} |" - echo "| archived this run | $archived_now |" - } >> "$GITHUB_STEP_SUMMARY" - [ "$fail" -eq 0 ] || exit 1 - - # ONLY ON A PASS -- no `if: always()`. A run whose view shrank must not - # write its own shrunken total forward as tomorrow's baseline, or the - # floor ratchets down to meet the defect and the check quietly stops - # being able to fire. Failing leaves the last TRUSTED count in place, so - # the comparison keeps pointing at the real board until it is fixed. - - name: Record this run's board size for the next one - # `!cancelled()`, not the assert step's success: the decision to record - # moved INTO that step, where `view_bad` can separate "the board is not - # clean" from "this run's count cannot be trusted". Gating the upload on - # the job instead made the red self-sustaining -- see the `view_bad` - # comment above. `if-no-files-found: ignore` because a refusing run - # deliberately writes no board.total, and that is the mechanism, not a - # fault. `always()` is avoided so a cancelled run records nothing. - if: ${{ !cancelled() && env.DRY_RUN != 'true' }} - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 - with: - name: board-baseline - path: board.total - if-no-files-found: ignore - retention-days: 30 diff --git a/.github/workflows/kanban-closure-caller.yml b/.github/workflows/kanban-closure-caller.yml deleted file mode 100644 index 0b54df5..0000000 --- a/.github/workflows/kanban-closure-caller.yml +++ /dev/null @@ -1,12 +0,0 @@ -name: Kanban closure routing - -on: - pull_request: - types: [closed] - issues: - types: [closed] - -jobs: - route: - uses: tracebloc/.github/.github/workflows/kanban-closure-router.yml@main - secrets: inherit diff --git a/.github/workflows/kanban-closure-router.yml b/.github/workflows/kanban-closure-router.yml deleted file mode 100644 index e1b0447..0000000 --- a/.github/workflows/kanban-closure-router.yml +++ /dev/null @@ -1,525 +0,0 @@ -name: Route kanban Status on closure - -# Reusable workflow. Called on PR closed + issue closed events. -# Sets the correct Status based on what actually happened: -# - PR merged to develop → On dev (automatic; no dev-side review — D6) -# - PR merged to staging → FR on staging (functional review on staging environment) -# - PR merged to main/master → Prod (shipped to prod) -# - PR closed without merging → Cancelled -# - Issue closed as completed → Done (terminal; how it was closed is irrelevant) -# (was: mirror the closing PR's Status, which put finished issues in deploy -# columns — see the branch below and backend#2722) -# - Issue closed as not_planned → Cancelled -# - Issue closed (no state_reason) → Cancelled (default to abandoned) -# -# NOTE: For PR merges, advance-deploy-env.yml also fires on the resulting branch -# push and sets the same Status. Both workflows are idempotent and converge on -# the same value; this one fires faster (PR close event) and serves as the -# primary signal, while advance-deploy-env covers the case of pushes that -# weren't a PR merge (e.g. fast-forward of develop → staging). - -on: - workflow_call: - inputs: - project-number: - type: number - default: 2 - org: - type: string - default: tracebloc - -jobs: - route: - runs-on: ubuntu-latest - steps: - # The mapping comes from .github, not from a copy in this file: one - # definition of branch -> Status (backend#2243). - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - repository: tracebloc/.github - ref: main - path: .kanban-map - persist-credentials: false - - # Authenticates as the tracebloc-release-train App (backend#2036) instead of - # one human's PAT. Needs both grants: `organization_projects: write` for the - # card, `issues: write` for the sibling-merge label and the closure comment. - # `owner:` makes the token ORG-scoped -- a repo-scoped one cannot reach an org - # ProjectV2, and the closer lookup below is cross-repo besides. - # - # No fallback to the old PAT: a fallback would let a broken App path look like - # a working migration, which is the defect class backend#1680 exists to remove. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # LEAST PRIVILEGE, DERIVED FROM THIS JOB'S OWN CALLS (backend#2157). - # Four steps share this token; this is the union of what they call: - # - # organization().projectV2 fields/options projects read - # repository.pullRequest|issue(n).projectItems PRs+issues read - # updateProjectV2ItemFieldValue projects WRITE - # gh issue comment (the D8 parked-card note) Issues WRITE - # GET repos/{r}/labels/{name} (does it exist?) Issues read - # POST repos/{r}/labels (create it) Issues WRITE - # POST repos/{r}/issues/{n}/labels Issues write - # - # The last one lands on a PULL REQUEST in the sibling-merge case, and it - # still only needs `issues: write`: GitHub documents the label endpoints - # as "at least one of" Issues write / Pull requests write, and every - # shared action on a PR -- labels, assignees, milestones -- is served by - # the Issues endpoints. So `pull-requests` stays at READ, which is all the - # `projectItems` lookup on a PR needs. - # - # `contents: READ` IS REQUIRED, and the previous revision of this comment - # got it wrong in a way worth recording. It proved contents:WRITE could - # drop and then concluded nothing reads content either -- in the same - # sentence that names the read. `branch_status_map.py` fetches the - # `.kanban.yml` override over the API (`FETCHED, NOT READ OFF DISK, - # because the router never checks the repo out`) and is passed - # `$REPO_FULL` -- the CALLER's repo, not this one -- with - # `GH_TOKEN: steps.app-token.outputs.token`. `persist-credentials: false` - # applies to the `.github` checkout, which is not the repo being read. - # - # AND IT WOULD NOT HAVE FAILED LOUDLY. The mapper's own contract treats - # 404 as "no override" but REFUSES on any other fetch failure, and the - # call sites below wrap it in `if ! STATUS=$(...)` which falls to the - # holding state by design. So a 403 parks EVERY merged PR at - # `override-unusable` with a green job -- silent mis-routing fleet-wide. - # Found by Bugbot and confirmed by saadqbal on .github#324. - # - # contents:write still drops, as do administration/actions/checks read. - # - # `repositories:` STAYS UNNARROWED -- the cross-repo callers are the - # projectItems reads and the issue comment/label writes listed above (a - # client or website PR routinely closes a `backend` issue), and - # `organization-projects` is an org-level grant regardless. The closer - # lookup used to be cited here too; it was removed with the routing change - # (backend#2722) and is no longer a reason for anything. - # - # NOT PROVEN BY READING. An under-scoped token fails at the call site, and - # this workflow's failure mode is a card the built-in "Item closed" - # automation then sets to `Cancelled` (.github#157). The next closed PR or - # issue in any repo is the real test; read the failing call before - # widening this list. - permission-contents: read - permission-issues: write - permission-pull-requests: read - permission-organization-projects: write - - - name: Determine target Status - id: target - env: - EVENT_NAME: ${{ github.event_name }} - PR_MERGED: ${{ github.event.pull_request.merged }} - BASE_REF: ${{ github.event.pull_request.base.ref }} - ISSUE_REASON: ${{ github.event.issue.state_reason }} - ISSUE_NUMBER: ${{ github.event.issue.number }} - # ORG + REPO_FULL are read by the project lookups, the .kanban.yml - # override mapper and the D8 parked-card comment. Keep them exported: - # without them `set -u` aborts the script before any status= output is - # written, leaving the built-in "Item closed" project workflow to set - # Status=Cancelled. (They were originally added for the closing-PR-base - # lookup, which backend#2722 removed; the other consumers remain.) - ORG: ${{ inputs.org }} - REPO_FULL: ${{ github.repository }} - # The PAT rather than the caller's GITHUB_TOKEN: this step's project reads - # and the override mapper are cross-repo. (This comment used to explain a - # closer lookup that ran tokenless and therefore never worked -- .github#126. - # backend#2722 removed the lookup entirely, so that history now describes no - # code and is dropped rather than left to mislead.) - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - - # STATUS="" means "make no change": the update step below is guarded on - # a non-empty status and skips entirely. Every branch here either sets - # a real column or deliberately leaves STATUS empty. - STATUS="" - - SIBLING="false" - # SET WHEN THE OVERRIDE COULD NOT BE USED (saadqbal on .github#295). Drives - # the same visibility mechanism as SIBLING: a card the weekly pass must - # disposition, marked so a human can see why. - # - # UNUSABLE, NOT UNREADABLE (backend#2324). Two causes reach this holding - # state now -- a `.kanban.yml` that cannot be fetched or parsed, and one - # that reads fine but names a Status the mapping does not declare -- and - # the second sends an operator hunting a read failure that never happened. - # The mapper says which on stderr; this flag only says "we could not tell". - UNUSABLE_OVERRIDE="false" - if [ "$EVENT_NAME" = "pull_request" ]; then - if [ "$PR_MERGED" = "true" ]; then - # ONE SHARED MAPPING (backend#2243) -- and this site is why the fix - # is a shared script rather than a second `yq` read: this workflow - # never checks the caller out, so the per-repo `.kanban.yml` override - # it was silently ignoring has to be FETCHED, not read off disk. - # A REFUSED OVERRIDE MUST NOT COST THE WRITE (Bugbot, .github#295). - # `read_override` refuses on an unreadable `.kanban.yml`, and a bare - # command substitution under `set -euo pipefail` then exits this step - # before any Status is published -- so the update step's non-empty - # guard skips and the project's built-in "Item closed" automation sets - # `Cancelled`, archiving shipped-via-parent work (.github#157). That is - # the no-write path this file already documents, reached by a new - # door -- and the door opens on any repo whose override cannot be - # read, which is not a hypothetical population. - # - # `resolve` REFUSES HERE TOO NOW (backend#2324), on an override naming - # a Status the mapping does not declare. That case used to succeed and hand - # this arm a name the update step below cannot resolve to an option id - # -- so it aborted mid-write and reached the same "Item closed" branch - # anyway, having skipped the holding state entirely. One exit code, one - # policy: whatever the mapper could not answer, the card gets parked. - # - # STDERR IS KEPT. It was discarded, which cost nothing while the only - # refusal was "could not fetch" -- the flag said that much. The - # unknown-Status refusal names the branch, the bad value and the - # accepted vocabulary, and that message IS the fix for the operator. - if ! STATUS=$(python3 .kanban-map/scripts/branch_status_map.py \ - "$BASE_REF" "$REPO_FULL" "$BASE_REF" \ - | jq -r '.status'); then - # THE HOLDING STATE, NOT THE DEFAULT MAPPING (saadqbal on - # .github#295, correcting my own first fix). Writing the default - # would be "silently apply the mapping we could not confirm" -- it - # claims a promotion happened on a read that failed. The holding - # state claims nothing: it is an explicit WE COULD NOT TELL, exactly - # what the sibling arm below writes for its own reason. - # - # And it must be a WRITE rather than an exit, because the built-in - # "Item closed" automation acts on the close INDEPENDENTLY of this - # workflow. A loud red run does not protect the card; the automation - # still wins the race and sets `Cancelled`. So "refuse rather than - # guess" has to be expressed AS A WRITE here -- refusing by doing - # nothing delegates the decision to something that decides wrongly. - echo "::warning::the .kanban.yml override for $REPO_FULL could not be used (see the error above: unreadable, or it names a Status the mapping does not declare), so the Status for $BASE_REF is UNKNOWN. Writing the non-terminal holding state and labelling the card for the weekly pass -- publishing nothing would let the built-in Item-closed automation set Cancelled (.github#157)." - STATUS="On dev" - UNUSABLE_OVERRIDE="true" - fi - # KEYED ON WHETHER THE MAPPER ANSWERED, not on a list of branch names - # (Bugbot, .github#295). A `case` over the four stock branches - # overwrote ANY other result with the floor below -- so a - # `.kanban.yml` key for, say, `release/*` was computed and then - # discarded on the same run, including the `rfcs` override this - # change exists to unblock. The last branch-name list in this - # workflow is gone with it. - if [ -z "$STATUS" ]; then - # A PR merged into a SIBLING feature branch deploys nothing by - # itself — its content travels onward inside the parent PR - # (this fallthrough silently stranded 6 such cards; - # backend#1437 mechanism 1). The sibling-merge label below - # makes the condition VISIBLE for the weekly board pass. The - # Status write is deliberately KEPT: skipping it lets the - # project's built-in "Item closed" automation set closed items - # to Cancelled, archiving shipped-via-parent work as abandoned - # (Bugbot, .github#157). On dev + label is the non-terminal - # holding state the weekly pass dispositions. - STATUS="On dev" - SIBLING="true" - fi - else - STATUS="Cancelled" - fi - elif [ "$EVENT_NAME" = "issues" ]; then - if [ "$ISSUE_REASON" = "completed" ]; then - # A COMPLETED ISSUE IS TERMINAL. It goes to Done, and nothing about how it - # was closed changes that (backend#2722). - # - # This used to mirror the closing PR's Status, so a PR-closed issue landed - # in `On dev` / `FR on staging` / `Prod`. That contradicted the board model - # -- deploy state is a property of a PR; an issue is either finished or it - # is not (RFC-BACKEND-1405 D8) -- and it cost real accuracy: on 2026-08-27 - # one session cleared 117 closed issues out of deploy columns by hand - # (18 + 55 from `FR on staging`, 44 from `On dev`). Two of those passes - # were functional-review batches, so the next prod payload would have read - # 73 items larger than the work it actually contained. - # - # WHY THIS RETIRES backend#1600 RATHER THAN FIGHTING IT. #1600 found that - # issues parked here by the old mirroring NEVER advanced when the code - # shipped, and drifted permanently (2026-08-06: all 20 drifted cards were - # closed issues, 0 PRs). It fixed the stranding by teaching - # advance-deploy-env to march them onward. It was right that a closed issue - # must not be stranded, and wrong about where to put it: `Done` is terminal, - # so there is nothing left to drift, and kanban-archive sweeps it off the - # board daily instead of it being carried by every hop. The matching - # closing-issue block in advance-deploy-env.yml is removed in the same - # change -- left in place it drags these cards straight back into the deploy - # columns, and a half-fix here would look fixed while doing nothing. - # - # THE CLOSER LOOKUP GOES WITH IT, which is a second win rather than - # collateral. It existed only to tell a hand-close from a PR-close so the - # two could route differently; with one destination there is nothing to - # tell apart. It was also the source of two fail-wrong defects -- a - # transient GraphQL error reading as a hand-close (.github#126), and five - # distinct situations collapsing into one "NONE" token (.github#127) -- and - # a call that is never made cannot fail. Verified before removing: - # CLOSER_TYPE and CLOSING_PR_BASE were read nowhere outside this branch. - # - # SAFE AGAINST THE FR GATE, checked rather than assumed: fr-gate ranks - # `Done` 11 against `On dev` 6, so a terminal card satisfies both the - # staging and the prod gate instead of blocking them. Its own comment - # records that being learned the hard way, when strict equality meant "a - # single Done card blocked every prod" promotion. - STATUS="Done" - else - # not_planned, or closed with no state_reason at all → abandoned. - STATUS="Cancelled" - fi - else - echo "Unexpected event '$EVENT_NAME' — skipping" - fi - - # Grouped: three consecutive individual redirects trip SC2129, and - # actionlint is a required check here. - { - echo "status=$STATUS" - echo "sibling=$SIBLING" - echo "unusable_override=$UNUSABLE_OVERRIDE" - } >> "$GITHUB_OUTPUT" - echo "Routing decision: status=${STATUS:-(unchanged)} sibling=$SIBLING unusable-override=$UNUSABLE_OVERRIDE" - - # Visibility for backend#1437 mechanism 1. Runs AFTER the Status update - # so a label failure can never suppress the column write - with the label - # first, its hard-fail skipped the update via the implicit success() - # condition and left the card wherever the built-in automation put it - # (Bugbot, .github#157). PAT rather than the caller's GITHUB_TOKEN: - # caller permission sets vary per repo. - - name: Update Status on the kanban - if: steps.target.outputs.status != '' - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - ORG: ${{ inputs.org }} - PROJECT_NUMBER: ${{ inputs.project-number }} - STATUS_NAME: ${{ steps.target.outputs.status }} - NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} - IS_PR: ${{ github.event_name == 'pull_request' }} - REPO_FULL: ${{ github.repository }} - run: | - set -euo pipefail - REPO_NAME="${REPO_FULL#*/}" - - # Look up project ID + Status field/option IDs - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - PROJ=$(gh api graphql -f query=' - query($org: String!, $num: Int!) { - organization(login: $org) { - projectV2(number: $num) { - id - fields(first: 50) { - nodes { - ... on ProjectV2SingleSelectField { id name options { id name } } - } - } - } - } - }' -F org="$ORG" -F num="$PROJECT_NUMBER") - - PROJECT_ID=$(echo "$PROJ" | jq -r '.data.organization.projectV2.id') - STATUS_FIELD=$(echo "$PROJ" | jq -r '.data.organization.projectV2.fields.nodes[] - | select(.name=="Status") | .id') - STATUS_OPT=$(echo "$PROJ" | jq -r --arg s "$STATUS_NAME" '.data.organization.projectV2.fields.nodes[] - | select(.name=="Status") | .options[] | select(.name==$s) | .id') - if [ -z "$STATUS_OPT" ] || [ "$STATUS_OPT" = "null" ]; then - echo "Could not resolve Status option '$STATUS_NAME' — aborting" - exit 1 - fi - - # Find item ID for this PR or issue - if [ "$IS_PR" = "true" ]; then - # shellcheck disable=SC2016 # $num is a GraphQL variable - must not expand in shell - FIELD_QUERY='pullRequest(number: $num) { projectItems(first: 10) { nodes { id project { number } fieldValueByName(name: "Status") { ... on ProjectV2ItemFieldSingleSelectValue { name } } } } }' - else - # shellcheck disable=SC2016 # $num is a GraphQL variable - must not expand in shell - FIELD_QUERY='issue(number: $num) { projectItems(first: 10) { nodes { id project { number } fieldValueByName(name: "Status") { ... on ProjectV2ItemFieldSingleSelectValue { name } } } } }' - fi - - # Retry briefly to let auto-add fire if needed - for i in 1 2 3 4 5; do - RESP=$(gh api graphql -f query=" - query(\$org: String!, \$repo: String!, \$num: Int!) { - repository(owner: \$org, name: \$repo) { - $FIELD_QUERY - } - }" -F org="$ORG" -F repo="$REPO_NAME" -F num="$NUMBER" 2>/dev/null) || RESP='{}' - - ITEM_ID=$(echo "$RESP" | jq -r --arg n "$PROJECT_NUMBER" ' - ([.. | objects | select(has("projectItems"))] | first).projectItems.nodes[]? - | select(.project.number == ($n | tonumber)) | .id' | head -1) - CURRENT_COL=$(echo "$RESP" | jq -r --arg n "$PROJECT_NUMBER" ' - ([.. | objects | select(has("projectItems"))] | first).projectItems.nodes[]? - | select(.project.number == ($n | tonumber)) | .fieldValueByName.name // ""' | head -1) - if [ -n "$ITEM_ID" ] && [ "$ITEM_ID" != "null" ]; then break; fi - echo "Item not yet on project, retry $i/5..." - sleep 5 - done - - if [ -z "$ITEM_ID" ] || [ "$ITEM_ID" = "null" ]; then - echo "Item #$NUMBER not on project after 5 retries — skipping." - exit 0 - fi - - # Done means "completed, nothing deployed" (D8), so it must never overwrite a - # column that records a deployment. kanban-reconcile.yml refuses this and the - # router did not, which is an asymmetry with a real path: reopen, then - # hand-close an issue sitting in On dev / FR on staging / Ready for prod, and - # the deploy state is gone -- after which kanban-archive.yml, which now - # archives Done, hides the card entirely (Bugbot, .github#126). - # - # Only Done is guarded. A PR-derived Status is a deploy fact and may advance a - # card normally. - # WHICH COLUMNS ARE DEPLOY STATES, ASKED OF THE BOARD (backend#1846). - # - # This was a hand-maintained list of six names, duplicated in - # kanban-reconcile.yml -- and it had already rotted once: it carried the - # pre-rename "Staging (human review)" and not the board's actual - # "Staging (agent review)", so a card hand-closed in that column lost its - # deploy state and kanban-archive.yml then hid it (.github#237). Fixing - # that instance left the CLASS: the next rename or inserted column - # reopens it, silently, in two files. - # - # The board already answers this. `$PROJ` carries the Status options in - # PIPELINE ORDER, so a deploy state is any column at or after "On dev" - # and at or before "Prod". An inserted column -- which is exactly how - # "Staging (agent review)" arrived -- is classified correctly with no - # edit here, and a renamed intermediate column keeps working because its - # POSITION is what matters, not its name. - # - # Two anchors instead of six names, and both are written by this same - # workflow, so .github#247's checker already asserts they exist. - col_index() { - echo "$PROJ" | jq -r --arg s "$1" \ - '[.data.organization.projectV2.fields.nodes[] - | select(.name=="Status") | .options[].name] | index($s) // -1' - } - # selftest:classify-start (byte-identical in kanban-reconcile.yml) - # Is this column a DEPLOY STATE? Answered from the board's own ORDER -- - # any column at or after "On dev" and at or before "Prod" -- so a column - # INSERTED between them is classified correctly with no edit here - # (backend#1846). Only `col_index` differs between the two workflows, - # because only their inputs differ; this decision must not. - # - # yes a deploy state - # no not one -- including NO COLUMN AT ALL, which is evidence - # nothing deployed rather than a column we cannot place - # unknown a column the board does not report - # noboard the board's anchors are MISSING OR INVERTED, so nothing can - # be placed. Inverted counts as much as missing: this helper's - # whole thesis is that POSITION decides, which makes option - # ORDER load-bearing. Drag "Prod" above "On dev" and the range - # _d.._p is unsatisfiable, so EVERY column -- "Prod" itself - # included -- comes back `no`, the router writes Done over - # shipped state and reconcile asserts it. A reordered board is - # UNREADABLE, not evidence that nothing deployed, so it fails - # closed exactly like a missing anchor (backend#1994). - classify_column() { - case "${1:-}" in - ""|"No status") echo no; return ;; - esac - _c=$(col_index "$1"); _d=$(col_index "On dev"); _p=$(col_index "Prod") - if [ "$_d" -lt 0 ] || [ "$_p" -lt 0 ] || [ "$_d" -gt "$_p" ]; then echo noboard; return; fi - if [ "$_c" -lt 0 ]; then echo unknown; return; fi - if [ "$_c" -ge "$_d" ] && [ "$_c" -le "$_p" ]; then echo yes; else echo no; fi - } - # selftest:classify-end - if [ "$STATUS_NAME" = "Done" ]; then - # UNKNOWN MUST NOT FALL OPEN -- the other half of #1846. A column this - # workflow cannot place used to sail past the `case` and let Done erase - # a deploy state. Refusing costs a card sitting where it is; falling - # open erases the fact that it shipped. - # selftest:policy-start - _ds=$(classify_column "${CURRENT_COL:-}") - case "$_ds" in - noboard) - echo "::error::the board's 'On dev'/'Prod' anchors are missing or out of order, so a deploy state cannot be recognised - refusing to set Done on #$NUMBER" - exit 1 ;; - unknown) - echo "::notice::#$NUMBER sits in '${CURRENT_COL:-}', which this board does not report as a column - NOT setting Done rather than guessing" - exit 0 ;; - yes) _protect=yes ;; - *) _protect=no ;; - esac - # selftest:policy-end - if [ "$_protect" = "yes" ]; then - echo "::notice::#$NUMBER hand-closed but sits in '$CURRENT_COL', a deploy state - NOT setting Done (D8: follow the PR's stage)" - # AND SAY SO WHERE SOMEONE WILL SEE IT. Refusing is right, but it - # parks the card with no way to self-heal, and a run-log notice is - # invisible by the time anyone looks at the board. The two real - # cases needed OPPOSITE answers -- backend#1493 had shipped via - # cli#452 and belonged in Prod; data-ingestors#488 was reverted and - # belonged in Done -- so no default is correct and only the person - # closing it knows which. - CLOSE_NOTE="Closed while the board still shows \`$CURRENT_COL\`, which records a deployment." - CLOSE_NOTE="$CLOSE_NOTE The automation will not overwrite a deploy state with \`Done\` (RFC-BACKEND-1405 D8)," - CLOSE_NOTE="$CLOSE_NOTE so this card stays where it is until someone says which happened:" - CLOSE_NOTE="$CLOSE_NOTE **it shipped** - move the card to the column it reached (\`Prod\` if it is in production);" - CLOSE_NOTE="$CLOSE_NOTE **nothing was deployed** (reverted, abandoned, superseded) - clear the deploy state, then \`Done\`." - CLOSE_NOTE="$CLOSE_NOTE Both cases are real and they need opposite answers, which is why this is not decided automatically." - gh issue comment "$NUMBER" --repo "$REPO_FULL" --body "$CLOSE_NOTE" >/dev/null 2>&1 \ - || echo "::warning::could not comment on #$NUMBER - it is parked in '$CURRENT_COL' with no note on the issue" - exit 0 - fi - fi - - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - gh api graphql -f query=' - mutation($p: ID!, $i: ID!, $f: ID!, $o: String!) { - updateProjectV2ItemFieldValue(input: { - projectId: $p, itemId: $i, fieldId: $f, - value: {singleSelectOptionId: $o} - }) { projectV2Item { id } } - }' -f p="$PROJECT_ID" -f i="$ITEM_ID" -f f="$STATUS_FIELD" -f o="$STATUS_OPT" > /dev/null - - echo "→ #$NUMBER → Status=$STATUS_NAME" - - name: Label a card whose override could not be used - if: steps.target.outputs.unusable_override == 'true' - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - REPO_FULL: ${{ github.repository }} - NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }} - run: | - set -euo pipefail - # Same write rule as the sibling label below: a swallowed failure means the - # card silently loses the only marker saying its Status is a placeholder. - # `override-unusable`, not `override-unreadable` (backend#2324): the label - # is the operator's entry point, and a card can now reach this state with a - # `.kanban.yml` that read perfectly and named a column that does not exist. - # - # NOTHING TO MIGRATE, AND THE REASON IS MEASURED RATHER THAN INFERRED - # (backend#2801). This used to rest on a claim that the fleet had no - # adopters of the override at all, which stopped being true. The - # CONCLUSION survives on its own evidence: `override-unreadable` is - # carried by ZERO issues or PRs org-wide (measured 2026-08-28), so there - # is still nothing to migrate. Do not reintroduce a clause about how many - # repos adopt the override -- it is a fact about the org that no comment - # can keep true, it was load-bearing for a different conclusion in - # scripts/branch_status_map.py, and that file's selftest now refuses it - # here too. - # - # KEEP THE DESCRIPTION UNDER 100 CHARACTERS (Bugbot, #302). That is the - # label API's cap; over it the create 422s, `set -euo pipefail` aborts the - # step, and the parked card never gets the marker this holding state exists - # to leave. My first wording was 133. The selftest measures it now. - if ! gh api "repos/$REPO_FULL/labels/override-unusable" >/dev/null 2>&1; then - gh api "repos/$REPO_FULL/labels" -f name=override-unusable -f color=d4c5f9 -f description="Holding state: .kanban.yml unreadable, or names an undeclared Status (backend#2324)" >/dev/null - fi - gh api -X POST "repos/$REPO_FULL/issues/$NUMBER/labels" -f "labels[]=override-unusable" >/dev/null - - - name: Label sibling-merged PR - if: steps.target.outputs.sibling == 'true' - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - REPO_FULL: ${{ github.repository }} - NUMBER: ${{ github.event.pull_request.number }} - run: | - set -euo pipefail - # Write failures FAIL this step (the PROJECTS_KANBAN_TOKEN write - # rule): a swallowed error means the card silently loses its only - # visibility marker (Bugbot, .github#157). Only the ensure-label - # probe may miss quietly - a 404 there just means create it. - if ! gh api "repos/$REPO_FULL/labels/sibling-merge" >/dev/null 2>&1; then - gh api "repos/$REPO_FULL/labels" -f name=sibling-merge -f color=bfdadc -f description="Merged into a sibling feature branch; content ships via the parent PR (backend#1437)" >/dev/null - fi - gh api -X POST "repos/$REPO_FULL/issues/$NUMBER/labels" -f "labels[]=sibling-merge" >/dev/null - diff --git a/.github/workflows/kanban-columns.yml b/.github/workflows/kanban-columns.yml deleted file mode 100644 index 0e6f0f7..0000000 --- a/.github/workflows/kanban-columns.yml +++ /dev/null @@ -1,101 +0,0 @@ -# The board's Status column names and the workflows that write them are two -# systems that must agree, and nothing checked that they did. -# -# The rename window for `FR on staging` was carried by hand across three PRs -# (backend#1592). In between, the writers emitted a column the board did not -# have, and only a fallback made it work — while an unresolvable Status was a -# `::warning::` on a GREEN run (fixed in #246), so a frozen board and a working -# board were indistinguishable. -# -# Runs on PRs that touch the writers OR this check, and daily — because the -# board can be renamed in the UI at any time, with no PR to hang a check on. -# That is the failure this is really for: a rename nobody pairs with a code -# change. -name: Kanban column conformance - -on: - pull_request: - # MUST LIST EVERY FILE IN `WRITERS`. It listed only the original two, so a PR - # touching just set-pr-status.yml or fr-pass-comment.yml never ran this check - # and a bad column write could merge until the next cron (Bugbot, #248). The - # selftest asserts this list against WRITERS, so they cannot drift apart. - paths: - - ".github/workflows/advance-deploy-env.yml" - - ".github/workflows/kanban-closure-router.yml" - - ".github/workflows/set-pr-status.yml" - - ".github/workflows/fr-pass-comment.yml" - # Added with their WRITERS entries (.github#295): a PR touching a file whose - # column names this check verifies must RUN the check, or the names are only - # ever checked by the daily cron. - - ".github/workflows/kanban-archive.yml" - - ".github/workflows/wip-limit-check.yml" - # backend#2348: its bug-to-ready job names `Backlog` and `Ready`, so a PR - # touching it must run this check rather than waiting for the daily cron. - - ".github/workflows/customer-priority-bump.yml" - - ".github/workflows/kanban-columns.yml" - # The mapping is an INPUT to the check now (backend#2243), so a PR that only - # touches it must run this (Bugbot, .github#295) -- otherwise a new Status can - # merge and go unchecked until the daily cron. - - "scripts/branch_status_map.py" - - "scripts/kanban-columns-check.py" - - "scripts/tests/kanban-columns-selftest.py" - schedule: - # 06:15 UTC, before caller-drift at 06:30 — a board rename breaks promotions, - # so it should be the first thing the morning tells you. - - cron: "15 6 * * *" - workflow_dispatch: - -permissions: - contents: read - -concurrency: - group: kanban-columns - cancel-in-progress: false - -jobs: - selftest: - name: Selftest the checker - runs-on: ubuntu-latest - timeout-minutes: 10 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # Guard the guard: a checker that cannot fail is worse than no checker, - # because its green is read as conformance. - - run: python3 scripts/tests/kanban-columns-selftest.py - - check: - name: Written Status names exist on the board - # Gate on selftest: a conformance PASS emitted by a checker whose own - # failure paths have regressed is a false green (the checker can't fail, so - # its success means nothing). Sibling guards caller-drift + standards-sync - # gate their audit on selftest for exactly this reason. (Bugbot, #247.) - needs: selftest - runs-on: ubuntu-latest - timeout-minutes: 10 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - # Authenticates as the tracebloc-release-train App (backend#2036) rather than - # a human's PAT. `owner:` yields an ORG-scoped installation token; a - # repo-scoped one cannot read an org ProjectV2. No fallback to the old PAT. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # LEAST PRIVILEGE, DERIVED FROM WHAT THIS RUNS (backend#2157). The job is - # `kanban-columns-check.py`, which READS the project's field vocabulary and - # writes nothing anywhere -- so `organization-projects: read` is the whole - # requirement, and every other permission the App holds was surplus. - # - # `repositories:` narrows the repo-level half to this repo. It does not - # affect the board: `organization_projects` is an ORG permission, measured - # unaffected by repo scoping in backend#2181's run 32255581084. - repositories: ${{ github.event.repository.name }} - permission-organization-projects: read - - name: Check - env: - # Reads the org project's Status field. - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: python3 scripts/kanban-columns-check.py diff --git a/.github/workflows/kanban-deploy-state-selftest.yml b/.github/workflows/kanban-deploy-state-selftest.yml deleted file mode 100644 index dbbd337..0000000 --- a/.github/workflows/kanban-deploy-state-selftest.yml +++ /dev/null @@ -1,54 +0,0 @@ -name: Kanban deploy-state selftest - -# The classification that decides whether `Done` may overwrite a card's column -# (backend#1846). It is asserted rather than trusted because its predecessor -- a -# hand-maintained list of six column names -- rotted silently and cost a card its -# deploy state (.github#237). -# -# The test reads `col_index()` OUT of kanban-closure-router.yml rather than -# copying it, so it cannot go green against a stale duplicate. Offline, no token. - -on: - pull_request: - paths: - - .github/workflows/kanban-closure-router.yml - # RECONCILE TOO (Bugbot, .github#252). It carries the second copy of the - # classification, and omitting it here meant a PR touching only that copy - # never ran this check -- the guard could stay green while the thing it - # guards changed underneath it. - - .github/workflows/kanban-reconcile.yml - - .github/workflows/kanban-deploy-state-selftest.yml - - scripts/tests/kanban-deploy-state-selftest.py - push: - branches: [main, develop, staging] - paths: - - .github/workflows/kanban-closure-router.yml - # RECONCILE TOO (Bugbot, .github#252). It carries the second copy of the - # classification, and omitting it here meant a PR touching only that copy - # never ran this check -- the guard could stay green while the thing it - # guards changed underneath it. - - .github/workflows/kanban-reconcile.yml - - .github/workflows/kanban-deploy-state-selftest.yml - - scripts/tests/kanban-deploy-state-selftest.py - -permissions: - contents: read - -concurrency: - group: kanban-deploy-state-selftest-${{ github.ref }} - cancel-in-progress: true - -jobs: - selftest: - runs-on: ubuntu-latest - timeout-minutes: 5 - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 - with: - python-version: '3.12' - # PyYAML only: the test parses the workflow to extract the function under - # test. `pip install` without setup-python is what left the bricked-PR - # audit unable to start on Ubuntu 24.04's PEP 668 python (.github#244). - - run: pip install --quiet pyyaml - - run: python scripts/tests/kanban-deploy-state-selftest.py diff --git a/.github/workflows/kanban-reconcile.yml b/.github/workflows/kanban-reconcile.yml deleted file mode 100644 index c6f22f6..0000000 --- a/.github/workflows/kanban-reconcile.yml +++ /dev/null @@ -1,1081 +0,0 @@ -name: Kanban reconcile - -# Weekly self-healing backstop for the engineer kanban (project #2). -# -# Catches stalls and drift that the event-driven workflows -# (advance-deploy-env, kanban-closure-router) silently miss: -# -# 1. Drift to Prod - items in RfP / FR-on-staging / FR-on-dev / Code review -# whose merge SHA is already on the repo's prod branch. -# Cause: the push event was missed, or the deploy -# happens out-of-band (e.g. averaging-service deploys -# a Docker image from staging without ever pushing main). -# -# 2. Cancelled - closed-not-merged PRs sitting in non-terminal columns. -# -# 3. Misplaced open issues - open issues that drifted into a post-merge -# column (RfP, FR-on-*, Ready-for-staging). -# Move back to Backlog. -# -# 4. Missing from board - open or recently-merged PRs/issues that were never -# added by the event-driven add-to-kanban workflow. -# That workflow is fire-and-forget (one shot on -# 'opened', no retry), so a single dropped webhook -# leaves a permanent silent gap that nothing else -# catches. We sweep every board-tracked repo, find -# items with no project #2 entry, and add them in a -# sensible starting column. Archived items count as -# present, so the daily archiver is never fought. -# -# 5. Closed-issue terminalize - a CLOSED issue in any non-terminal column. -# The closure-router only moves issues closed by a linked -# PR (and misses even those to a close-time race), and -# never moves hand-closed ones -- so completed issues pile -# up wherever they were. Route by state_reason: -# not_planned/duplicate -> Cancelled, else -> Prod. -# -# Safety cap: if the script would move > 100 items OR add > 60 items in one -# run, it aborts and reports - that's almost certainly a bug, not real drift. - -on: - schedule: - - cron: '0 4 * * 1' # 04:00 UTC Mondays — weekly backstop (event-driven flows are the primary path) - workflow_dispatch: - inputs: - dry-run: - description: "Log moves without applying them" - type: boolean - default: false - -permissions: - contents: read - -concurrency: - group: kanban-reconcile - cancel-in-progress: false - -jobs: - reconcile: - runs-on: ubuntu-latest - env: - ORG: tracebloc - PROJECT_NUMBER: 2 - DRY_RUN: ${{ github.event.inputs.dry-run || 'false' }} - MAX_MOVES: 100 - steps: - # CHECKED OUT BECAUSE THIS JOB NOW RUNS A SCRIPT (Bugbot, .github#295). My - # note said "branch_status_map.py is local: this job only ever runs in - # tracebloc/.github" -- true of the REPOSITORY and false of the WORKSPACE. A - # hosted runner starts empty, so the stage-derivation step would have failed on - # a missing file and taken the weekly backstop down mid-sweep. `persist- - # credentials: false` because nothing here pushes. - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - persist-credentials: false - - # Authenticates as the tracebloc-release-train App (backend#2036) instead of - # one human's PAT -- the credential this sweep shares with fifteen other - # workflows and with its owner's own shell. Needs both grants: - # `organization_projects: write` to move and archive cards, `issues: write` - # for the keep-open shield. `owner:` makes the token ORG-scoped; a repo-scoped - # one cannot reach an org ProjectV2. - # - # GH_TOKEN moved from job-level env to per-step, because a job-level env cannot - # read a step output. The file now names which steps hold a credential. - # - # No fallback to the old PAT: a fallback would let a broken App path look like - # a working migration. - - name: Mint an installation token - id: app-token - uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0 - with: - app-id: ${{ secrets.RELEASE_TRAIN_APP_ID }} - private-key: ${{ secrets.RELEASE_TRAIN_APP_PRIVATE_KEY }} - owner: ${{ github.repository_owner }} - # NARROWED, NOT MADE READ-ONLY (backend#2157). This backstop genuinely - # writes: three ProjectV2 mutations, and it CREATES labels - # (`POST repos/{o}/{r}/labels`) and applies them - # (`POST .../issues/{n}/labels`) for `keep-open` and `sibling-merge`. - # Pretending otherwise would break the Monday run. - # - # THE WIN HERE IS `contents`. It was minting contents:WRITE across - # every repo the installation covers -- the capability that can push - # code anywhere -- to run a job whose only content access is READING - # `.github/workflows/add-to-kanban.yml` and a couple of branch refs. - # That drops to read, and administration:read / actions:read / - # checks:read go entirely. - # - # DERIVED FROM THE CALLS: - # contents/.github/workflows/*, branches/{main,master} -> contents: read - # labels create + apply -> issues: WRITE - # the same labels applied to PRs -> pull-requests: WRITE - # addProjectV2ItemById / updateProjectV2ItemFieldValue / - # archiveProjectV2Item -> projects: WRITE - # - # `pull-requests: write` is here because the labels land on PRs as well - # as issues and the labels endpoint is shared. If a dry-run dispatch - # shows `issues: write` alone covers it, this can narrow again -- but - # guessing it short would fail on the one path that only runs weekly. - permission-contents: read - permission-issues: write - permission-pull-requests: write - permission-organization-projects: write - - - name: Resolve project + field/option IDs - id: ids - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - PROJ=$(gh api graphql -f query=' - query($org: String!, $num: Int!) { - organization(login: $org) { - projectV2(number: $num) { - id - fields(first: 50) { - nodes { - ... on ProjectV2SingleSelectField { - id name options { id name } - } - } - } - } - } - }' -F org="$ORG" -F num="$PROJECT_NUMBER") - - PROJECT_ID=$(echo "$PROJ" | jq -r '.data.organization.projectV2.id') - STATUS_FIELD=$(echo "$PROJ" | jq -r '.data.organization.projectV2.fields.nodes[] | select(.name=="Status") | .id') - opt() { - echo "$PROJ" | jq -r --arg n "$1" '.data.organization.projectV2.fields.nodes[] - | select(.name=="Status") | .options[] | select(.name==$n) | .id' - } - # Rename window (backend#1592): ask for the NEW column name, fall back to - # the OLD one. The UI rename flips the board in a single instant with no - # period where both exist, so every resolver has to work on both sides of - # it. Removed in step 3 of #1592. - opt_either() { - _v=$(opt "$1") - if [ -z "$_v" ] || [ "$_v" = "null" ]; then _v=$(opt "$2"); fi - printf '%s' "$_v" - } - { - echo "project_id=$PROJECT_ID" - echo "status_field=$STATUS_FIELD" - echo "prod_opt=$(opt Prod)" - echo "done_opt=$(opt Done)" - echo "cancelled_opt=$(opt Cancelled)" - echo "backlog_opt=$(opt Backlog)" - echo "code_review_opt=$(opt 'Code review')" - echo "in_progress_opt=$(opt 'In progress')" - echo "on_dev_opt=$(opt 'On dev')" - echo "fr_staging_opt=$(opt_either 'Staging (human review)' 'FR on staging')" - # THE BOARD'S COLUMN ORDER, published because `$PROJ` does not survive - # this step -- every `run:` is a new shell (Bugbot, .github#252). The - # classify step needs the ORDER to tell a deploy state from a planning - # column without a hand-maintained list (backend#1846), and it has no - # project JSON of its own; it works off the option ids above. - # Tab-separated: a Status column may contain spaces and parentheses, - # but never a tab. - printf 'status_order=%s\n' "$(echo "$PROJ" | jq -r ' - [.data.organization.projectV2.fields.nodes[] - | select(.name=="Status") | .options[].name] | join("\t")')" - } >> "$GITHUB_OUTPUT" - - - name: Pull non-terminal items - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - : > items.ndjson - cursor="null" - while :; do - if [ "$cursor" = "null" ]; then - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - OUT=$(gh api graphql -f query=' - query($org: String!, $num: Int!) { - organization(login: $org) { - projectV2(number: $num) { - items(first: 100) { - pageInfo { hasNextPage endCursor } - nodes { - id - fieldValueByName(name: "Status") { - ... on ProjectV2ItemFieldSingleSelectValue { name } - } - content { - __typename - ... on PullRequest { - number title state mergedAt baseRefName headRefName - mergeCommit { oid } - repository { name defaultBranchRef { name } } - } - ... on Issue { - number title state stateReason - repository { name defaultBranchRef { name } } - } - } - } - } - } - } - }' -F org="$ORG" -F num="$PROJECT_NUMBER") - else - # shellcheck disable=SC2016 # the $names here are GraphQL variables, not shell - keep literal - OUT=$(gh api graphql -f query=' - query($org: String!, $num: Int!, $c: String!) { - organization(login: $org) { - projectV2(number: $num) { - items(first: 100, after: $c) { - pageInfo { hasNextPage endCursor } - nodes { - id - fieldValueByName(name: "Status") { - ... on ProjectV2ItemFieldSingleSelectValue { name } - } - content { - __typename - ... on PullRequest { - number title state mergedAt baseRefName headRefName - mergeCommit { oid } - repository { name defaultBranchRef { name } } - } - ... on Issue { - number title state stateReason - repository { name defaultBranchRef { name } } - } - } - } - } - } - } - }' -F org="$ORG" -F num="$PROJECT_NUMBER" -F c="$cursor") - fi - # We also pull the planning columns (Backlog / In progress / No status - # / Ready / North Stars) so that (a) closed promotion-PR cards hiding - # there can be archived and (b) CLOSED issues stranded there get - # terminalized. Classify leaves everything else in them untouched. - echo "$OUT" | jq -c '.data.organization.projectV2.items.nodes[] - | select((.fieldValueByName.name // "No status") as $s - | ["Code review","On dev", - "Staging (agent review)", - "Staging (human review)","FR on staging","Ready for prod", - "Backlog","In progress","No status", - "Ready","North Stars"] | index($s))' \ - >> items.ndjson - - HAS_NEXT=$(echo "$OUT" | jq -r '.data.organization.projectV2.items.pageInfo.hasNextPage') - cursor=$(echo "$OUT" | jq -r '.data.organization.projectV2.items.pageInfo.endCursor') - [ "$HAS_NEXT" = "true" ] || break - done - echo "Found $(wc -l < items.ndjson) non-terminal items to evaluate" - - - name: Shield North Stars from the stale sweep - # stale-backlog.yml is column-blind (backend#1597, from the #1408 - # audit): a North Stars epic quiet for 6+8 weeks gets warned, closed, - # and routed to Cancelled - a strategic priority silently archived. - # The stale sweep already exempts the `keep-open` label, so this step - # keeps that label in sync with the board: every ISSUE currently in - # North Stars gets keep-open (idempotent - adding an existing label is - # a no-op). One-way by design: removal stays with humans, because the - # label is also applied manually and an auto-remove would fight them. - # Write failures FAIL the step (the board-write rule) -- but at the END of - # the sweep, not on the first one, so one unwritable repo cannot cost every - # issue behind it its shield. Named for the rule, not the credential: - # backend#2036 moved these writes to the App, and a rule named after a - # secret goes stale the moment the secret does. - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - SHIELDED=0 - shield_fail=0 - while IFS= read -r line; do - STATUS=$(echo "$line" | jq -r '.fieldValueByName.name // "No status"') - TYPE=$(echo "$line" | jq -r '.content.__typename // ""') - STATE=$(echo "$line" | jq -r '.content.state // ""') - # Only OPEN North Stars issues need the stale-sweep shield; a CLOSED - # issue would get a sticky, human-removal-only keep-open that the same - # run's classify step may then terminalize (Bugbot .github#166). - if [ "$STATUS" != "North Stars" ] || [ "$TYPE" != "Issue" ] || [ "$STATE" = "CLOSED" ]; then - continue - fi - REPO=$(echo "$line" | jq -r '.content.repository.name') - NUM=$(echo "$line" | jq -r '.content.number') - if [ "$DRY_RUN" = "true" ]; then - echo "[DRY] would ensure keep-open on $REPO#$NUM" - continue - fi - # COUNTED, NOT FATAL-ON-FIRST -- the same shape the membership sweep - # below uses, and for a reason this loop makes sharp. These are bare - # writes under `set -e`, so the FIRST failure aborted the whole sweep - # and every North Stars issue after it silently kept no `keep-open`: - # one unwritable repo cost the shield for all the others, and the - # step's own log showed only where it stopped, not what it skipped. - # - # The likely failure is a 403 from a repo outside the App's - # installation (`repository_selection: selected`), which is not a - # property of the issue being processed -- so it must not decide the - # fate of the issues behind it. - # A failed CREATE only WARNS, and the write is still attempted -- the - # same order the sibling-merge sequence below uses. The existence GET - # and the create are not the assertion; the LABEL WRITE is. A 422 - # `already_exists` after a GET that failed for its own reasons (a race, - # a blip) means the label is there and the POST will succeed, so - # giving up here would count a failure and leave an issue unshielded - # over a probe, not over the thing being asserted. - if ! gh api "repos/$ORG/$REPO/labels/keep-open" >/dev/null 2>&1; then - gh api "repos/$ORG/$REPO/labels" -f name=keep-open -f color=0e8a16 -f description="Exempt from the stale-backlog sweep" >/dev/null \ - || echo "[WARN] $REPO#$NUM: could not CREATE the keep-open label; attempting the write anyway" - fi - # KEEP THE ERROR, exactly as the sibling-merge write does. `2>&1 - # >/dev/null` captures stderr (the API error) and drops the label JSON - # nobody reads; the other order would throw away the only diagnostic - # the [FAIL] line has. A bare "the write failed" cannot tell a 403 - # from a rate limit from a missing scope -- and a 403 from a repo - # outside the App's installation is the specific failure this change - # exists to survive, so the log has to be able to name it. - if ! label_err="$(gh api -X POST "repos/$ORG/$REPO/issues/$NUM/labels" -f "labels[]=keep-open" 2>&1 >/dev/null)"; then - echo "[FAIL] $REPO#$NUM: keep-open write failed: ${label_err:-(no output)}" - shield_fail=$((shield_fail+1)) - continue - fi - SHIELDED=$((SHIELDED+1)) - echo "[OK] keep-open ensured on $REPO#$NUM (North Stars)" - done < items.ndjson - echo "North Stars shield: $SHIELDED issue(s) ensured, $shield_fail failed" - # Still fails the step -- the board-write rule is unchanged. What changed - # is WHEN: after every issue has been attempted, so the run reports the - # complete set of unshielded issues instead of the first one. A North - # Stars epic without `keep-open` is closed by the stale sweep and routed - # to Cancelled, which is exactly what this step exists to prevent. - if [ "$shield_fail" -ne 0 ]; then - echo "::error::$shield_fail North Stars issue(s) could not be shielded (see the [FAIL] lines above). They stay exposed to stale-backlog.yml, which closes and Cancels them." - exit 1 - fi - - - name: Classify + plan moves - id: plan - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - : > moves.tsv - : > skip.tsv - - declare -A PROD_BRANCH_CACHE - resolve_prod_branch() { - local repo="$1" default_branch="$2" - if [ -n "${PROD_BRANCH_CACHE[$repo]:-}" ]; then - echo "${PROD_BRANCH_CACHE[$repo]}"; return - fi - # Prod is conventionally main or master. Probe those directly instead - # of trusting a repo-controlled .kanban.yml override: a repo could - # commit `branch_status_map: {develop: Prod}` and self-grant prod - # status to every develop merge (D27-L4). Read from the repo, not a - # file the PR author controls. Fall back to the default branch only - # if neither main nor master exists. - local branch="$default_branch" - if gh api "/repos/$ORG/$repo/branches/main" --jq '.name' >/dev/null 2>&1; then - branch="main" - elif gh api "/repos/$ORG/$repo/branches/master" --jq '.name' >/dev/null 2>&1; then - branch="master" - fi - PROD_BRANCH_CACHE[$repo]="$branch" - echo "$branch" - } - - sha_on_branch() { - local repo="$1" branch="$2" sha="$3" - { [ -z "$sha" ] || [ "$sha" = "null" ]; } && return 1 - local status - status=$(gh api "/repos/$ORG/$repo/compare/$branch...$sha" \ - --jq '.status' 2>/dev/null) || return 1 - [ "$status" = "behind" ] || [ "$status" = "identical" ] - } - - PROD_OPT="${{ steps.ids.outputs.prod_opt }}" - DONE_OPT="${{ steps.ids.outputs.done_opt }}" - CANCELLED_OPT="${{ steps.ids.outputs.cancelled_opt }}" - BACKLOG_OPT="${{ steps.ids.outputs.backlog_opt }}" - # `ON_DEV_OPT` AND `FR_STAGING_OPT` USED TO BE RESOLVED HERE, and the - # comment said they were "needed by the router-miss backstop below: - # when the closure router failed to set a stage, this step derives it - # from the closing PR's base branch". backend#2722 removed that - # derivation -- a completed issue is terminal, not a deploy state -- so - # both became dead here and shellcheck said so (SC2034). They are still - # resolved in the step that sweeps drifted PR cards, which is the only - # place a deploy column is a legitimate destination. - # The board's Status columns IN ORDER, resolved in the `ids` step -- - # `$PROJ` does not survive into this shell (Bugbot, .github#252). - STATUS_ORDER="${{ steps.ids.outputs.status_order }}" - - # WHICH COLUMNS ARE DEPLOY STATES, ASKED OF THE BOARD (backend#1846). - # The twin of the router's helper (.github#249), and the reason the class - # stayed open after #237: TWO hand-maintained lists had to agree and one - # rotted -- the router carried the pre-rename "Staging (human review)" - # while the board's column is "Staging (agent review)", so a card - # hand-closed there lost its deploy state. The board's own ORDER is the - # thing neither copy can drift from. - # awk, NOT `grep | head | cut`. Under `set -euo pipefail` a grep that - # finds nothing fails the pipeline, and `_ci=$(col_index ...)` is a bare - # assignment -- so an UNRECOGNISED column would abort the whole classify - # step instead of being handled as unplaceable. Verified: the assignment - # form never reached the next line. Same trap as release-train#73, and - # this is the reason a helper that reports "not found" must not do it by - # failing. awk always exits 0 and prints -1 for a miss. - col_index() { - printf '%s' "$STATUS_ORDER" | tr '\t' '\n' \ - | awk -v s="$1" 'BEGIN{i=-1} i<0 && $0==s {i=NR} END{print i}' - } - # selftest:classify-start (byte-identical in kanban-reconcile.yml) - # Is this column a DEPLOY STATE? Answered from the board's own ORDER -- - # any column at or after "On dev" and at or before "Prod" -- so a column - # INSERTED between them is classified correctly with no edit here - # (backend#1846). Only `col_index` differs between the two workflows, - # because only their inputs differ; this decision must not. - # - # yes a deploy state - # no not one -- including NO COLUMN AT ALL, which is evidence - # nothing deployed rather than a column we cannot place - # unknown a column the board does not report - # noboard the board's anchors are MISSING OR INVERTED, so nothing can - # be placed. Inverted counts as much as missing: this helper's - # whole thesis is that POSITION decides, which makes option - # ORDER load-bearing. Drag "Prod" above "On dev" and the range - # _d.._p is unsatisfiable, so EVERY column -- "Prod" itself - # included -- comes back `no`, the router writes Done over - # shipped state and reconcile asserts it. A reordered board is - # UNREADABLE, not evidence that nothing deployed, so it fails - # closed exactly like a missing anchor (backend#1994). - classify_column() { - case "${1:-}" in - ""|"No status") echo no; return ;; - esac - _c=$(col_index "$1"); _d=$(col_index "On dev"); _p=$(col_index "Prod") - if [ "$_d" -lt 0 ] || [ "$_p" -lt 0 ] || [ "$_d" -gt "$_p" ]; then echo noboard; return; fi - if [ "$_c" -lt 0 ]; then echo unknown; return; fi - if [ "$_c" -ge "$_d" ] && [ "$_c" -le "$_p" ]; then echo yes; else echo no; fi - } - # selftest:classify-end - - while IFS= read -r line; do - ITEM_ID=$(echo "$line" | jq -r '.id') - COL=$(echo "$line" | jq -r '.fieldValueByName.name // "No status"') - TYPE=$(echo "$line" | jq -r '.content.__typename') - REPO=$(echo "$line" | jq -r '.content.repository.name') - DEFAULT=$(echo "$line" | jq -r '.content.repository.defaultBranchRef.name // "main"') - NUM=$(echo "$line" | jq -r '.content.number') - - if [ "$TYPE" = "PullRequest" ]; then - STATE=$(echo "$line" | jq -r '.content.state') - SHA=$(echo "$line" | jq -r '.content.mergeCommit.oid // ""') - HEADREF=$(echo "$line" | jq -r '.content.headRefName // ""') - - # Release-train promotion PRs are plumbing, not work items: the - # fr-gate ignores them and the membership sweep never adds them. - # Archive the card once the PR is no longer open; leave open ones - # alone (the train manages them). - if [ "${HEADREF#release-train/}" != "$HEADREF" ]; then - if [ "$STATE" = "OPEN" ]; then - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "-" "$REPO#$NUM open promotion PR (train-managed)" "waiting" >> skip.tsv - else - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "ARCHIVE" "$REPO#$NUM promotion PR card ($COL -> archive)" "archive-promotion" >> moves.tsv - fi - continue - fi - - # Pre-merge columns are pulled only for the promotion-PR archive - # above -- leave everything else in them to the event-driven flows. - case "$COL" in - "Backlog"|"In progress"|"No status") - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "-" "$REPO#$NUM PR in $COL (pre-merge, no action)" "no-action" >> skip.tsv - continue ;; - esac - - if [ "$STATE" = "CLOSED" ]; then - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "$CANCELLED_OPT" "$REPO#$NUM closed-not-merged ($COL)" "cancelled" >> moves.tsv - continue - fi - - if [ "$STATE" = "MERGED" ] && [ -n "$SHA" ]; then - PROD=$(resolve_prod_branch "$REPO" "$DEFAULT") - if sha_on_branch "$REPO" "$PROD" "$SHA"; then - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "$PROD_OPT" "$REPO#$NUM SHA on $PROD ($COL -> Prod)" "drift-to-prod" >> moves.tsv - continue - fi - fi - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "-" "$REPO#$NUM PR in $COL (legit waiting)" "waiting" >> skip.tsv - continue - fi - - if [ "$TYPE" = "Issue" ]; then - STATE=$(echo "$line" | jq -r '.content.state') - SR=$(echo "$line" | jq -r '.content.stateReason // ""') - - # A CLOSED issue in ANY non-terminal column is stale clutter. The - # closure-router only moves issues closed by a linked PR (and can - # miss even those to a close-time race), and never moves hand-closed - # ones -- so completed issues pile up wherever they were closed. - # Terminalize by reason: not_planned / duplicate -> Cancelled, - # otherwise (completed / unspecified) -> Prod. Safe now that - # fr-gate.yml attributes promotions via commits->PRs, so a card - # parked in Prod cannot satisfy the gate. - if [ "$STATE" = "CLOSED" ]; then - if [ "$SR" = "NOT_PLANNED" ] || [ "$SR" = "DUPLICATE" ]; then - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "$CANCELLED_OPT" "$REPO#$NUM closed issue $SR ($COL -> Cancelled)" "cancelled" >> moves.tsv - else - # Done means "completed, nothing deployed" (D8), so it needs TWO things - # to be true: the card carries no deploy state, AND nothing closed it. - # A closed-completed issue already sitting in a deploy column got there - # from a real deploy signal -- so Done would - # assert nothing shipped about work that did, and would overwrite what - # kanban-closure-router correctly set (Bugbot, .github#126). - # - # D8's routing for those is "follow the PR's stage", i.e. leave them. - # That does leave a closed card in a non-terminal column, which D8's - # third invariant forbids -- the two rules genuinely conflict for work - # that shipped to dev/staging and then stopped. Asserting something - # false is the worse of the two, so this reports and moves on; the - # conflict is flagged on backend#1411 for a human to settle. - # A deploy state is any column at or after "On dev" and at or - # before "Prod", in the board's own order -- so a column - # INSERTED between them (which is exactly how "Staging (agent - # review)" arrived) is classified correctly with no edit here. - # UNKNOWN MUST NOT FALL OPEN, and it leans the OPPOSITE way to - # the router's: this loop decides whether to ASSERT Done, so a - # column it cannot place -- or a board missing an anchor -- is - # treated AS a deploy state and left alone. Leaving a card where - # it is costs nothing; asserting "nothing shipped" over work - # that did erases the fact. - # selftest:policy-start - _ds=$(classify_column "$COL") - if [ "$_ds" != "no" ]; then _skip=yes; else _skip=no; fi - # selftest:policy-end - if [ "$_skip" = "yes" ]; then - echo " [SKIP] $REPO#$NUM closed-completed in $COL - a deploy state (or unplaceable), so NOT Done; D8 says follow the PR's stage" - else - # The column is NOT sufficient evidence on its own. It was the only - # check here until now, and it is circular: the column is exactly - # what is wrong when kanban-closure-router misses a close -- its - # closer lookup can fail, in which case it deliberately leaves - # Status alone and the card stays wherever it was. Asserting Done - # off that column claims "nothing deployed" about work that shipped, - # and kanban-archive then hides it (Bugbot, .github#127). - # - # So ask the question directly: is there a closer? Only a genuinely - # closer-less issue is "completed, nothing deployed". - # shellcheck disable=SC2016 # $names are GraphQL variables - CLOSER=$(gh api graphql -f query=' - query($org: String!, $repo: String!, $num: Int!) { - repository(owner: $org, name: $repo) { - issue(number: $num) { - timelineItems(last: 10, itemTypes: [CLOSED_EVENT]) { - nodes { ... on ClosedEvent { - closer { __typename ... on PullRequest { baseRefName } } - } } - } - } - } - }' -F org="$ORG" -F repo="$REPO" -F num="$NUM" \ - --jq '((.data.repository.issue.timelineItems.nodes // []) | last) as $e - | if $e == null then "NOEVENT|" - else (($e.closer.__typename // "NONE") + "|" + ($e.closer.baseRefName // "")) end' \ - 2>/dev/null) || CLOSER="ERROR|" - CLOSER_BASE="${CLOSER#*|}" - CLOSER="${CLOSER%%|*}" - case "$CLOSER" in - # NONE means the timeline HAD a ClosedEvent and its closer was - # null -- an actual hand-close. "No ClosedEvent at all" is - # NOEVENT, and a null issue or repository is too; those five - # shapes were one token until Bugbot flagged it on .github#127. - # NOEVENT falls through to the SKIP below, where absence of - # evidence belongs. - NONE) - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "$DONE_OPT" "$REPO#$NUM closed issue completed, no closer ($COL -> Done)" "closed-issue-terminalized" >> moves.tsv - ;; - PullRequest|Commit) - # TERMINAL, NOT A DEPLOY STATE (backend#2722). This arm used - # to derive a stage from the closer's base branch and write - # `On dev` / `FR on staging` / `Prod`, mirroring the router. - # The router no longer does that for ISSUES, and a backstop - # that still did would be the slower job silently undoing the - # faster one -- the exact failure .github#295's comment above - # was written to prevent, arriving from the other direction - # (Bugbot, High). - # - # A deploy column is a property of a PR. A finished ISSUE - # belongs in `Done` whether or not its fix shipped, and - # leaving issue cards in deploy columns is what stopped - # `kanban-archive` clearing them: on 2026-08-25, 16 closed - # issues sat in `Ready for prod` whose fixes were only on - # `develop`, which is three columns of overstated progress - # that nothing swept. - # - # Having a closer is still what distinguishes this from the - # NONE arm in the LOG, so the reason stays legible -- but both - # now reach the same column, which is the whole point of the - # rule. - # - # SAFE FOR THE SAME REASON THE NONE ARM IS: the enclosing - # `_skip` guard already required `classify_column "$COL"` to - # be `no`, so this card is in neither a deploy state nor a - # column the board cannot place. Writing Done here cannot - # erase a deployment. - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "$DONE_OPT" "$REPO#$NUM closed issue completed, closed by $CLOSER (base=${CLOSER_BASE:-none}) ($COL -> Done)" "closed-issue-terminalized" >> moves.tsv - ;; - NOEVENT) - # The read succeeded and the timeline carried no ClosedEvent at - # all. Silence is not a hand-close, so it is not Done either. - echo " [SKIP] $REPO#$NUM closed-completed but its timeline carries no ClosedEvent - no evidence either way, so NOT Done" - ;; - *) - # Unreadable. Not evidence of absence. - echo " [SKIP] $REPO#$NUM closed-completed but the closer lookup failed ($CLOSER) - refusing to assert Done" - ;; - esac - fi - fi - continue - fi - - # OPEN issue in a pre-merge / planning column: it belongs there (or - # the event flows handle it) -> no action. - case "$COL" in - "Backlog"|"In progress"|"No status"|"Ready"|"North Stars") - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "-" "$REPO#$NUM open issue in $COL (no action)" "no-action" >> skip.tsv - continue ;; - esac - - # OPEN issue that drifted into a post-merge column -> back to Backlog. - # `Staging (agent review)` belongs here with its siblings: it is a - # post-merge column, so an OPEN issue in it has drifted just as it - # would have in `On dev`. Added in the READ hop (#1577) so reconcile - # already treats it correctly before anything writes it (#1578). - case "$COL" in - "Ready for prod"|"Staging (agent review)"|"Staging (human review)"|"FR on staging"|"On dev") - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "$BACKLOG_OPT" "$REPO#$NUM open issue in $COL -> Backlog" "misplaced-issue" >> moves.tsv - continue ;; - esac - printf '%s\t%s\t%s\t%s\n' "$ITEM_ID" "-" "$REPO#$NUM issue in $COL (no action)" "no-action" >> skip.tsv - fi - done < items.ndjson - - PLANNED=$(wc -l < moves.tsv | tr -d ' ') - echo "planned=$PLANNED" >> "$GITHUB_OUTPUT" - echo "Planned moves: $PLANNED" - echo "::group::Planned moves" - column -t -s$'\t' moves.tsv || cat moves.tsv - echo "::endgroup::" - - if [ "$PLANNED" -gt "$MAX_MOVES" ]; then - echo "::error::$PLANNED moves exceeds MAX_MOVES=$MAX_MOVES - aborting. Investigate." - exit 1 - fi - - - name: Apply moves - if: steps.plan.outputs.planned != '0' && env.DRY_RUN != 'true' - env: - GH_TOKEN: ${{ steps.app-token.outputs.token }} - run: | - set -euo pipefail - PROJECT_ID="${{ steps.ids.outputs.project_id }}" - STATUS_FIELD="${{ steps.ids.outputs.status_field }}" - ok=0; fail=0 - # `label` is field 4 of moves.tsv (the move category, consumed by awk in the - # Per-label summary step). It is read here only so that `read` does not fold - # field 4 into $reason - without a trailing variable the [OK]/[FAIL] lines - # below would print "