docs(gates): inventory all 81 gates/tiers/batteries/hooks with evidence β 67 have a named catch, 0 earn a DROP - #1375
Conversation
β¦ce β 67 have a named catch, 0 earn a DROP clawgate task #525. Every gate, tier, battery and hook devrc runs, each row carrying either a named catch (session id, CI run, or commit sha) or the explicit string `no evidence found` with the search that was run, plus a KEEP/DROP/TIER verdict. Nothing that runs is changed. No gate, target, battery or hook is dropped, disabled or made conditional, and no per-path selection rule is implemented β both are the task's explicit non-goals, and the verdict column is a recommendation the operator decides on. Enumeration is DERIVED at scan time, not taken from the task's own list: 31 pytest targets (TARGET_FLOORS), 5 node suites (SUITES), 2 nix check derivations, 10 mutation batteries, 16 claude-hooks modules, 13 drift-check arms, the githooks pre-push gate and 2 Tekton checks. 81 rows. Base sha 112a522. METHOD β four transcript scanners over 5,836 .jsonl (6.8 GB), each with a negative AND a positive control, all reported in Β§2. This session's own transcript is excluded from every count. π΄ One control FAILED and the table was not trusted until it was explained. scan_drift.py's rc=0 positive control returned 0. Cause read out of the source rather than assumed: drift-check.sh:3554 guards the `(rc=$rc)` line behind `if [ "$rc" != 0 ]`, so a clean run is structurally incapable of emitting that format. Re-run against the clean-path sentence: 34 sessions. Control passes. The co-requirement in scanner 1 is doing real work, and this is the measurement that shows it: on the same string, text-only finds 97 sessions and text+`toolDenialKind` finds 75. The 22-session gap is sessions that DISCUSSED a guard without being blocked by it. WHAT THE EVIDENCE SHOWS - 67 of 81 rows carry a named catch. All 10 mutation batteries; 11 of 14 bash-guard arms; 11 of 13 drift-check arms; 28 of 31 pytest targets red at least once; 4 of 5 node suites; 8 of 8 hooks that are actually guards. - 14 read `no evidence found`, each with its search recorded, and none of them on idleness grounds. Six are modules that are not gates and emit nothing to find; two are drift-check arms structurally unable to fire yet; three are never-red targets covering guards that HAVE fired; three have the evidence in a sibling tier. - 0 rows earn a DROP. THREE FINDINGS, none of them a drop: 1. `scripts/devhost-tests` is in TARGET_FLOORS (31 entries) and NOT in HERMETIC_TARGETS (30). It runs in 4 sessions where its neighbours run in 200-590, while its floor still counts toward the derived global floor. This is already the failure mode the task warns about β a target that silently stopped running β arrived at by omission rather than by decision. Decide it explicitly, either way. 2. No mutation battery is wired into any gate. `grep -rn "mutants-"` over gate.sh, run-tests.sh, run-node-tests.sh, flake.nix and githooks/ returns nothing. Defensible as manual one-shot instruments, but currently undocumented β say so in their headers so the next reader does not assume they run. 3. The proposed tiering's saving is UNMEASURED and smaller than it looks: the always-run set is 14,053 of the 21,202 pytest floor-sum (66%), capping the benefit at ~34% and only on single-subsystem diffs. Floors are not wall-clock; measure per-target time before building the selection layer. Also lands scripts/check-gate-inventory.py, which asserts criteria 3 and 4 over the table: 81 rows checked, 0 problems. It carries its own controls (`--self-test`: four malformed row shapes each rejected, a well-formed row accepted, and a scoping control), and it errors rather than passing when it matches zero rows β a vacuous green there would certify an empty table. It is deliberately NOT wired into any gate and NOT placed under scripts/tests/, since a file there would change that target's collected count and force a TARGET_FLOORS re-pin. Follow-on filed as clawgate task #528: add emit_invocation to the gates and guard hooks recording the OUTCOME (and, for bash-guard, WHICH ARM), so this is answerable from activity.events directly next time rather than by transcript proxy. An event recording only "the gate ran" would reproduce the exact gap this document had to work around, so #528 makes the outcome field a criterion and requires the emitter to be fail-open and demonstrated so. Note for the reader: criterion 1 names `ZacxDev/devrc`; this repo's origin is `innovation-upstream/devrc`. Read as "the devrc repo" and targeted at origin. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXxTANDijZTVxj398NrWBU Claude-Session-Id: 45e79797-ec0e-45c7-ad21-5d623afb297b
Claude-Session-Id: 45e79797-ec0e-45c7-ad21-5d623afb297b
Claude-Session-Id: 45e79797-ec0e-45c7-ad21-5d623afb297b
β¦ were a SET inferred from behaviour instead of read from its registry Adversarial audit of #1375 returned 2 deploy-blocking and 9 should-fix findings. Both π΄ are retracted in the document rather than quietly patched, because the shape they share is the transferable part. π΄ RETRACTED 1 β the bash-guard arm table was wrong in BOTH directions. It was assembled from the guard's deny MESSAGES rather than from `guard_core._CLAUDE_CODE_CHECKS`, which is the list `bash-guard.py:60` (POLICY = "claude-code") actually runs. * `check_git_commit_to_main` was ABSENT from the table. Re-measured independently of the audit: 435 blocks across 270 sessions β the THIRD most-fired arm in the repo, and the one enforcing this repo's own π΄ never-commit-to-main rule. * `rm -r $HOME` was listed as a bash-guard arm. It is not one: it lives in `_IRREVERSIBLE_CHECKS`, an opencode-only policy, under a 20-line comment arguing the exclusion and ending "Do not 'finish the job'." Measured 3 blocks / 1 session β from opencode's guard, not bash-guard. (The audit reported 0/1; re-measuring found 3/1, so the attribution was wrong, not the firing.) Why it mattered: round 1's keep-without-evidence argument read "a regex on a string already being matched for eleven other arms". For `rm -r $HOME` under claude-code that is false β it is not matched at all, so its zero was a POLICY fact, not a rarity fact. An operator persuaded by that argument might have moved `check_rm_rf_critical` into the claude-code policy, reversing a deliberate and argued decision. Corrected: 12 of 14 arms have fired; the zero-firing set is two. π΄ RETRACTED 2 β row A31's finding (`scripts/devhost-tests`) was false on all three of its claims, and its proposed remedy would have redded the sandbox tier. * "arrived by omission" β `run-tests.sh:876-898` is a dedicated `DEVHOST_TARGETS` array under a π΄ comment explaining the choice. * "silently stopped running" β `test_nvim_clipboard_osc52.py` parses that array and asserts the registration exists. * "its floor is counted into the global floor regardless" β false; `run-tests.sh:900` sets `TARGETS=("${HERMETIC_TARGETS[@]}")`. Round 1's OWN gate output said `floor: 21281 = sum of 30 per-target floors` against 31 targets. The refuting evidence was in hand and was misread. The tests need `nvim`, which the nix sandbox lacks β which is why they were moved out. No action; the design is correct as it stands. RE-ANCHORED c5e425c -> 01956bf. The audit found the base had moved: #1377 retired `initiatives` and `repo-cos`, so round 1 carried rows for two DELETED targets and proposed selection rules for two retired subsystems. TARGET_FLOORS 31 -> 29, floor-sum 21,202 -> 20,214, always-run share 66.3% -> 69.5%, cap on the proposed saving ~34% -> ~30.5%. 81 rows -> 79. Other corrections, each measured: * The documented search needle was WRONG. Runners emit TWO spaces (`FAIL <target> (`); the doc said one. Round 1's regex used `\s+` so its COUNTS were right, but a reader following the appendix got 0 for nearly every row and would have read that as confirmation β the positive-control failure the document preaches about, in its own appendix. * Row C3 counted READERS, not firings. `gate.sh:250` emits `GATE: RESULT=UNVOUCHED`; it emits `could-not-vouch` nowhere β that string lives in CLAUDE.md, which carries both of round 1's needles in one sentence. Re-measured: 18 sessions, not 241. The KEEP survives; the figure overstated ~13x. A fabricated-name negative control CANNOT catch this, and that limit is now stated in Β§2. * Instrument 3 cannot separate the two tiers: both runners emit an identical `PASS <dir> (` and three dirs are both a pytest target and a node suite. Those six rows are marked AMBIGUOUS and C2's reason no longer leans on a separation the instrument cannot provide. * Β§10's always-run triggers omitted `scripts/testlib/**` β referenced by 122 tracked files across 8+ targets, so the omission was a live instance of the risk Β§11 states. Added. Dropped `pytest.ini`/`pyproject.toml`: neither exists. * "8 of 8 hooks that are actually guards" matched no partition of the rows and contradicted the next paragraph. Replaced with the full accounting: of 16 E rows, 6 are not gates, 10 are, 5 of those can block, 5 are advisory. * Two rows were contaminated by the audit itself β it ran the gate with `--set all`, putting devhost verdict lines in the corpus. The audit session is now excluded from every count alongside the authoring session; citing it would make the document evidence for itself. CHECKER β three findings, all mutation-verified after fixing: * The scoping control was VACUOUS. Its fixture had no `## 3.` line, so `in_scope` was False from the start regardless of SECTION_END; deleting the branch it claims to cover left the self-test fully green. New fixture opens the scope first. Mutation-checked: with SECTION_END deleted the control now fails with ITS OWN error (rows=2, problems=1), i.e. it is reachable. * Evidence acceptance was vocabulary, not evidence β the bare word `CONFIRMED` and a bare `ΓN` count both passed. Decisive control from the audit: replacing all 81 evidence cells with `CONFIRMED` yielded RESULT: PASS. Now requires a lookup-able identifier (UUID session id, `agent-<12+ hex>`, a >=7-hex sha in backticks, or `#<digits>`). Same control now fails all 79 rows. * The zero-row guard was narrower than its docstring: a document truncated to ONE row passed. The document now declares its own size and tally in two machine-read markers; a mismatch, or a missing marker, exits 2. π΄ And the tally marker immediately earned its place: the hand-written tally was wrong AGAIN this round (47/32 written, 49/30 counted) β as it was in round 1 (51/30 written, 49/32 counted) β while the row LISTS beside it were correct both times. It is now machine-checked rather than proofread. Negative control: feeding the wrong tally back in makes the checker go red. Gate: both tiers on the merged tree β named in the PR comment with the base sha. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXxTANDijZTVxj398NrWBU Claude-Session-Id: 45e79797-ec0e-45c7-ad21-5d623afb297b
Claude-Session-Id: 45e79797-ec0e-45c7-ad21-5d623afb297b
β¦s became 81 Re-anchored c5e425c -> 01956bf while round 2 was in flight. Not drift in the sense the appendix disclaims: two mutation batteries LANDED, so the inventory was missing rows, which is a coverage gap. D11 mutants-diagnose-disk-accounting.sh (ffac18f, #1366) β a gate for root-only bash nothing had ever run; THREE live defects found while writing it, all fixed. 19 mutants, all killed. D12 mutants-nebula-relay.sh (01956bf, #1272) β the audit found three blockers in the apply script, all downstream of rc 0. Both have a named catch, so: 12/12 batteries, 81 rows, KEEP 51 / TIER 30 / DROP 0. pytest floor-sum 20,214 -> 20,348; always-run share 69.5% -> 69.1%; cap on the proposed tiering saving ~30.9%. Both machine-read markers updated in the same commit, which is what they are for β `check-gate-inventory.py` fails on a stale marker rather than passing quietly. Verified: 81 rows checked, declared 81, tally counted 51/30/0, problems 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HXxTANDijZTVxj398NrWBU Claude-Session-Id: 45e79797-ec0e-45c7-ad21-5d623afb297b
π΄ Correcting this PR's description β round 1's headline finding is RETRACTEDPosting rather than silently editing the body, because a reviewer may already have read the The PR body's finding #1 β that
π΄ And "add it to Second retraction: the
That one mattered beyond bookkeeping: the body's keep-without-evidence argument read "a regex on Also corrected in round 2, each measured rather than reasoned:
The checker had three defects of its own, all now mutation-verified
π΄ The tally marker immediately earned its keep: the hand-written tally was wrong again this The pattern, which is the part worth keepingBoth π΄ were a set inferred from its members' behaviour instead of read from the registry that What held up was the mechanical arithmetic β all floors, both floor-sums, 12/12 battery shas, |
β¦oc but a STALE copy of its instrument (#1422) #1375 merged with the document at its round-2 content and `scripts/check-gate-inventory.py` at its round-1 content. Measured on origin/main: the doc is byte-identical to the branch tip, the checker is 166 lines against the branch's 286 and contains zero of the round-2 changes. Cause: the first `gh pr merge --squash --delete-branch` errored locally on `fatal: 'main' is already used by worktree`, but the API merge had already fired against the PR head as GitHub knew it, before the last push propagated. The local error read as "the merge did not happen"; it had, partially. The state that left on main is worse than either version alone: the document declares `<!-- inventory-rows: 81 -->` and `<!-- inventory-tally: β¦ -->`, and the checker beside it does not read either marker β so the document asserts it is machine-checked while nothing checks it. It also reinstated all three defects an audit had found in that script: * the scoping control was VACUOUS β its fixture had no `## 3.` line, so `in_scope` was False from the first line regardless of the SECTION_END branch it claimed to cover. Deleting that branch left the self-test fully green. The round-2 fixture opens the scope first; mutation-checked, the control now fails with ITS OWN error when the branch is deleted. * evidence acceptance was VOCABULARY, not evidence β the bare word `CONFIRMED` and a bare `ΓN` count both passed. Replacing all 81 evidence cells with `CONFIRMED` yielded RESULT: PASS. It now requires a lookup-able identifier (UUID session id, `agent-<12+ hex>`, a >=7-hex sha in backticks, or `#<digits>`); the same control now fails all 81 rows. * the zero-row guard was narrower than its docstring β a document truncated to ONE row passed. It now cross-checks the row count AND the verdict tally against the document's own markers, and exits 2 when either is missing. Verified on this tree (origin/main + this one file): --self-test all 9 controls pass vs the doc ON main rows checked: 81 (declared: 81) problems: 0 No document change: the doc already on main is the correct round-2 text. This is the instrument catching up to it. Claude-Session: https://claude.ai/code/session_01HXxTANDijZTVxj398NrWBU Claude-Session-Id: 45e79797-ec0e-45c7-ad21-5d623afb297b Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
β¦ERGED; #1415 round-1 fixed with Claude-Session-Id: 5542cd95-4967-4463-8fe2-0f0a75194e9d
Closes clawgate task #525.
An inventory of all 81 gates, tiers, batteries and hooks devrc runs. Every row carries
either named evidence of a real catch β a session id, a CI run, or a commit sha β or the
explicit string
no evidence foundwith the search that produced it, plus aKEEP/DROP/TIERverdict.
π΄ Nothing that runs is changed by this PR. No gate, target, battery or hook is dropped,
disabled or made conditional, and the per-path selection rule in Β§10 is proposed, not
implemented. Both are the task's explicit non-goals; the verdict column is a recommendation
the operator decides on.
How the enumeration was derived
From the repo at scan time, not from the task's own list (which the document then contradicts in
two places β see below). Commands and counts are in Β§1 so they can be re-run and disagreed with:
31 pytest targets, 5 node suites, 2 nix check derivations, 10 mutation batteries, 16
claude-hooks modules, 13 drift-check arms, the githooks pre-push gate, 2 Tekton checks.
Method, and why the numbers are worth anything
Four transcript scanners over 5,836
.jsonl(6.8 GB), each with a negative and a positivecontrol, all reported in Β§2. This session's own transcript is excluded from every count.
π΄ One control FAILED, and the table was not trusted until it was explained.
scan_drift.py'src=0positive control returned 0 hits. Cause read out of the source rather than assumed:scripts/drift-check.sh:3554guards the(rc=$rc)line behindif [ "$rc" != 0 ], so a cleanrun is structurally incapable of emitting that format β it prints
drift-check: no drift on the host(s) CHECKEDinstead. Re-run against that sentence: 34sessions. Control passes.
The
toolDenialKindco-requirement was load-bearing, and here is the measurement that showsit: on the same search string, text-only finds 97 sessions and text-plus-marker finds
75. Those 22 sessions discussed a guard without being blocked by it, and would have
inflated every hook row.
What the evidence shows
bash-guardarms;11 of 13 drift-check arms; 28 of 31 pytest targets red at least once; 4 of 5 node suites; 8 of
8 hooks that are actually guards.
no evidence found, each with its search recorded, and none on idlenessgrounds: six are modules that are not gates and emit nothing to find; two are drift-check
arms structurally unable to fire yet; three are never-red targets covering guards that have
fired; three have the evidence in a sibling tier.
The clearest case for keep-despite-no-evidence is Β§7's three zero-firing
bash-guardarms(
mkfs,ddto a block device,rm -r $HOME): their cost is a regex on a string already beingmatched for eleven other arms, and the loss they prevent is total and unrecoverable. Frequency
is the wrong axis for a guard whose downside is unbounded β which is the same reason a
run-count could never have answered this question.
Three findings, none of them a drop
scripts/devhost-testsis inTARGET_FLOORS(31 entries) and not inHERMETIC_TARGETS(30). It appears in 4 sessions where its neighbours appear in 200β590, while its floor
still counts toward the derived global floor. This is already the failure mode the task's
non-goals warn about β a target that silently stopped running β arrived at by omission rather
than by anyone choosing it. Decide it explicitly, either way.
grep -rn "mutants-" scripts/gate.sh scripts/run-tests.sh scripts/run-node-tests.sh flake.nix githooks/returns nothing. Defensible as manual one-shot instruments, but currently undocumented β worth
a line in each battery's header so the next reader does not assume they run.
always-run set is 14,053 of the 21,202 pytest floor-sum (66%), capping the benefit at
~34% and only on single-subsystem diffs. Floors are not wall-clock. Β§11 also lists two risks
worth answering before building it: a path rule is itself a guard that can fail silently, and
conditional selection interacts badly with the derived global floor.
Also in this PR
scripts/check-gate-inventory.pyasserts criteria 3 and 4 mechanically over the table β81 rows checked, 0 problems. It carries its own controls (
--self-test: four malformed rowshapes each rejected, a well-formed row accepted, plus a scoping control), and it errors rather
than passing when it matches zero rows, because a vacuous green there would certify an empty
table. Its first run reported 11 false positives against Β§10's rule table, which reuses row ids;
fixed by scoping to the inventory sections rather than by loosening the check.
It is deliberately not wired into any gate and not under
scripts/tests/β a file therewould change that target's collected count and force a
TARGET_FLOORSre-pin, which thisdocument does not need.
Follow-on
Clawgate task #528 β add
emit_invocationto the gates and the five blocking guard hooks,recording the outcome (and, for
bash-guard, which arm), so this is answerable fromactivity.eventsdirectly next time instead of by transcript proxy. An event recording only"the gate ran" would reproduce the exact gap this document had to work around, so #528 makes the
outcome field a criterion and requires the emitter to be fail-open and demonstrated so.
(#526 was already taken by the session that filed #525.)
Note on criterion 1
The task says "a PR against
ZacxDev/devrc". This repo's origin isinnovation-upstream/devrc(git remote -v). Read as "the devrc repo" and targeted at origin.Flagged rather than silently resolved.
π€ Generated with Claude Code
https://claude.ai/code/session_01HXxTANDijZTVxj398NrWBU