Skip to content

xtask: only the binary judges the selftest work-unit budget (fixes the the_known_spellings_still_parse flake) - #786

Merged
MattJackson merged 1 commit into
predevfrom
p6-flakes-10
Oct 12, 2026
Merged

MattJackson merged 1 commit into
predevfrom
p6-flakes-10

Conversation

@MattJackson

Copy link
Copy Markdown
Collaborator

Fixes the flake in xtask/src/cli.rs cli::tests::the_known_spellings_still_parse, which went red on #784, #676, #703 and on predev during full-workspace nextest runs.

Root cause

The test calls gate --jobs 2 segregation --selftest through cli::main in-process. That path goes run_selftest -> gates::verify_selftest, and verify_selftest also checked the work-unit budget (wall clock divided by a CPU ruler) against segregation's 9 448 units. The segregation battery spends its time on I/O and starting subprocesses (cargo metadata, tree walks). The ruler is a CPU-only hash. When the cores are shared with a whole workspace's tests and builds, the battery slows down much more than the ruler does. It goes over budget at --jobs 2, gets re-run serially, is still over, and the test exits 1 instead of 0. The CI log for #784 shows exactly this: contended 11910 → serial 10744 work units (budget 9448), 36 s. Ruling (a) says a test must not judge the budget (Law 8), but the dispatcher judged it for every in-process caller. The same cause also reddens tests/cli.rs::selftest_proves_a_named_gates_red_proof (gate segregation --selftest, selftest segregation).

Fix

The budget is now judged only by the binary, the same way the process watchdog already works. xtask's main calls gates::enable_selftest_budget() next to enable_process_watchdog(). verify_selftest reads the budget only when that switch is on. In a test process it checks verdict and coverage only (verify_report). cargo xtask selftest <gate> and gate <gate> --selftest still judge the budget, including the serial re-run, exactly as before. The test's assertions are unchanged.

A new test, gates::budget_retake_tests::the_dispatcher_judges_the_budget_only_in_the_binary, checks three things: the budget is off in a test process; verify_selftest_within(.., None) is verdict and coverage only; and the RED arm, where verify_selftest_within(.., Some(0.0)) still refuses an over-budget battery.

Proof (M3 prover VM, 8 cores)

Load: two looped full cargo nextest run -p xtask suites, plus a looped clean cargo check --workspace -j 8. The target test ran 30 times in the foreground.

arm target test load-suite sibling selftest_proves_a_named_gates_red_proof
base 1f5571d216 15 / 30 FAIL (all ~34–52 s, contended X → serial Y work units (budget 9448)) red in most of the 30 load passes
this branch 0 / 30 FAIL (readings up to 39 037 units, not judged) 0 failures in 26 load passes (923/923 each)

cargo fmt --all --check and cargo clippy -p xtask --all-targets -- -D warnings are clean.

…ests::the_known_spellings_still_parse never reads it under a shared-core test run

`gate --jobs 2 segregation --selftest` reached run_selftest through cli::main in-process, and
verify_selftest judged the work-unit budget there: under a full-workspace nextest run the ruler
read 2.5ms a unit and segregation came in at 10744 units serially against 9448, so an
argument-parsing test went red (exit 1) on a timing it never asked about (ruling (a)). The
budget now switches on in xtask's main beside the process watchdog, and a test process judges
verdict and coverage only. The RED arm of the budget stays pinned with an explicit budget.
@github-actions

Copy link
Copy Markdown

promote into predev: all green @ecf0de440

Already red on predev, not this PR's: 10
gate row detail
construction one-pick-site 3 production call site(s) of 'pick_among(' (ceiling 2): crates/busbar-kernel-egress/src/walk.rs:298; crates/busbar-llm/src/engine/exhaustion/fallback.rs:120; crates/busbar-llm/src/engine/pipeline.rs:8
kind-isolation kind-isolation:deps 3 finding(s), 93 shipped edge instance(s) over 30 class(es), 93 declaration(s); 56 '[[dep]]' row(s), 0 question(s): unlisted-dep-edge legacy -> plane busbar-llm -> busbar-plane-llm is a shipped edge w
kind-isolation kind-isolation:law0 7 hit(s) off the [[law0]] ceilings, 4 finding(s) over 14 neutral crate(s): law0-rise busbar × instance:secret 37 hit(s) against a ceiling of 35: this landing grew a neutral crate's instance vocabulary
kind-isolation-ship kind-isolation:deps 10 finding(s) over 93 shipped edge(s): ship-edge kernel -> hooks busbar-kernel -> busbar-hook-ranking is 'not-allowed': the architecture grants no kernel -> hooks edge, and the ship criterion is the a
kind-isolation-ship kind-isolation:test-deps 8 finding(s) over 33 test edge(s): ship-edge cleanliness -> export busbar-core-admin -> busbar-export-prometheus is 'not-allowed': the architecture grants no cleanliness -> export edge, and the ship c
kind-isolation-ship kind-isolation:law0 7 hit(s) off the [[law0]] ceilings, 4 finding(s) over 14 neutral crate(s): law0-rise busbar × instance:secret 37 hit(s) against a ceiling of 35: this landing grew a neutral crate's instance vocabulary
kind-isolation-ship kind-isolation:faces 2 finding(s) over 40 crate(s): foreign-entry crates/busbar busbar is kind 'root' and implements 'Transport' 1 time(s) in shipped source — the entry face of kind 'transport'. A trait implementation is
kind-isolation-ship kind-isolation:legacy-drain 3 finding(s): transitional-live qa/kind-isolation.toml 'busbar-a2a -> busbar-core-admin' (legacy drain: the retiring A2A engine's tests drive the admin surface that drained into the cleanliness tier)
ship-ready ship-ready:ship-twin 'kind-isolation-ship' is not green: kind-isolation:deps (a dependency the architecture does not grant is still in the graph); kind-isolation:test-deps (a dependency the architecture does not grant is
structure-lint structure-lint:plane-dup:unledgered 22 finding(s): PLANE-DUPLICATE (module): 'config.rs' — a2a:crates/busbar-a2a/src/a2a/config.rs decisions:crates/busbar-plane-decisions/src/config.rs voice:crates/busbar-voice/src/config.rs (the ledger

tests passed: 0, failed: 0. Run: https://github.com/GetBusbar/busbar/actions/runs/38239093691 . Artifact verdict-ecf0de440e0713f30351390c0200dde010e05881 (failures.json, junit.xml, raw.log; 90 days).

@MattJackson
MattJackson added this pull request to the merge queue Oct 12, 2026
Merged via the queue into predev with commit 740a8f5 Oct 12, 2026
11 checks passed
@MattJackson
MattJackson deleted the p6-flakes-10 branch October 12, 2026 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant