Repository navigation
xtask: only the binary judges the selftest work-unit budget (fixes the the_known_spellings_still_parse flake) - #786
Merged
Merged
Conversation
…ests::the_known_spellings_still_parse never reads it under a shared-core test run `gate --jobs 2 segregation --selftest` reached run_selftest through cli::main in-process, and verify_selftest judged the work-unit budget there: under a full-workspace nextest run the ruler read 2.5ms a unit and segregation came in at 10744 units serially against 9448, so an argument-parsing test went red (exit 1) on a timing it never asked about (ruling (a)). The budget now switches on in xtask's main beside the process watchdog, and a test process judges verdict and coverage only. The RED arm of the budget stays pinned with an explicit budget.
promote into
|
| gate | row | detail |
|---|---|---|
| construction | one-pick-site |
3 production call site(s) of 'pick_among(' (ceiling 2): crates/busbar-kernel-egress/src/walk.rs:298; crates/busbar-llm/src/engine/exhaustion/fallback.rs:120; crates/busbar-llm/src/engine/pipeline.rs:8 |
| kind-isolation | kind-isolation:deps |
3 finding(s), 93 shipped edge instance(s) over 30 class(es), 93 declaration(s); 56 '[[dep]]' row(s), 0 question(s): unlisted-dep-edge legacy -> plane busbar-llm -> busbar-plane-llm is a shipped edge w |
| kind-isolation | kind-isolation:law0 |
7 hit(s) off the [[law0]] ceilings, 4 finding(s) over 14 neutral crate(s): law0-rise busbar × instance:secret 37 hit(s) against a ceiling of 35: this landing grew a neutral crate's instance vocabulary |
| kind-isolation-ship | kind-isolation:deps |
10 finding(s) over 93 shipped edge(s): ship-edge kernel -> hooks busbar-kernel -> busbar-hook-ranking is 'not-allowed': the architecture grants no kernel -> hooks edge, and the ship criterion is the a |
| kind-isolation-ship | kind-isolation:test-deps |
8 finding(s) over 33 test edge(s): ship-edge cleanliness -> export busbar-core-admin -> busbar-export-prometheus is 'not-allowed': the architecture grants no cleanliness -> export edge, and the ship c |
| kind-isolation-ship | kind-isolation:law0 |
7 hit(s) off the [[law0]] ceilings, 4 finding(s) over 14 neutral crate(s): law0-rise busbar × instance:secret 37 hit(s) against a ceiling of 35: this landing grew a neutral crate's instance vocabulary |
| kind-isolation-ship | kind-isolation:faces |
2 finding(s) over 40 crate(s): foreign-entry crates/busbar busbar is kind 'root' and implements 'Transport' 1 time(s) in shipped source — the entry face of kind 'transport'. A trait implementation is |
| kind-isolation-ship | kind-isolation:legacy-drain |
3 finding(s): transitional-live qa/kind-isolation.toml 'busbar-a2a -> busbar-core-admin' (legacy drain: the retiring A2A engine's tests drive the admin surface that drained into the cleanliness tier) |
| ship-ready | ship-ready:ship-twin |
'kind-isolation-ship' is not green: kind-isolation:deps (a dependency the architecture does not grant is still in the graph); kind-isolation:test-deps (a dependency the architecture does not grant is |
| structure-lint | structure-lint:plane-dup:unledgered |
22 finding(s): PLANE-DUPLICATE (module): 'config.rs' — a2a:crates/busbar-a2a/src/a2a/config.rs decisions:crates/busbar-plane-decisions/src/config.rs voice:crates/busbar-voice/src/config.rs (the ledger |
tests passed: 0, failed: 0. Run: https://github.com/GetBusbar/busbar/actions/runs/38239093691 . Artifact verdict-ecf0de440e0713f30351390c0200dde010e05881 (failures.json, junit.xml, raw.log; 90 days).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the flake in
xtask/src/cli.rscli::tests::the_known_spellings_still_parse, which went red on #784, #676, #703 and on predev during full-workspace nextest runs.Root cause
The test calls
gate --jobs 2 segregation --selftestthroughcli::mainin-process. That path goesrun_selftest->gates::verify_selftest, andverify_selftestalso checked the work-unit budget (wall clock divided by a CPU ruler) againstsegregation's 9 448 units. The segregation battery spends its time on I/O and starting subprocesses (cargo metadata, tree walks). The ruler is a CPU-only hash. When the cores are shared with a whole workspace's tests and builds, the battery slows down much more than the ruler does. It goes over budget at--jobs 2, gets re-run serially, is still over, and the test exits 1 instead of 0. The CI log for #784 shows exactly this:contended 11910 → serial 10744 work units (budget 9448), 36 s. Ruling (a) says a test must not judge the budget (Law 8), but the dispatcher judged it for every in-process caller. The same cause also reddenstests/cli.rs::selftest_proves_a_named_gates_red_proof(gate segregation --selftest,selftest segregation).Fix
The budget is now judged only by the binary, the same way the process watchdog already works.
xtask'smaincallsgates::enable_selftest_budget()next toenable_process_watchdog().verify_selftestreads the budget only when that switch is on. In a test process it checks verdict and coverage only (verify_report).cargo xtask selftest <gate>andgate <gate> --selfteststill judge the budget, including the serial re-run, exactly as before. The test's assertions are unchanged.A new test,
gates::budget_retake_tests::the_dispatcher_judges_the_budget_only_in_the_binary, checks three things: the budget is off in a test process;verify_selftest_within(.., None)is verdict and coverage only; and the RED arm, whereverify_selftest_within(.., Some(0.0))still refuses an over-budget battery.Proof (M3 prover VM, 8 cores)
Load: two looped full
cargo nextest run -p xtasksuites, plus a looped cleancargo check --workspace -j 8. The target test ran 30 times in the foreground.selftest_proves_a_named_gates_red_proof1f5571d216contended X → serial Y work units (budget 9448))cargo fmt --all --checkandcargo clippy -p xtask --all-targets -- -D warningsare clean.