Repository navigation
Make local check-structure runs as reliable as CI (swarm 0.16.1, work-system 1.29.2) - #96
Merged
Merged
Conversation
gering
force-pushed
the
task/stabilize-local-check-structure
branch
3 times, most recently
from
October 31, 2026 11:14
b6cd3a1 to
c5ebfaa
Compare
- test_codex_models: separate the deadline under test from the time a working peer gets. Only deadline cases run with a short bound; success and invalid-response cases get the helper's maximum and must not end by "timed out". 12 parallel runs: 0/12 green before, 12/12 after. - test_kimi_acp: write the fake kimi once per class instead of once per helper run. macOS assesses every new executable on first exec (~150 ms, serialized), which was 80% of the test's wall time: 67 s -> 13 s alone, 419-590 s under parallel load before. - test_kimi_acp: the fake waits up to 10 s, not 1 s, for the client's kill before it carries on to end_turn, so a slow kill under load cannot pass for a client that waited. - test_grok_compat: link one shared fake grok into each Env (21 s -> 13 s). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WFDBrBNHxBX8MUG6oAC8n6
- sessions.py: SESSIONS_POLL_MS test seam for the pause between transcript looks while a reload is verified (default stays 1 s). test_sessions sets 100 ms: 70 s -> 33 s. - test_agent_registry, test_herdr_shell: write each distinct stub once and link it into every Env. macOS assesses every new executable on its first exec (~150 ms, serialized across processes); ~190 fresh stubs were most of agent_registry's wall time (35 s -> 21 s alone, 444 s under parallel load before). herdr_shell: 53 s -> 30 s. - test_insights_handoff: the temp-file leak check gets its own TMPDIR; in the shared one a parallel run's live files read as leftovers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WFDBrBNHxBX8MUG6oAC8n6
The 120 s per-test bound is a hang guard, but on a busy machine a long test crossed it while CI stayed green. TEST_TIMEOUTS gives each measured long test its own bound and keeps the default tight; an entry naming a test that no longer exists is an error, so the table cannot rot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WFDBrBNHxBX8MUG6oAC8n6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WFDBrBNHxBX8MUG6oAC8n6
gering
force-pushed
the
task/stabilize-local-check-structure
branch
from
October 31, 2026 11:30
c5ebfaa to
25ae1ae
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Local
python3 scripts/check-structure.pywent red while CI stayed green: per-test timeouts (F7) andtest_codex_models/test_kimi_acpfailures under load (F8).What the measurements showed
Per-test wall time measured alone, with 3 full runs in parallel, and with 12 parallel runs of a single test:
Envspent most of their time waiting on it:test_kimi_acptook 67 s wall for 12 s CPU, and 419–590 s in parallel.test_agent_registrytook 444 s in parallel. Linux CI has no such check. The mechanism is confirmed by research; the pile-up under load is inferred from the measurements.test_codex_modelsgave the helper deadlines of 0.4–2 s, and a working fake peer had to start inside them. It failed in 12 of 12 parallel runs.test_sessionswaited a fixed 1 s per pane poll insessions.py.test_insights_handofflooked for leftover temp files in the shared$TMPDIRand saw files from parallel runs.Changes
SESSIONS_POLL_MStest seam; content-addressed stub links intest_agent_registryandtest_herdr_shell; a privateTMPDIRfor the leak check.check-structure.py: aTEST_TIMEOUTStable. Onlytest_mandate(75 s) gets its own 300 s; the 120 s default stays. A stale entry is an error.deployment/ci-structure-checks.md.Proof
test_codex_models, 12 runs in parallel: 0/12 → 12/12 green.check-structure.pyruns, all OK.Review
One local
/swarm:reviewround (Claude lenses, gpt-6.1-sol, grok-4.7; 15 of 15 voices) found 9 issues. 6 are fixed in this PR:HANG_BOUNDis 8 s, below the helper's 10 s default, so it still catches a helper that ignores--timeout;SESSIONS_POLL_MScan only shorten the poll, never lengthen it;pluginVersionstamp is updated.3 were not acted on:
check-structure.pypasses on the fixed head.Not done
test_kimi_acp1 s kill race never reproduced in 12 parallel runs. The 10 s bound comes from reading the code and costs nothing on the passing path.test_herdr_launchis out of scope (its own lane).🤖 Generated with Claude Code
https://claude.ai/code/session_01WFDBrBNHxBX8MUG6oAC8n6