gc-stress matrix (4/4) has been red on main since roughly 2026-09-03. It is a
deterministic failure — byte-identical counters across every run and every SHA I
checked — and the arm split points at a full-collection root-scanning bug, i.e. a
potential collected-live-object class of defect.
The failure
run [8/20] test=test_gap_repsel_gc_stress env=_base arm=gen_gc_off
FAIL test_gap_repsel_gc_stress (output-mismatch cycles=43 evacuated=0 scavenged=0 reclaimed=1327926376)
run [8/20] test=test_gap_repsel_gc_stress env=PERRY_WRITE_BARRIERS_0 arm=wb_off
FAIL test_gap_repsel_gc_stress (output-mismatch cycles=43 evacuated=0 scavenged=0 reclaimed=1327926376)
run [8/20] ... arm=gen_off_verify -> FAIL (same signature)
run [8/20] ... arm=wb_off_force -> FAIL (same signature)
run [8/20] ... arm=all_four -> FAIL (same signature)
repsel_gc_stress PASS PASS PASS PASS PASS PASS FAIL FAIL FAIL FAIL FAIL PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS PASS
summary: PASS=360 UNVER=75 XFAIL=0 FAIL=5
Example run: https://github.com/PerryTS/perry/actions/runs/33829521137 (job 100904513692).
Identical on later runs, e.g. https://github.com/PerryTS/perry/actions/runs/33926006467.
The discriminating evidence: which arms fail
Exactly five arms fail — gen_gc_off, wb_off, gen_off_verify, wb_off_force,
all_four — and those are precisely the arms that disable generational GC or write
barriers, forcing a full / mark-sweep collection instead of a generational minor. All
fifteen other arms pass, including default, evac, force-evacuate and every
representation-selection off-switch.
evacuated=0 scavenged=0 with a large reclaimed=1327926376 is consistent with a
non-moving full sweep reclaiming substantially more than it used to.
Window
| SHA |
gc-stress matrix (4/4) |
run |
dae0ae34c8cf |
green |
33796583016 (2026-09-03 19:27) |
d239929b4c46 |
red, this signature |
33814875005 (2026-09-03 22:49) |
Every push between those two had its CI run cancelled (superseded) before any job started,
so there is no direct evidence for the intermediate SHAs. That leaves a 17-commit range,
dae0ae34c8cf..d239929b4c46.
Prime suspect — correlation only, NOT a proven bisection
aa8d2ade0 — "fix(gc): stop two passes retaining unreachable objects (#9628, #9629)"
(2026-09-03 22:01) is the only commit in the range that touches GC-cycle root-scanning
semantics. In crates/perry-runtime/src/gc/cycle.rs it makes the remembered set a root
source only for minor collections:
let mark_from_set = self.minor_only; // #9629: only a MINOR needs old->young
// edges as roots; a full trace reaches
// them through the live old objects
// themselves...
So on any non-minor (full) collection the remembered set is no longer marked as roots —
which is exactly the code path the five failing arms force. test_gap_repsel_gc_stress is
purpose-built to hold representation-selected locals (Ptr<Shape>-proven objects and
friends) live across escaping churn, which is the shape that would break if such an object's
old-gen reachability depended on the remembered-set path now skipped on full traces.
The very next commit, 51bfe741d ("test(gc): the remembered-set slicing tests drive a
minor, not a full cycle"), rewrote two unit tests to match the new behaviour — so the
reduced retention was intended, but this stress fixture suggests an edge case those unit
tests do not cover.
Stated plainly: this is a code-review correlation, not a bisection. No CI run exists at
aa8d2ade0 itself, so it is not isolated from its 16 neighbours in the range. I did not run
the fixture at aa8d2ade0 and its parent because scripts/gc_repsel_matrix.sh requires a
full compiler build (cargo build --profile <p> of perry plus the runtime/stdlib
staticlibs) before it can compile and run the corpus — two of those plus two matrix runs is
well beyond a cheap check.
Correction: an earlier triage of mine attributed this to the #9644 idle-compaction
commits (3812cc973, 42cd82fdf). That was wrong — both land after d239929b4c46,
which was already red. Please do not chase them.
Paths
- Fixture:
test-files/test_gap_repsel_gc_stress.ts
- Harness:
scripts/gc_repsel_matrix.sh (computes the cycles/evacuated/scavenged/reclaimed line and the arm matrix)
- Tier driver:
scripts/release_sweep_tiers/tier04_gc_stress.sh
- Suspect code:
crates/perry-runtime/src/gc/cycle.rs
- Suspect's own tests:
crates/perry-runtime/src/gc/tests/retention_9628_9629.rs
Suggested next step
Build at aa8d2ade0 and at its parent 658007137, run
scripts/gc_repsel_matrix.sh restricted to test_gap_repsel_gc_stress and the
gen_gc_off arm, and compare. That converts the correlation into a bisection in two
builds. If it reproduces, the question is whether a full trace can in fact always reach
old->young targets without the remembered set when the young half was already swept.
https://claude.ai/code/session_01YPfnmWZmSpSWpmnoXvH8z2
gc-stress matrix (4/4)has been red onmainsince roughly 2026-09-03. It is adeterministic failure — byte-identical counters across every run and every SHA I
checked — and the arm split points at a full-collection root-scanning bug, i.e. a
potential collected-live-object class of defect.
The failure
Example run: https://github.com/PerryTS/perry/actions/runs/33829521137 (job 100904513692).
Identical on later runs, e.g. https://github.com/PerryTS/perry/actions/runs/33926006467.
The discriminating evidence: which arms fail
Exactly five arms fail —
gen_gc_off,wb_off,gen_off_verify,wb_off_force,all_four— and those are precisely the arms that disable generational GC or writebarriers, forcing a full / mark-sweep collection instead of a generational minor. All
fifteen other arms pass, including default, evac, force-evacuate and every
representation-selection off-switch.
evacuated=0 scavenged=0with a largereclaimed=1327926376is consistent with anon-moving full sweep reclaiming substantially more than it used to.
Window
gc-stress matrix (4/4)dae0ae34c8cfd239929b4c46Every push between those two had its CI run cancelled (superseded) before any job started,
so there is no direct evidence for the intermediate SHAs. That leaves a 17-commit range,
dae0ae34c8cf..d239929b4c46.Prime suspect — correlation only, NOT a proven bisection
aa8d2ade0— "fix(gc): stop two passes retaining unreachable objects (#9628, #9629)"(2026-09-03 22:01) is the only commit in the range that touches GC-cycle root-scanning
semantics. In
crates/perry-runtime/src/gc/cycle.rsit makes the remembered set a rootsource only for minor collections:
So on any non-minor (full) collection the remembered set is no longer marked as roots —
which is exactly the code path the five failing arms force.
test_gap_repsel_gc_stressispurpose-built to hold representation-selected locals (
Ptr<Shape>-proven objects andfriends) live across escaping churn, which is the shape that would break if such an object's
old-gen reachability depended on the remembered-set path now skipped on full traces.
The very next commit,
51bfe741d("test(gc): the remembered-set slicing tests drive aminor, not a full cycle"), rewrote two unit tests to match the new behaviour — so the
reduced retention was intended, but this stress fixture suggests an edge case those unit
tests do not cover.
Stated plainly: this is a code-review correlation, not a bisection. No CI run exists at
aa8d2ade0itself, so it is not isolated from its 16 neighbours in the range. I did not runthe fixture at
aa8d2ade0and its parent becausescripts/gc_repsel_matrix.shrequires afull compiler build (
cargo build --profile <p>ofperryplus the runtime/stdlibstaticlibs) before it can compile and run the corpus — two of those plus two matrix runs is
well beyond a cheap check.
Correction: an earlier triage of mine attributed this to the #9644 idle-compaction
commits (
3812cc973,42cd82fdf). That was wrong — both land afterd239929b4c46,which was already red. Please do not chase them.
Paths
test-files/test_gap_repsel_gc_stress.tsscripts/gc_repsel_matrix.sh(computes the cycles/evacuated/scavenged/reclaimed line and the arm matrix)scripts/release_sweep_tiers/tier04_gc_stress.shcrates/perry-runtime/src/gc/cycle.rscrates/perry-runtime/src/gc/tests/retention_9628_9629.rsSuggested next step
Build at
aa8d2ade0and at its parent658007137, runscripts/gc_repsel_matrix.shrestricted totest_gap_repsel_gc_stressand thegen_gc_offarm, and compare. That converts the correlation into a bisection in twobuilds. If it reproduces, the question is whether a full trace can in fact always reach
old->young targets without the remembered set when the young half was already swept.
https://claude.ai/code/session_01YPfnmWZmSpSWpmnoXvH8z2