gc: asyncpipe collects at 1200-1650 ns/object, including a 122 ms minor that handled zero objects

[proggeramlug]

Summary

Measured while establishing the per-cycle GC picture for the retain cluster (gc-handoff/RETAIN4-NOTES.md):

asyncpipe collects at 1,200–1,650 ns/object when it collects at all — including a 122 ms minor that handled ZERO objects.

For scale, the retain cluster's in-place promotion runs at 8.9 ns/object and a real evacuation of 245,752 objects costs 30.2 ms. So asyncpipe's collections are two orders of magnitude more expensive per object, and one of them did no per-object work whatsoever yet still cost 122 ms — longer than the entire program's 127 ms runtime.

A collection whose cost is independent of the number of objects it processes is a fixed overhead being paid per cycle, not collection work. That points at root scanning, side-table maintenance, or a per-cycle setup/teardown cost that scales with something other than the live set.

Relationship to #7909

#7909 covers the incremental old-gen work asyncpipe pays while running zero cycles. This issue is different: it is about the cost of the cycles that do run, once the workload is scaled far enough to trigger them (per gc-handoff/ASYNC2-NOTES.md, 120 batches → zero cycles, 240 → two copying minors, 1200 → GC-dominated).

The two may share a root cause. They should be investigated together but neither should be closed by the other without evidence.

Acceptance

• Explain what a 122 ms minor spends its time on when it processes zero objects.
• A gate that asserts its subject was live (per CLAUDE.md's "four ways a gate can be unable to fail") — e.g. assert a nonzero cycle count and a bounded per-cycle fixed cost, not merely that nothing threw.

Note on measuring this

gc-handoff/apps/asyncpipe_big.ts is not a valid scaled proxy for asyncpipe — the zero-GC property is a knife-edge of the exact batch count. Any scaling used to trigger collections must be reported explicitly, along with confirmation that the mechanism under study survived the scaling.

[proggeramlug]

Lead from #7909: the per-cycle cost is the ROOT SCAN, and the denominator is wrong

Investigating #7909 I had to unblock asyncpipe's collector (a starved budgeted
incremental cycle was rejecting every precise safepoint via
gc_safepoint_moving_minor's budgeted entry guard). That produced two real
copying minors on the 1× program, traced with PERRY_GC_TRACE=1 PERRY_GC_DIAG=1:

	minor #1	minor #2
pause_us	61 400	134 076
copied_objects	153 056	0
copied_bytes	12 053 704	0
dominant phase	copying_nursery 55 682 µs	copying_nursery 132 791 µs
runtime_mutable_scanners.registered_scanners	82	82
runtime_mutable_scanners.slots_scanned	187 670	234 837
runtime_mutable_scanners.pointer_roots	171 068	218 455
runtime_mutable_scanners.rewritten_slots	85 404	109 091
layout_scans.pointer_slots_read	263 828	536 873

Minor #2 is a 134 ms collection that copied zero objects and zero bytes —
longer than the entire 127 ms program, with no per-object work at all. Its time
is 82 registered runtime mutable root scanners exposing 218 455 pointer
roots, all scanned and roughly half rewritten.

That is the fixed overhead this issue is looking for, and it says the
ns/object figure has the wrong denominator: the work is per registered
root, not per copied object. asyncpipe allocates ~95 600 promises and
~24 000 closures; each registry entry is reached through a registered mutable
root scanner, so the root set of a young collection is effectively the whole
heap — young collection stops being O(survivors) and becomes O(all registered
objects).

Two consequences worth checking before optimising the copy loop:

1. a minor's cost here is independent of how much it reclaims, so scavenging
   more often (a smaller nursery cap) multiplies the fixed cost with no
   corresponding win — consistent with the nursery-shrink result already
   measured on this program;
2. on asyncpipe, collecting is worse on every axis: 1927 → 2918 M
   instructions and 48.9 → 76.7 MB RSS. The scavenge nursery cap, whose purpose
   is to hold RSS down, is counterproductive on this shape.

Full trace, counters and reproduce commands: gc-handoff/GC7909-NOTES.md §4.1.
The [gc-incremental] diagnostic that made the stall visible is PR #7923.

[proggeramlug]

Both halves of the premise are counter artefacts — and the real cause is bigger

Measured on origin/main (ebefba51a), asyncpipe at 240 batches,
PERRY_GC_TRACE=1 PERRY_GC_DIAG=1. Draft PR #7931 ships the instrument this used.

1. The "minor that handled zero objects" relocated 200 201 of them

[gc-copy-minor] ran … survival_permille=895
  copied_objects=0 copied_bytes=0
  promoted_objects=200201 promoted_bytes=15860160 …

copied_objects counts survivor-space copies only, so on a promoting cycle it
is structurally zero. 101 ms / 200 201 objects = 505 ns/object — not a fixed
per-cycle cost. Same shape as CLAUDE.md's "a gate must assert its subject was
live": a counter that cannot be non-zero on the path being measured.

2. "82 scanners over 218 455 roots" is one scanner

Per-scanner attribution ([gc-scanner-profile], #7931), minor #1:

scanner	µs	slot visits	ptr roots	rewrites
crate::box::scan_box_roots_mut	18 452	186 776	186 760	93 380
scan_closure_dynamic_props_roots_mut	745	4 458	4 196	2 065
descriptor_state::scan_descriptor_roots_mut	523	342	342	171
promise_mutable_root_scanner	43	12	10	5
other 78 scanners	< 25 each

92 % of the pointer roots and 89.5 % of root-scan time is one registry. The
suspected promise registry visits 12 slots. (Counts sum both passes; #7645
already skips the third.)

3. …and that 89.5 % is the object trace, not root-scan overhead

A cheap slot visit costs ~4 ns (intern_table_mutable_root_scanner: 16 384 slots,
1 144 pointer roots, 68 µs). The box scanner costs 98.8 ns/slot because
essentially every box slot is a from-space pointer, so visiting it in
CopyingMark evacuates the object it names — 93 380 of minor #1's 156 236
evacuations come straight out of box roots, and 98.8 ns/pointer-root is the same
price as a measured full evacuation (123 ns/object).

The fixed per-root overhead this issue targets is ~4 ns × 265 000 ≈ 1 ms of a
65–101 ms pause. All three suggested directions (generational filtering, dirty
tracking, cheaper representation) share that 1 ms budget. I built the third —
slab-allocated box cells so the scan streams instead of chasing scattered
mallocs in hash order — and it measured +0.03 % instructions, −1.8 % cycles, no
RSS change, overlapping pause distributions over 7 runs each. Reverted; the
mechanism above explains why it cannot pay.

4. The real cause: 96 % of the young generation is retained by dead activations

One-env-var probe (temporary), scan_box_roots_mut skips marking but still
rewrites:

	control	box roots do not mark
minor #1 young_survival_permille	695	24
minor #1 copied_objects	156 236	6 634 (−95.8 %)
minor #1 freed_bytes	5 456 696	17 451 568 (3.2×)
minor #2	200 201 promoted, 15.9 MB	never happens
exit	0, correct output	134 (SIGABRT)

asyncpipe's real live young set is ~6 600 objects. Everything else is alive
only because a completed async activation's boxed locals are still GC roots —
the async-to-generator transform boxes every body local of every async
function, and nothing ever frees or clears one, so every local of all 48 000
completed activations is a permanent root. At 6 634 live objects a minor costs
~0.8 ms instead of 62 ms.

For context, asyncpipe is a churn program by construction (each batch is built,
folded into six numbers and dropped) yet survives at 695–895 ‰, against 0–4 ‰ for
every other churn program in gc/promote_in_place.rs's own census.

The SIGABRT is the control, not a failure: not marking is unsound, which is why
the fix is clearing an activation's boxes at its terminal state, not skipping
them. Clearing keeps BOX_REGISTRY monotonic and the address valid — so
is_registered_box_ptr's perry#4898 rejection and the positive pointer cache are
untouched — while removing the retention. It is a codegen change in the async
transform and deserves its own issue.

Full write-up with reproduction: gc-handoff/ROOTS-NOTES.md §2–8.

Suggested disposition

Keep this issue open only if it is re-scoped to the ~1 ms of genuine root-scan
overhead; the 60× lever is the activation-local retention above.