[roodboi]

Summary

Fix unit G30: native e2e and storage budgets (HACK-1324, HACK-1320, HACK-1321).

HACK-1324: storage preparation gets its own named budget

Root cause, confirmed on three hosted first attempts (#365 twice, #370):

• Every storage carrier took its absolute deadline from the start budget
  (Date.now() + resolveComposeStartupTimeoutMs() in
  src/lib/native-compose-command-storage.ts). native-config-down-hooks runs
  with HACK_COMPOSE_STARTUP_TIMEOUT_MS=15000, so its first storage carrier had
  15 s for the dependency check, target observation and helper
  create/policy/start. On loaded Blacksmith runners preparation alone takes
  longer, and the scenario failed at 18.3-18.5 s with storage-target,
  storage-helper-*, probe-timeout/probe-cancel/unclassified.
• In native-config-build, about a dozen full storage proofs run in the build
  and pre-spawn guards (fresh() in buildNativeComposeImages, then
  runNativeComposeOwnedFileChild). They consumed the 90 s start deadline, so
  the spawn admission refused (compose-child / private-uncertain, run
  39676161586) or the outcome was E_STARTUP_INCOMPLETE (run 39663510147).

Changes:

• HACK_NATIVE_STORAGE_TIMEOUT_MS (resolveNativeStorageTimeoutMs, default
  90 s, range 1,000-600,000 ms) bounds each storage carrier. It is never derived
  from the start budget. Default behavior on an unset host is unchanged (the old
  start default was also 90 s).
• createNativeComposeStartDeadline: guard storage proofs during a native
  up/restart are excluded from the start budget; their monotonic elapsed time
  extends the start deadline. Overlaps count once, rejected proofs still count,
  the deadline never moves earlier, and each excluded proof stays bounded by its
  own carrier budget. Build and spawn admission read the live deadline.
• Whole-phase cap on excluded storage time (7daca0d): the total excluded from one
  up/restart start deadline may not exceed 16 x HACK_NATIVE_STORAGE_TIMEOUT_MS
  or 1 hour, whichever is smaller (24 min at the 90 s default; 1 h from S = 225 s
  up). The deadline moves at most that far; a guard proof that would start past
  the cap is refused before it runs; one that ends past it refuses the phase with
  a closed, value-free NativeStorageTotalBudgetError (effect reason
  storage-budget, stage guard-storage). The phase ends within the start
  budget + the cap + at most one in-flight guard proof.
• Closed effect reason storage-budget: a refusal that finds its carrier's own
  deadline passed (not cancelled) names the budget; the human summary names
  HACK_NATIVE_STORAGE_TIMEOUT_MS.
• Scenario budgets: build hack commands 120 s -> 300 s, down-hooks 90 s -> 240 s.
  Both scenarios now log per-command duration plus closed storage phase timings
  from the passive phase trace, so hosted runs record measured preparation
  durations.

No check, ownership fence, retention or recovery rule changed; per-probe caps
(10-15 s) are unchanged.

HACK-1320: networks whole-run budget

• Named NATIVE_NETWORK_FIXTURE_SCENARIO_TIMEOUT = 90 min. 44 measured hosted
  passes took 2113-3494 s (median ~2650 s); this PR's own run hit 3494 s, 97% of
  the old 60-minute budget. 90 min keeps 1.55x the slowest pass; with cleanup and
  two command bounds the hard bound stays inside the 150-minute shard job.
• Whole-run and cleanup budgets now gate only the start of a command; a started
  command keeps its full 180 s. Run 39431629301 failed because the remaining
  whole-run budget clipped and SIGKILLed an in-flight down carrier command,
  leaving a created helper that saved down --recover correctly refuses to
  repair.

HACK-1321: M6 and monotonic group absence

• The unrecorded-command group-absence deadline is now monotonic
  (performance.now), named UNRECORDED_GROUP_BUDGET_MS (3 s).
• New fake-clock test pins the probe-budget refusal on the no-record path.
• Carrier doc lists the fixed 10-15 s probe caps.

Release note

• Storage carriers no longer follow HACK_COMPOSE_STARTUP_TIMEOUT_MS in either
  direction (a short start budget no longer shortens them, a long one no longer
  lengthens them). Hosts that set HACK_COMPOSE_STARTUP_TIMEOUT_MS should also
  review HACK_NATIVE_STORAGE_TIMEOUT_MS, which now sets both the per-carrier
  bound and the whole-phase storage cap.

Evidence

Rework at 7daca0d (verifier gaps)

• Wiring test: tests/native-compose-start-storage-exclusion.test.ts runs the
  real source CLI up with a 3 s start budget. The Docker stand-in reads the
  CLI's own phase trace (stderr written to a file, so it is on disk before the
  carrier spawns Docker) and delays the carrier dependency check (image inspect) by 4 s only while a guard.storage span is open. No call counting.
  About 43 s on a loaded Mac (the earlier prototype took about 80 s). Mutation:
  with start.excludeStorage(...) removed around the guard's storage.verify,
  the test fails (up exits 1, the workload is never admitted); restored, it
  passes.
• Cap tests in tests/compose-startup-budget.test.ts (fake clocks). Each of four
  mutations fails them: no post-proof check, no entry check, no min on the
  deadline extension, and the cap error not classified as storage-budget.
• Docs: docs/cli.md, docs/reference/native-compose-commands.md and
  docs/reference/native-compose-storage-xattr-carrier.md describe the per-proof
  bound, the phase cap and how the two variables interact.
• Gates: CLI and root typecheck, root bun run check (privacy-check ok), biome
  on the changed files (only an existing complexity warning in
  native-compose-command.ts), gitleaks 8.30.1 over origin/next..HEAD (no leaks).
• M3 full suite at 7daca0d: 7721 pass, 126 skip, 1 fail
  (repository catalog > the checked-in .hack/tasks.json recipes are all reviewed). It passes locally at the same head and in hosted test, and this
  PR does not touch .hack/ or the task catalog. I could not rerun it on M3 next
  because SSH to M3 failed (local agent signing error).
• Hosted CI at 7daca0d: all checks pass, including all four docker-e2e shards.

Measured storage durations (hosted shard 4, this branch, 9 attempts)

From the new per-command phase-trace lines:

Measure	Range across 9 attempts
Longest single storage proof	3.6-11.7 s
Cold enrollment (storage.enroll, first up)	11.7-35.0 s
native-config-build first up	50-176 s, of which 16 proofs took up to 145 s

Old behavior on the slowest attempt: the 145 s of guard proofs alone exceed the
90 s start budget, and the 35 s cold enrollment exceeds the 15 s budget the
down-hooks scenario gave every carrier. New budgets: 90 s per carrier (2.6x the
slowest observed carrier work, 7.7x the slowest proof); build hack commands
300 s (1.7x the slowest up); down-hooks hack commands 240 s.

Hosted CI on this branch (11 attempts across 0af0fb6, 94ce188, cee65eb, 4e2b056)

Scenario	Passes	Durations
native-config-build	11/11	116.7-336.9 s
native-config-down-hooks	11/11	84.9-181.4 s
native-config-networks	11/11	1766-3494 s

The merged head 4e2b056 (next a25160d merged in) passed both of its full
runs (run 39721132414 attempts 1 and 2). Before this change, shard 4 failed on
first attempts of #365, #370, #374 and on next itself (39663080763,
39670689406).

Other attempt failures, all unrelated to these files:
tests/native-authored-project-start.test.ts frontend cascade (also seen on 6+
other branches and next), tests/private-env-file.test.ts symlinked .env
ENOENT, and one runtime-core Rust test
(rebind::tests::healthy_fixed_selection_is_noop_and_complete_shared_slot_refreshes_together).

Gate vs PR shard-4 comparison

Same runner pool and labels (blacksmith-4vcpu-ubuntu-2404, runner 2.337.0),
identical shard composition and order, no leftover volumes before
native-config-build (end-of-job cleanup volumes are the failed scenarios'
own retained fixtures). The failing jobs (including next push run
39663080763) ran unrelated earlier scenarios 1.3-1.9x slower, so the
difference is runner speed, not composition or state. The logs print no
docker info storage-driver lines, so that dimension could not be compared.

Mutation proofs (each killed by a focused test)

M6 (groupAbsent deadline -> plain refuse()), M6b (wall-clock deadline),
carrier deadline from the start budget, no start-deadline exclusion,
nested exclusion double-count, build deadline read once, carrier reason ignores
budget, budget reported over cancellation, xattr lifetime unmarked, effect
reason ignores mark, summary not naming the variable, networks command clipped
to the whole-run remainder.

Local and M3 gates

• bun run typecheck and bun run check: 0 errors at 4e2b056.
• gitleaks 8.30.1 over origin/next..HEAD: no leaks.
• M3 full suite at 0af0fb6: 7587 pass, 126 skip, 0 fail.
• M3 full suite at 4e2b056 (host load 25-39): 7681 pass, 126 skip, 12 fail.
  10 of the 12 pass when their files rerun serially; the other 2
  (tests/native-authored-project-recovery.test.ts no-host death/recovery)
  fail the same way on plain next a25160d. None touch the changed code.
• M3 is not a qualifying host for the native e2e scenarios; hosted CI is the
  e2e proof.

Gaps

• The whole-phase cap is proven at unit level only. A command-level cap test
  cannot isolate it, because at S = 1 s the per-carrier budget fires first.
• An in-flight guard proof is not aborted when the cap is reached. It finishes
  within its own per-carrier bound, then the phase refuses.
• The networks scenario still SIGKILLs a command that exceeds its own 180 s
  bound; that genuine hang still leaves storage evidence for explicit recovery.

🤖 Generated with Claude Code