Account for grouped checkpoint, head, and enclosing forward buffers by bradhilton · Pull Request #898 · OpenPipe/ART

bradhilton · 2026-09-15T08:07:56Z

Account for identified grouped checkpoint, head and enclosing-forward buffers in memory admission. Carry the shape-aware demand through plan selection, splitting, recovery and diagnostics; retain empirical floors and mark replay incomplete when immutable runtime facts are unavailable.

Merged with Brad’s explicit approval as a655c051, after #888 landed as e2e8969f. An ancestry-only successor 25bff2e9 preserved the complete reviewed 4a519f6d tree. The final merge also preserves concurrently landed #945; three independent composition reviews cleared exact tree 1a015493, and 75 focused report/evidence tests passed against that combination. Relative to #888, the physical-free measurement, reserve and cache-release policy are unchanged. The new demand estimates can change which plans are accepted, split or refused. No public API or art.megatron change is included.

Additional independent review: Fable cleared this exact #898 diff with explicit behavior limits. It adds partial memory components and reprices selected slots; admission, splitting and execution order can change in either direction across different profiles/plans. Additional metadata walks and exact-plan construction have unmeasured latency. Grouped planner-report replay remains incomplete where immutable runtime facts are missing. Brad explicitly accepted both #888’s recovery behavior and #898’s admission behavior after reviewing these limits.

The completed combined native run used source f80dba32: the full eight-pair prefix returned six backwards and one optimizer update, followed by the unchanged flat16 target—16 requests, 101,280 packed/logical rows—and its backward. Seven output windows and 660 parameter entries were retained. Original native/container exits were 0; the four owned remote UIDs and final known host union were closed. Root accepted source-bound completion and owned cleanup in B/combined-full8-native-v6-final-root-acceptance.json (47cb19ea), joining independent review 72f896c5, strict result b18a78f0 and final closure ae3f0e39. Here B = /home/brad/.local/share/schulman/art848-resume-20260921-root.

Two additional single-H200 workloads have now completed on f80dba32:

The full hidden-output probe completed 256 items / 384 requests / 5,622,779 logical tokens through one API call, with the planner choosing 27 internal waves. The caller did not manually split the input, and original output retention/pooling produced finite 256 × 2048 features. Diagnostic capture remains incomplete because the private reader rejected a source hardlink; no admission-accuracy or memory-peak claim follows. Receipt: B/052-local-forward-success-20260923/acceptance.json (b9f155f3).
A two-group, four-alternative DPO-shaped callback completed discriminator-gradient accumulation, mixed policy/reference passes and one finite joint optimizer update. This is a structural surrogate, not the exact historical failed step. Both original caller cache-release sites remain, so it does not qualify automatic recovery budgeting. Receipt: B/053-local-structural-success-20260923/acceptance.json (a90b2033).

Both original tasks and managed processes returned 0, and known owned resources are closed. Independent readout reviews cleared completion within those limits. These runs exclude the later #931 workspace allowances; the same-source #888 ON/OFF comparison also completed. Its strict result is INCONCLUSIVE because execution layouts and reserved-cache state differed; separate full-work timings were 518.711 s enabled versus 504.801 s disabled (+2.756%), descriptive only. Exact readout and limits are in #888 and B/888-current3b0-onoff-result-20260923/.

The landing tree 06fc8182 has all 14 planner modules, 37 related test files and CI routing byte-identical to f80d. It additionally preserves current main's Tinker/Responses/docs changes; the two upstream modules already passed 32 tests. This is a source-equivalence mapping, not a claim that the whole 4a519 package was executed.

Supporting #888 evidence remains source-scoped: one positive pressure release followed by backward/update (B/888-current674-pressure-final-root-acceptance.json), grouped save with exact fresh-state reload and a further update (B/888-current674-grouped-save-fresh-load-root-acceptance.json), and the fixed 12-update observation of +0.21668% (B/888-current-device-timing-observation-root-acceptance.json). The timing pair had no explicit releases and strict warmed status UNKNOWN; it is not a sustained overhead bound.

Focused planner/report checks passed 98 tests; the combined recovery/evidence fixture checks passed 15 cases + 2 subtests. The exact successor's required default prek ran once: Ruff, formatting and lock checks passed; ty retained the same 11 diagnostics in unchanged files, with no new diagnostic or byte change. At exact head 4a519f6d, quality CI and two-H200 validation both passed; Package Install was skipped. Three independent source reviewers cleared this exact composition. The local baseline type-check failure remains preserved.

The strict full8 reader reports source-bound completion only. Counter continuity/admission qualification, numerical comparisons and an old-runtime counterfactual remain unavailable; historical RNG, allocator and dataloader state were not recreated. Prior refusal, setup/readout and numerical failures remain preserved in /home/brad/.local/share/schulman/pr888-pr898-evidence-draft-20260923-cold-v1/ARTIFACTS.md. This closes the specific missing prefix-update/flat16 completion observation, not generic #848/#870, gradient parity, all-shape memory safety or a universal 5% overhead claim. Brad approved landing this admission change; the broader qualification limits remain.

Final successor 25bff2e9 passed quality CI and two-H200 validation. Root verified the actual merge parent, complete tree, file modes and patch against the reviewed combination. Post-merge main CI and image publication are separate automatic follow-up checks; no manual deployment was performed. Exact receipts: B/merge888898-20260923/.

bradhilton