[juliankang4]

What and why

This is 9.5 item 2 of docs/contrib/dgx-spark.md and the fault copy of its item 7, the words row of the P5 plan in #127. The maintainer agreed to this design in #127. The branch is four commits on main (a13125a8, with #96's Link seam merged), so the diff to read is a13125a8..01bf2300. Part of #13.

A row's go counter is added to by the stage card and taken back by the tier card, and its done counter the other way. On two pages, a copy of the writer's value would erase the reader's take back. A captured stream wait keeps its captured value on every replay (the HostFlags doc, crates/gpu/src/graph.rs:754), so 9.5 item 2 as first written, a greater-or-equal wait on a target that rises each replay, cannot be captured. This PR takes the design in 6139195763: only the counters that cross the link become generations. The card operations, the graphs and the host's signal code are one path over every link, and only the Link impl behind them differs.

• Commit 1 moves the host's word accesses into Link calls: the progress and fault reads, the release, the rest check, the reset, the fault clear and the fault word's first value. The accessors that handed out a word or an address go: Link::host_at, TierPage::word, TierPage::word_at, and the pub(super) Link::word, whose name stays on a private helper behind the link's own calls. On the shared page each call is the load or store the host did before, with the same ordering, and the release, rest check and reset are Link::store and Link::load there. The one change is the fault word's first value at load: a write_volatile before, Link::store's release store now. Both are one aligned 32-bit store, made before any card runs. tools/unsafe-ratchet.txt comes down from 1750 to 1749 for the unsafe block that goes.
• Commit 2 adds a second impl, the loopback, picked by BLOOMERY_TIER_LINK=loopback (unset: shared). It has two pages on one machine, the stage card's and the tier cards', each allocated in its own context, and a host pump thread that carries each crossing counter from the writer's page to the reader's.
• Commit 3 releases all of a tier's counters in one call, under one hold of the loopback pump's lock (below, "The order of a release").
• Commit 4 is a refactor with no change in behavior: TierPage::counters is the one list of a tier's counters, which the release, the rest check and the reset read.

For each crossing counter (a row's go and a row's done, per tier):

Word	Page	Written by	Read by
generation	the writer's	the writing card, +1 only (the shared page's operation)	the pump
counter	the reader's	the pump, +1 per new generation; the reading card, -1 after its wait (the shared page's operation)	the reading card's wait for at least 1 (the shared page's operation)

• What a step means. The pump compares the writer's generation with the one it last carried, wrapping. A step of 0 carries nothing, a step of 1 carries one, and any other step stops the pump by name, since a row has at most one go or done in flight. The step is judged before anything is published: a bad step publishes no bytes, no progress and no counter. The host then sees the tier behind, the next deadline names the tier as lost, and the lost detail and the rest check's message add the link's reason (the counter's byte offset and the step).
• The order of a carry. For a step of 1 the pump copies the bytes the counter publishes to the reader's page, then stores the progress word (for a done), then adds one to the reader's counter.
• What each counter publishes. A go publishes the row's image. A done publishes the row's routed rows and the tier's fault copy, which is item 7's fault copy on a mirror.
• Progress. The host's progress word on the stage card's page is the count of dones the pump has carried, plus the tier's progress at the last reset. This differs from 6139195763, which had the pump copy the tier's progress word and hold each done until that word caught up. Counting the carried dones gives the same value, since each signal adds one to a done and to the progress. It also never runs ahead of the rows carried with those dones, and it assumes no order between the tier's done and progress writes.
• Release, rest and reset. A release takes all of a tier's counters in one call. On the loopback that call holds the pump's lock while it stops carrying every one of them and writes RELEASE to each reader's counter. The other tiers' counters keep carrying. At rest means the reader's counter at 0, the writer's generation carried, and no bad step seen. A reset, taken once both cards' streams have drained, takes the writer's generation as carried, restarts the progress count from the tier's page, zeroes the reader's counter and resumes that counter alone. A counter still released keeps what its writer adds until its own reset. A pump stopped by a bad step stays stopped across every reset, as a lost card stays lost.
• Other details. Link::store (the fault clear, and the fault word's first value at load) writes the tier cards' page first, since the pump carries from there to the stage card's. Link::broken also names a pump thread that has ended. The pump thread starts through threads::helper::spawn_helper with Placement::Float, so a load thread pinned to one cpu does not share its core.

On the shared page nothing changes: the same addresses, values, card operations and graph structure (checked below). The loopback is for checking the protocol on one machine, and its pump spins on one core while the model is loaded, which the lever's doc says. The relay works over any page type (Page in host/link/pump.rs), and its step rule and its order (the bytes, then the progress, then the generation) carry to the transport of #127. The rest is the transport's: a pump on each node, the inbox word between them, the release of a remote node's words, the per-tier overlap of spans, and item 7's check for a stale or doubled image.

The order of a release (third commit)

The second commit released a tier one counter at a time: the go, then the done, with the pump's lock dropped in between. Once the go is released, the tier card runs the cancelled graph's remaining layers, and each layer adds one to the done, which still carries. A pass of the pump between the two releases then sees the done more than one generation ahead, the step rule stops the pump for good, and the first round after the reset never arrives.

The second commit's recovery test shows it. Run alone 20 times on the second commit's code with only the third commit's failure message added, it was red in 4 runs, each on the round after the reset, and each names the done at byte 64 (three of them "stepped by 2", one "stepped by 3"):

assertion `left == right` failed: the pump: Some("the tier page's counter at byte 64 stepped by 3 between two relays: a row has at most one go or done in flight")
  left: Err("the stage's done 2004 never came")
 right: Ok(())

The third commit releases all of a tier's counters in one call. TierPage::release passes every row's go and done to Link::release at once, and on the loopback that call holds the pump's lock across all of them, so no pass falls between two of them. The lock closes the case. Under one hold, stopping and raising each counter in turn would do the same. Relays::release also finds every counter's line before it stops any, so a counter that does not cross panics with nothing stopped. The recovery test's failure now prints the pump's reason.

A new test, hw_a_release_takes_a_tiers_counters_in_one_hold_of_the_pump in host/tier.rs, makes the case without timing. It opens a loopback page on two Gpus of one card and calls TierPage::release for its one tier. A test-only step (Link::run_on, under cfg(test)) plays the released tier card: after each release it adds three to the done on the tier cards' page and makes one pass. Released in one call, the done is already stopped and nothing is carried. With TierPage::release split back into one call per counter, the pass between the go and the done stops the pump, and the test fails every time:

left: Some("the tier page's counter at byte 64 stepped by 3 between two relays: a row has at most one go or done in flight")
right: None

The test enters at TierPage::release, which TierCard::release calls with its index; a TierCard itself needs resident weights, the tier kernels and an architecture's experts, which a unit test does not have. The four relay tests stay green under that split, since they call Relays::release directly.

The engine's TierCard::release had the same split, so a cancelled graph over the loopback could have broken the link the same way [inferred]. The gate's clauses do not make that case: in T2 and B2 nothing moves between the release and the reset, and in the lost-card clauses the card has already stopped.

Where each part lives

• crates/gpu/src/host/link/pump.rs (new): Relay (take, :72), the step rule for one counter; Relays, every crossing counter's line with its carry (:182), and the release (:229), rest check (:244) and reset (:260). Four host tests (below).
• crates/gpu/src/host/link.rs: Link::new (:124) picks the kind. Link::loopback (:152) refuses a crossing that is not 4-aligned inside the page by name, allocates the tier side in the tier's context and starts the pump thread. load, store, release, at_rest and reset (:266 to :314) are the host's word calls. On the shared page each word access is one load or store, and a release one store a counter; on the loopback each acts on the side of the counter it names, and a release on all of its counters under one hold of the pump's lock. broken (:322) is the pump's reason, or an ended pump thread. run_on (:340, test only) is the released card of the release test.
• crates/gpu/src/host/tier.rs: TierLayout::crossings (:507) lists each counter, its reader and the bytes it publishes. TierPage::new (:589) passes the kind to Link::new. TierPage::counters (:789) lists a tier's counters, and TierPage::release (:799) passes them to the link in one call. TierPage::broken_note (:806) is the link's reason for a message. fault_copy, progress, lost_detail, release, at_rest, reset and clear_fault (:1216 to :1519) call the link or the page. The release test is at :1912.
• crates/gpu/src/host/mod.rs:1747: HostTier::attach_tiers reads the lever; settle_checks (:2332) adds the link's reason to its not-at-rest message.
• crates/levers: the BLOOMERY_TIER_LINK row (registry.rs:345, class M, shared | loopback), TierLink and tier_link (lib.rs:2553, :2563), read below every binary's main as nvtier_levers is.
• justfile, weekly-gpu-ds41-lost: the first line also runs the tier gate's T1, T2, B7 and B2 and the lost-card clauses over BLOOMERY_TIER_LINK=loopback. tools/defaults-twins.tsv has the row for that pin, and tools/box-fill.tsv names it.
• tools/gate-paths.tsv:205: the weekly-gpu-ds41-lost trigger row lists host/link.rs and host/link/pump.rs.
• tools/spawn-allow.txt: the relay tests' two threads in pump.rs (the pump thread itself goes through spawn_helper).

Code shape

The review reads these first. A box left unticked needs one line of why.

• No second copy. The host's word operations have one owner, the Link, and a tier's list of counters one owner, TierPage::counters. The relay rule lives in host/link/pump.rs alone, over a page trait, so a mirrored impl over RDMA uses it rather than copying it.
• Model code holds only architecture facts. No model crate changes. GLM, Qwen3.8 and V4.1 reach the changed code through the common TierCard.
• Dead code is gone. Link::host_at, TierPage::word, TierPage::word_at, the pub(super) Link::word (a private helper of that name remains), and the atomic imports in tier.rs. No load-time setter beside Link::store.
• No silent failure. A step other than 0 or 1 publishes nothing and stops the pump, and the lost tier's detail names it; an ended pump thread is named too. A crossing outside the page or not 4-aligned is refused by name at load. A value of BLOOMERY_TIER_LINK other than shared or loopback is refused by the lever parser.
• Comments state what is true now.
• No relaxed gate. No band, tolerance or lint moved. One weekly recipe gains the loopback lines, and its trigger row gains host/link/pump.rs (above).
• cargo fmt and cargo clippy are clean on what I touched; the lint count did not go up.

Duplication found or removed (path:line), or "none": the loop over a tier's go and done words, written out in TierCard::release, at_rest and reset, is now TierPage::counters (host/tier.rs:789). The page's first fault words go through Link::store (host/tier.rs:602), and on the shared page Link::release, at_rest and reset call Link::store and Link::load (host/link.rs:287, :303, :314). The relay tests take RELEASE from host/step.rs:28 (host/link/pump.rs:297).

What I ran

On the shared page this is the move class in AGENTS.md's table. The maintainer's answer in #53 lets the structural proof replace a same-lease A/B on the box if three conditions hold. Each one, with its evidence. The runs are on the current base a13125a8 and the current head 01bf2300, from git archive trees that hold every file of their commit unchanged.

1. just ptx-scan is identical for the bins it covers. Holds. For the five bins scanned, the md5 block of every kernel entry and the entry tables equal the base's. The table is in "PTX" below.
2. The graph node counts are the same, and eager equals replay. Holds. just gate-gpu-e2e, as shipped, prints graph graph_nodes=648 want=648 eager_vs_graph_identical=true ok on base and head, with the same node kinds and the same comparisons against the ik references. In the V4.1 tier gate (with the stand-ins below) the stage graph has 288 nodes and the tier graphs 64 step and 127 pair nodes on the base, on the head and on the head over the loopback, and T1 checks its graph run and its eager run each bit for bit against the reference on all three.
3. The shared Link impl adds no host work per step: no new atomic, lock or dynamic call. Holds, read from the code. On the shared page Link::mirror is None: no pump thread, no Relays, no lock. The calls a step makes read the same words with the same orderings as on main. The progress word (TierCard::progress, read by behind, wait_for and wait_caught_up) and the fault copy (fault_copy) go through Link::load, which is the same MappedHost::atomic_u32(off) and one Acquire load that TierPage::word did on main, with no branch on the link's kind. pump::Page::word(&*self.page, off) names a trait method on the concrete MappedHost, so it is a static call. The settle path's rest check and fault clear (at_rest and clear_fault, once a stream has drained) and the reset and release add a check or two of mirror before the same loads and stores; the pump's mutex is only in the loopback's arm, which the shared page never takes. A release builds one Vec of the tier's offsets, on the lost-card path only. Link::run_on and its call in release are cfg(test). So the change does not touch the dispatch path, and by the rule in dgx-spark: what is in progress on the Spark side, as a checklist #53 it needs no same-lease A/B.

For the loopback, the owning gate's clauses pass over it (with the stand-ins), and the relay's host tests fail first under seven mutants.

On a Mac

At 01bf2300, on the base a13125a8. mac-static's combos step reuses a shape that is green in its ledger for the same inputs, so 28 of the 42 shapes the scope selects ran:

$ just mac-static a13125a8
mac-static: fmt-check rc 0, rustfmt diff blocks: 0
mac-static: lint rc 0, warnings 41 = the ratchet 41 in tools/lint-ratchet.txt
mac-static: combos rc 0, 28 shapes run, 0 red, 14 green in the ledger
mac-static: check-arch, check-comments, check-defaults, check-levers, check-loads, check-ptx-names, check-records,
            check-recipes, check-rustflags, check-unsafe, check-waits: rc 0
mac-static: ok

$ just mac-test
mac-check: test rc 101; 12 crates: bloomery-refset=101, the other 11 = 0;
           1201 passed, 1 failed, 179 ignored in 43 test binaries

The one failure is bloomery-refset's arch::draft::tests::a_plain_target_with_no_draft_beside_is_the_family_path, listed in #137: the test's temporary path is listed both as /var/folders/... and as its canonical /private/var/folders/.... It failed the same way on main 4a50226c on the same Mac (1199 passed, 1 failed, 179 ignored, the same test). This PR does not touch crates/refset.

On a DGX Spark

One DGX Spark (GB10, driver 595.84), in the CUDA 13.3.1 dev container (rustc 1.100.0-nightly (e457a7b0d 2026-08-27), LLVM 21.1.8, just 1.58.0), through tools/box.sh local mode. Base and head were built with the same toolchain in the same container. The recipes ran as the justfile has them; BLOOMERY_ARCH=sm_86 is the arch ptx-scan takes.

Recipe	a13125a8	01bf2300
just gate-gpu-lib	rc 0: 1 + 233 + 4 passed	rc 0: 1 + 238 + 4 passed
just gate-gpu-e2e	PASS	PASS
BLOOMERY_ARCH=sm_86 just ptx-scan <bin> --features <f> --no-jit, 5 bins	rc 0	rc 0, every md5 block and entry table the base's
just gate-gpu-ds41-tier, BLOOMERY_TIER=fixture	rc 66 before any clause	rc 66 before any clause
just weekly-gpu-ds41-lost, BLOOMERY_TIER=fixture	rc 66 before any clause	rc 66 before any clause

The card and driver, read on the same machine (inside the container nvidia-smi reads the same name and driver):

$ nvidia-smi --query-gpu=name,driver_version --format=csv,noheader
NVIDIA GB10, 595.84

1. Host tests. just gate-gpu-lib on the head runs five tests more than on the base: the four in host/link/pump.rs and hw_a_release_takes_a_tiers_counters_in_one_hold_of_the_pump (it is hw_, so it runs under the recipe's --include-ignored). They are:

• a_relay_delivers_each_generation_once_across_the_wrap: the step rule across u32::MAX.
• the_pump_carries_every_go_and_done_once_with_its_bytes_across_a_recovery: two model cards on two pages with the pump thread running, from a generation near the wrap. 2000 goes and dones, then a release of both counters, 3 more layers on both cards with nothing carried, the go's reset, a window with the pump running, the done's reset, and 2000 more. Every go carries its image, every done its rows, the progress counts every done, and both counters end at rest.
• a_reset_reopens_its_own_counter_alone: two counters released, the done's writer adds two, the go's reset leaves the done stopped, and the done's own reset takes the two as carried; the next generation is carried once and counted once.
• a_doubled_done_publishes_nothing: a done that steps by two publishes no rows (a raised fault copy among them), no progress and no counter, the pump stops by name, and the stop holds across a reset.
• The release test of the third commit (above).

2. Structure: gate-gpu-e2e. With BLOOMERY_REF_MODEL naming DeepSeek-V2-Lite-Chat.Q3_K_M.gguf (mradermacher's DeepSeek-V2-Lite-Chat-GGUF) (sha256 642abebcf2f1efdb7a204759425865a1552b3120a9f6d3d82eb1adb8271d43fd) and the ik CUDA reference files, base and head print the same result lines:

forced_exact positions=1056 disagree=42 buckets=[15 21 4 2] ge_floor=6 pin<=6 max_margin=2.018 ok
first_token diverged_at_index0=0 ok
determinism two_eager_runs_identical=true ok
graph graph_nodes=648 want=648 eager_vs_graph_identical=true ok
graph kinds kernel=621 memcpy=27 memset=0 host=0 other=0 host_want=0 ok
seed depth=128 graph_nodes=648 want_nodes=648 seeded(pos,pos_buf,n_keys)=(129, 128, 129) decoded=(129, 128, 129) want=(129, 128, 129) ok

V2-Lite has no expert tier, so this gate does not reach the tier page; it shows the shared engine's graph and eager paths unchanged.

3. PTX. The three bins of tools/recipes.py scan-set plus generate_ds41 and gate_e2e. The scanner's argument check refuses BLOOMERY_ARCH=sm_121 for ptx, so these are the sm_86 PTX. The hash in the table is the md5 of the scan's ptx-scan-md5 block (one line per entry: name, md5, lines).

bin	features	entries	md5 block, a13125a8	01bf2300
bloomery-serve-ds41	deepseek41,vision	317	6aefac7caecd01b1	6aefac7caecd01b1
gate_kquant	gpu	238	123b985337344215	123b985337344215
gate_swap	gpu	235	088824c73145e1d3	088824c73145e1d3
generate_ds41	gpu,deepseek41	301	d7d96c3761e95e64	d7d96c3761e95e64
gate_e2e	gpu	234	e43e0ce1f9514ecc	e43e0ce1f9514ecc

The entry tables (registers, spills, shared memory and the rest of the ptxas columns) are equal row for row. The extracted modules are not byte-identical: the .oxart section is 232 bytes larger on the head in each bin. With the numeric suffix of the module-level symbols (__device_global_<hash>_<n>, __shared_mem_<hash>_<n>) stripped, each module holds the same lines; only their order and the suffixes differ. The kernel bodies are equal by the md5 block.

4. The owning gate as shipped does not run here. tools/gate-paths.tsv:205 makes weekly-gpu-ds41-lost the owner of these files, and gate-gpu-ds41-tier builds the same bin. Both stop at the recipe's dspark_model before any clause, on base and head alike:

ref-paths.sh: BLOOMERY_TIER=fixture: /models/fixtures/v41/draft holds 1 files (/models/fixtures/v41/draft/*.gguf), want the one DSpark fixture draft (`fixture generate` writes it); not running a draft on the real file (exit 66)

The v41 fixture here (from fixture fetch, at /models/fixtures/v41/) has no draft, and fixture generate needs the real DSpark draft, which I do not have. The real tier needs the 323 GiB V4.1 file and the draft. The second line of weekly-gpu-ds41-lost needs two cards (BLOOMERY_CARD=both, which box.sh's local mode refuses with 64).

5. The tier gate's clauses, with stand-ins. So the tier clauses ran on a copy of each tree with one file changed, crates/gpu-gates/src/bin/gate_deepseek41_tier.rs: the plan takes a draft reserve of 0 B, and the tier depth is capped at the real file's 23 (a zero reserve on GB10 would give the whole stack). BLOOMERY_DSPARK_MODEL names a path that does not exist, so dspark_model passes it through and no draft is read. With the stand-ins it is not the owning gate's run, so I do not count gate-gpu-ds41-tier or weekly-gpu-ds41-lost as green. Each run is the recipe's own box command (cargo oxide build ... --bin gate_deepseek41_tier && bash tools/gpu-gate.sh gate_deepseek41_tier <clauses>) with BLOOMERY_TIER=fixture, BLOOMERY_GATE_CARD=any, and for the loopback BLOOMERY_TIER_LINK=loopback in BLOOMERY_BOX_ENV.

Clauses	shared, a13125a8	shared, 01bf2300	loopback, 01bf2300
--union --fault --two --batch --batch2 --bfault	PASSED	PASSED	PASSED
--bfirst --blost --lost (the lost-card line of weekly-gpu-ds41-lost)	PASSED	PASSED	PASSED

All three landing runs print ok T1 structure: stage graph of Some(288) nodes, the reference's and graph nodes step Some(64) pair Some(127). T1 runs the prompt, 32 steps and the pairs as graphs from a reset, then the prompt and 4 steps eagerly from a reset, and checks every position bit for bit against the reference: 76 positions, tokens and logits bit for bit the reference's on all three. Of the 192 settles, 190 are early on the base, 189 on the head and 176 over the loopback.

Over the loopback at 01bf2300, abridged:

ok T1 (step, 32 steps, pairs, eager): 76 positions, tokens and logits bit for bit the reference's
ok T1 structure: stage graph of Some(288) nodes, the reference's
tier: 532 layers asked, 192 settles (176 early), hits 198 over 7 layers; graph nodes step Some(64) pair Some(127)
ok T2: the step fails with the tier's fault: layer 2 site quant_column; sites raised in layer 2 (Deepseek41 step order): quant_column
ok T2: after the reset the prompt's last position is the reference's
ok B7 (one batch against the steps, then greedy steps): 5 positions, tokens and logits bit for bit the reference's
ok B7 group (two batches against the reference's call, then greedy steps): 5 positions, tokens and logits bit for bit the reference's
ok B2 (after the reset, the prompt call against the steps): 5 positions, tokens and logits bit for bit the reference's
PASSED: gate_deepseek41_tier
ok B3f: the prompt call fails in 10.5 s naming the lost card: Hybrid::serve_batch: layer 2: the expert tier on GB10 did not finish its se[...]
ok B3: a reset is refused: Hybrid::settle: the host tier is poisoned (Hybrid::serve_batch lost the expert tier's card GB10 at layer 2: [...]
ok T3: the step fails in 10.1 s naming the lost card: Hybrid::serve: layer 3: the expert tier on GB10 served 28 of the 35 layers asked of[...]
ok T3: a reset is refused: Hybrid::settle: the host tier is poisoned (Hybrid::serve lost the expert tier's card GB10 at layer 3: [...]
PASSED: gate_deepseek41_tier

6. FAIL-first. The four relay tests were first run against the reviewed relay of an earlier version of this branch (its pump.rs with the second commit's tests): red, 3 of 4 failed (a_doubled_done_publishes_nothing, a_reset_reopens_its_own_counter_alone and the recovery test). The third commit's test was run against its own mutant, split in the table: TierPage::release hands each counter to Link::release in a call of its own, so the pump's lock is dropped between a tier's go and its done, as in the second commit. It is red, and only that test fails, by the pump's own reason: the tier page's counter at byte 64 stepped by 3 between two relays. The recovery test supports it, but only by chance. That test releases through Relays::release directly, so it does not cross TierPage::release. In its second commit shape it releases the go and the done in two calls. Run alone 20 times, it is red in 4 at the second commit's shape and green in 20 with the third commit, where it releases both in one call. These counts depend on timing.

The mutants below patch a copy of the head's tree: split patches host/tier.rs, the others host/link/pump.rs. The relay mutants' host tests ran at 01bf2300 with rustc --test on a Mac, pump.rs wrapped in a crate of its own beside a host::step that holds RELEASE; the head passes all four there. split and the engine column ran on GB10 at earlier versions of this branch whose host/tier.rs release and pump.rs relay are the head's; the engine column is gate_deepseek41_tier --union --fault --batch --bfault over the loopback, with the stand-ins.

Mutant	What it changes	Host tests	Engine, over the loopback
split	a tier's counters released one call each, the pump's lock dropped between them	red, 1 fails: hw_a_release_takes_a_tiers_counters_in_one_hold_of_the_pump ("stepped by 3 between two relays")	not run: no engine gate makes a card step between two counters of one release [inferred]
abs	the reader's counter takes the generation's absolute value: the shared page's operation over a mirror	red, 2 fail: a_reset_reopens_its_own_counter_alone and the recovery test	red: the tier is lost at layer 8, its pump stopped by name ("the tier page's counter at byte 0 stepped by 2 between two relays")
nodata	a generation is carried without the bytes it publishes	red, 1 fails: the recovery test, on its 20 s deadline	red: T1, T2 and B7 differ from the reference
anystep	any step is carried as one generation	red, 2 fail: a_relay_delivers_each_generation_once_across_the_wrap and a_doubled_done_publishes_nothing	not run: on one machine the pump never sees a step other than 0 or 1
noresync	a reset keeps the relay's last generation	red, 2 fail: a_reset_reopens_its_own_counter_alone and the recovery test, on its 20 s deadline	not run (below)
allresume	a reset resumes every stopped counter, not its own alone	red, 2 fail: a_reset_reopens_its_own_counter_alone and the recovery test, on its 20 s deadline	not run (below)
postfirst	a bad step publishes the bytes and the progress before it stops	red, 1 fails: a_doubled_done_publishes_nothing	not run (below)

noresync and allresume matter only when a writer adds to a counter between a release and the reset that follows it. In T2 and B2 the tier's fault is read after the tier's last signal, so nothing moves in between [inferred], and a lost card's reset is refused. The recovery test builds that case: after the release both cards run three more layers, and the two resets are taken apart while the pump runs. postfirst needs a bad step, which one machine does not produce. The engine column is run only where the gate's clauses can tell.

On the loopback a release writes the reader's counter on the same machine, so the lost-card clauses over it check the release of waits on one node. Releasing the waits of a remote node's card is the transport's.

Not run here

• gate-gpu-ds41-tier and weekly-gpu-ds41-lost as shipped, past their dspark_model (4 above), and the two-card gates. They run the same host code as before on the shared page.
• The loopback on two physical cards, and every family's tier gate other than V4.1's over it.
• The ordering checks on two Sparks (strict ordering, stale lines, node 1's own drain deadline). One card's coherent memory would likely hide an ordering bug [inferred]. They are the transport row of dgx-spark: P5 goes ahead with generations only on the link #127.

Docs

docs/contrib/dgx-spark.md described the shared page alone and cited lines that this change moves. The PR updates each of those places to what the tree holds now. Line numbers are at 01bf2300:

• The P5 row of section 1 (l. 42) and its file list (l. 46).
• Section 9's opening (l. 466 on), 9.6 item 3 (l. 687 on) and the Now line of answer 20 (l. 1236). They record that the maintainer agreed to the counter relay in dgx-spark: P5 goes ahead with generations only on the link #127. 9.6 item 3 also records that the maintainer takes the relay on gpu, gate-paths: the tier page is reached over its link; host/link.rs holds the shared page, and the stage card, each tier card and the host take the page's addresses and words there #96's structural proof in place of a same-lease A/B.
• 9.5's paragraph on the seam's impls (l. 637 on), and 9.5 item 2 (l. 648 on), rewritten for the relay and, in the third commit, for the order of a release.
• "What carries to N = 4" (l. 728).
• The cuMemHostAlloc call, cited as host/tier.rs:561 and now host/link.rs:143 (l. 352 and l. 635).
• The host/tier.rs lines it cites: :20-24 (l. 605), :37-42 (l. 672), :49-54 (l. 674 and l. 969), :42-44 (l. 676) and the spin wait :1476 (l. 1293).

Hardware (if the change is for a machine)

• One DGX Spark (GB10, cc 12.1), host driver 595.84, in a CUDA 13.3.1 dev container, tools/box.sh local mode. The gate kernels were built for sm_86 and JIT-compiled on the card. Inputs: the v41 fixture from fixture fetch, the published midagedev/bloomery-refsets, DeepSeek-V2-Lite-Chat.Q3_K_M.gguf and the ik CUDA reference files for gate-gpu-e2e.
• The Mac: Apple M5, rustc 1.100.0-nightly (e457a7b0d 2026-08-27).
• These are pass and fail results, not timings.

AI help

AI 코딩 에이전트가 코드를 작성한 후 설명을 썼고, 테스트를 마쳤습니다. diff 리뷰와 PR 초안 리뷰는 제가 직접 했습니다.

[github-actions]

Verdict: MUST_FIX: Two proven duplicates can reuse existing owners.

Must fix

1. Remove the redundant load-time setter

crates/gpu/src/host/link.rs:267 duplicates crates/gpu/src/host/link.rs:281: both set a word on both pages (kind B, R14). The sole caller, crates/gpu/src/host/tier.rs:602, initializes the fault word before any card reaches the page. The existing tier-first release stores also satisfy initialization.
Deleting it now leaves one setter to keep the pages consistent.

Delete Link::init. Replace its sole call in TierPage::new with:

            link.store(layout.word_off(t, TWord::Fault), FAULT_NONE);

Replace the module documentation line that mentions init with:

//! word ([`Link::load`]), stores one ([`Link::store`]),

2. Use the production release constant in relay tests

crates/gpu/src/host/link/pump.rs:325 repeats the cancellation sentinel owned by crates/gpu/src/host/step.rs:28 (kind B, R14). Recovery tests now have a second definition of the production value.
Importing the owner now prevents the test value from drifting when that value changes.

Replace the local constant with:

    use crate::host::step::RELEASE;

Non-blocking

⚠ Current gate evidence: justfile:1633. The reported runs use earlier commits and modified gate inputs; they do not verify 84f77b01..b046bc1e.
Attach the covering PTX scans, structural comparisons and unmodified owning-gate output for the current base and head, including the model path and card line. Checking these commits before merge separates this change from the intervening base changes.

Checked

Duplicates: both setter bodies and both release constants confirmed; both fixes reuse existing owners and remove definitions.
Complexity: bloomery_levers::read adds scope match arms and a predicate, routine registry growth.
Silent failure: one discarded Result followed, the pump join during drop. A stopped pump is named by broken() on the lost-tier path; teardown publishes no success result.
Tests: static mutation audit covers pump.rs:73 (non-wrapping subtraction), :198/:210 (omit bytes/store an absolute counter), :266 (resume every line), :186 (accept a doubled done), and tier.rs:801 (split release). Each makes its corresponding new test fail. Registry-row removal and main-scope refusal fail the changed lever assertions.
Gates: unsafe-block ceiling tightened 1747 → 1746; no bands relaxed. The new release test has FAIL-first evidence.
Escape catalog: none of C1–C4 matched.

Notes

Static review. No tests or CUDA gates were executed here. The supplied runs use modified gates and predate the current base and head.

_{Card check: needs-gpu. This pull request changes crates/gpu/src/host/link.rs, crates/gpu/src/host/link/pump.rs, crates/gpu/src/host/mod.rs … and carries no gate run on a card (CONTRIBUTING's evidence list); a maintainer adds gpu-verified after one.}

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[github-actions]

Reuse the existing cancellation sentinel so recovery tests share the production value (R14).

Suggested change

	const RELEASE: u32 = 1 << 30;
	use crate::host::step::RELEASE;

[juliankang4]

Both must-fix items are fixed, folded into the commits that brought the code in. The branch is now rebased onto main a13125a8 (#177), so the head is 01bf2300 (commits b951ce53, 76f00cfc, a3d2ff61, 01bf2300).

1. The load-time setter. Link::init is gone, and TierPage::new writes each tier's first fault word with link.store(layout.word_off(t, TWord::Fault), FAULT_NONE) (host/tier.rs:602). The module doc names Link::store alone. The two did the same thing: both wrote the word on every page before any card reaches it. store uses a release store and writes the tier cards' page first, which is at least as strong. At that point the pump carries nothing, since Relays::new has just taken every counter as carried. The first commit's message now says the first fault words are Link::store's release store.

The same check found three more copies of the same kind, in the shared page's arm of Link::release, Link::at_rest and Link::reset. Each wrote or read the word directly, as store and load already do. They now call the owners: self.store(off, v), self.load(off) == 0 and self.store(off, 0) (host/link.rs:287, :303, :314). The orderings are the same. With that, Link::word has two users, load and store.

2. The release constant in the relay tests. host/link/pump.rs's tests import crate::host::step::RELEASE (:297) and the local constant is gone.

Other candidates I checked and left alone:

• TierLayout::crossings computes the routed rows' bytes (n_used * cols * hidden) and the fault copy's two words itself. Neither has an owner to reuse: TierPage::new and TierCard compute the same product inline, and a new helper for it would be its own change.
• The relay tests' GO, CNT and the other offsets describe a model page of their own. They are not the tier page's TWord offsets (PROG and IMAGE differ), and TWord is private to host/tier.rs.

Rebase. The only conflict was the gpu line of tools/unsafe-ratchet.txt. Main's new PIN line (1747 -> 1750, Branch::fork) stays as it is, and the pin is 1749: main's 1750 less the volatile write's block that the first commit removes. tools/check-unsafe.sh passes at 01bf2300. git range-diff shows commits 2, 3 and 4 with the same patch as before the rebase. Against the reviewed b046bc1e, commits 1, 2 and 3 differ only by the fixes above (and commit 1 by its message and the ratchet line), and commit 4 is the same patch.

Gate evidence for a13125a8..01bf2300. On a DGX Spark (GB10, driver 595.84), in the CUDA 13.3.1 dev container, through tools/box.sh local mode. Each tree is git archive of its commit, and every file was checked against the commit before and after the runs. The recipes ran as shipped, with just in the container:

Recipe	a13125a8	01bf2300
just gate-gpu-lib	rc 0: 1 + 233 + 4 passed	rc 0: 1 + 238 + 4 passed (the four relay tests and hw_a_release_takes_a_tiers_counters_in_one_hold_of_the_pump among them)
just gate-gpu-e2e, BLOOMERY_REF_MODEL naming DeepSeek-V2-Lite-Chat.Q3_K_M.gguf (sha256 642abebc...)	PASS	PASS
BLOOMERY_ARCH=sm_86 just ptx-scan <bin> --features <f> --no-jit, 5 bins	rc 0	rc 0, every md5 block and entry table the base's
just gate-gpu-ds41-tier, fixture tier	rc 66	rc 66
just weekly-gpu-ds41-lost, fixture tier	rc 66	rc 66

• Graph node count and eager = replay. gate-gpu-e2e prints the same lines on both: graph graph_nodes=648 want=648 eager_vs_graph_identical=true ok, graph kinds kernel=621 memcpy=27 memset=0 host=0 other=0 host_want=0 ok, determinism two_eager_runs_identical=true ok, and the seeded cache at 648 nodes. Its forced and greedy comparisons against the ik references are identical too (forced_exact positions=1056 disagree=42 ... ok).
• PTX. The five bins are the three of tools/recipes.py scan-set plus generate_ds41 and gate_e2e. The md5 blocks are 6aefac7caecd01b1 (bloomery-serve-ds41, 317 entries), 123b985337344215 (gate_kquant, 238), 088824c73145e1d3 (gate_swap, 235), d7d96c3761e95e64 (generate_ds41, 301) and e43e0ce1f9514ecc (gate_e2e, 234), on base and head. scan-args.sh refuses BLOOMERY_ARCH=sm_121 for ptx, so these are the sm_86 PTX.
• The V4.1 tier gate as shipped does not run on this machine. Both V4.1 recipes stop before any clause at dspark_model: ref-paths.sh: BLOOMERY_TIER=fixture: /models/fixtures/v41/draft holds 1 files (/models/fixtures/v41/draft/*.gguf), want the one DSpark fixture draft (`fixture generate` writes it); not running a draft on the real file (exit 66). The v41 fixture here (from fixture fetch) has no draft, and fixture generate needs the real DSpark draft, which I do not have. The real tier needs the 323 GiB V4.1 file and the draft. weekly-gpu-ds41-lost's second line needs two cards (BLOOMERY_CARD=both, which box.sh's local mode refuses with 64).

So for the tier clauses I can only offer the stand-in run again, now on these two commits. It is not the owning gate's run. The patch to gate_deepseek41_tier.rs gives the plan a draft reserve of 0 B and caps the tier depth at the real file's 23; nothing else in the tree changes. BLOOMERY_DSPARK_MODEL names a path that does not exist, so dspark_model passes it through and no draft is read. The runs use BLOOMERY_TIER=fixture, the v41 fixture at /models/fixtures/v41/ and BLOOMERY_GATE_CARD=any, through tools/box.sh and tools/gpu-gate.sh as the recipes call them:

Clauses	shared, a13125a8	shared, 01bf2300	loopback, 01bf2300
--union --fault --two --batch --batch2 --bfault	PASSED	PASSED	PASSED
--bfirst --blost --lost	PASSED	PASSED	PASSED

All three landing runs print ok T1 structure: stage graph of Some(288) nodes, the reference's and graph nodes step Some(64) pair Some(127). T1 checks the graph run and the eager run each bit for bit against the reference, over 76 positions. Of the 192 settles, 190 are early on the base, 189 on the head and 176 over the loopback.

The body's "What I ran" now carries these runs in place of the earlier ones, with the commands, the card line and the model paths.

[github-actions]

Verdict: CLEAN: Neither finding meets the must-fix bar.

Non-blocking

⚠ Run the shipped owning gate: justfile:1633. The PR reports that both owning recipes stop before their clauses; the successful tier runs change the gate's draft reserve and depth. Supply output from just gate-gpu-ds41-tier and just weekly-gpu-ds41-lost on unmodified trees with the required fixtures and cards. A maintainer can supply the runs the contributor could not perform, as CONTRIBUTING.md allows.

⚠ Check the copied spans beyond their first word: crates/gpu/src/host/link/pump.rs:383. The recovery test writes and checks only the first word of each 64-byte payload. Omitting each span's final word from the production loop at line 199 would go undetected by the four relay tests. Fill and check the complete spans so truncated images and rows fail the host test.

Checked

Duplicates: release, rest and reset share the counter list in TierPage::counters.

Complexity: bloomery_levers::read adds another scope and matching arms, following the existing parser structure.

Silent failure: teardown discards the pump's join result. During service, Link::broken reports an ended pump; invalid generations publish nothing and the lost-tier detail includes the link's reason.

Tests: wrap, reset, invalid-step and single-call release assertions inspected statically.

Gates: the unsafe-block cap tightens from 1750 to 1749; the weekly recipe adds loopback clauses.

Escape catalog: no C1–C4 pattern matches the changed recipe.

Notes

Validation was static. Tests and CUDA gates were not rerun; execution evidence is reported in the PR body.

_{Card check: gpu-evidence. This pull request changes crates/gpu/src/host/link.rs, crates/gpu/src/host/link/pump.rs, crates/gpu/src/host/mod.rs … and carries a gate run on the author's card, which counts as verified.}

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[juliankang4]

The review is done and the contribution check passed, so I'm merging this.