[juliankang4]

What and why

Part of #13: S7 of docs/research/tierengine.md §5, which the maintainer left to contributors with its devmap clause, to be checked on
any card (#13 (comment),
#13 (comment)). It follows the plan in my offer on #13
(#13 (comment)). Two things I found on the way, a pinned
allocation off a 4 KiB boundary and the D1 mutant's verdict per card, are in #109; this PR fixes the first and writes
the second into tierengine.md.

Closes #109.

A new lever, BLOOMERY_NVTIER_MEM, picks the memory of the NVMe expert tier's RAM arena. anon is today's anonymous
mapping. pinned is one cuMemHostAlloc for the whole tier with PORTABLE | DEVICEMAP, plus WRITECOMBINED on an
integrated device only, carved by layer_bases as before. auto (unset) is pinned on an integrated device and
anon on a card with its own memory, as tierengine.md §8 Q5 answered (point 1). On a pinned arena a pin gives the
card's address of each part of a filled slot (TierPin::device_slot), which the card leg will read from. Until then
the host leg reads the slots on the CPU, as it does today. Both arenas gave the same tokens in every run below.

What changed:

• MappedHost::read_in_place (crates/gpu/src/graph.rs) is the constructor for bytes the card and the CPU both read.
  It does not zero the pages; the arena treats them as unwritten until a fill writes them (next item).
  in_place_flags adds WRITECOMBINED when CU_DEVICE_ATTRIBUTE_INTEGRATED is set and nothing otherwise.
  MappedHost keeps one cuMemHostAlloc call (MappedHost::alloc), and MappedHost::new keeps its flags and its
  zero fill. MappedHost::atomic_u32 now checks the word's address for alignment instead of assuming a page-aligned
  allocation, which gpu: a small pinned arena from cuMemHostAlloc can start off a 4 KiB boundary, and S7's D1 mutant is red only on write-combined pages #109 showed the driver does not promise.
• DirectFile::read_span_uninit (crates/engram/src/direct.rs) reads into a &mut [MaybeUninit<u8>], because a
  &mut [u8] over the pinned arena's unwritten pages would claim they are initialized. The one read loop now calls
  libc::pread on the buffer as it stands. std's read_at is the same pread on Linux, so a fill makes the same
  system call as before. read_span(&mut [u8]) hands its bytes to the same loop, and its callers are unchanged.
• NvTierMem (crates/levers/src/lib.rs) holds the lever's three words, and NvTierMem::pinned(integrated) is the
  one place auto is resolved. ArenaKind::of (crates/gpu/src/host/nvtier.rs) asks it with the device's
  integrated bit, read by the same raw_device_integrated that in_place_flags is given, so auto pins exactly the
  arenas that would be write-combined; nvme_tier (crates/gpu/src/model.rs) calls it. accessors_read_their_rows
  reads each word and holds its answer on both kinds of device.
• ArenaMem (crates/gpu/src/host/nvtier.rs) holds Anon(AnonMap) or Pinned(MappedHost). slice and read_into
  share one bounds check. read_into reads through read_span_uninit, and slice lends only a part its fill wrote:
  a fill that placed fewer bytes than the part needs is refused before its slot reads filled. ArenaMem::pinned
  allocates len + DIRECT_ALIGN bytes and starts the arena at the allocation's first DIRECT_ALIGN-aligned byte, and
  device_start takes the same offset, so the host's bytes and the card's address stay the same slot.
  cuMemHostAlloc promises no alignment: on an RTX 5090 (driver 595.99.02) and on a DGX Spark (GB10, driver 595.84),
  a request that fits the free space of a 2 MiB chunk shared with earlier small allocations comes out of it at a
  512 B step, with or without WRITECOMBINED (gpu: a small pinned arena from cuMemHostAlloc can start off a 4 KiB boundary, and S7's D1 mutant is red only on write-combined pages #109). If the allocation fails, the error names
  BLOOMERY_NVTIER_MEM=anon as the way to keep the anonymous arena. give_back does nothing on Pinned, because the
  allocation is the load's. On write-combined pages a MADV_DONTNEED leaves them unbacked: the next fill's read fails
  with EFAULT, and a CPU read or a cuMemcpyDtoH from the device address gets SIGBUS (on the DGX Spark, and on
  the RTX 5090 with WRITECOMBINED). NvTier::of_paged takes the kind (ArenaKind), which carries the card's
  context for Pinned; host_run and nvme_tier pass the context down from each body. The carve at the end of
  of_paged (the arena's memory, each layer's base and card address, the slot books, the reader and the dropper) is
  now its own constructor, NvTier::carve, which of_paged returns through and the devmap tests call.
• TierPin::device_slot(id, part) (crates/model/src/moe.rs) returns the part's first byte as the card addresses
  it. It returns None for an id the pin does not hold and on an anonymous arena. LayerArena::span_of is the one
  place a part's position in the mapping is computed. LayerArena::part_at (which TierSlots::slot and
  device_slot call), NvTier::staged_slot and the audit all call it. TierSlots::slot's doc now names the pin its
  caller holds while it reads, which the ArenaMem SAFETY lines cite.
• devmap: two hw tests in nvtier.rs over one fixture, one filling through the page cache and one O_DIRECT. The
  fixture is a tier built by NvTier::carve over two paged layers, so the tested layer's base comes from
  layer_bases and is past 0; its arena kind comes from ArenaKind::of with the lever's pinned. Calling
  of_paged itself would need a routed plan and a model file, so the fixture starts after the plan's checks. Each
  test runs three rounds of fill, NvTier::pin, a DtoH copy through the pin's device_slot and the tier's
  TierSlots::slot, with a give-back between rounds, and checks every part against the file's bytes. The fixture
  first makes a one-byte allocation of the arena's kind and keeps it alive, so the arena is allocated where a shared
  chunk can put it off a page boundary. It gives each stack row its own file and range, maps the leg's parts to other
  rows ([2, 0, 1]) and fills an expert in the slot before. It reads the arena's flags with cuMemHostGetFlags and
  wants WRITECOMBINED exactly on an integrated device. Then it carves an arena with the lever's anon and wants
  None from device_slot and the file's bytes from TierSlots::slot, and it wants auto to give Pinned exactly
  on an integrated device. A unit test holds in_place_flags's two answers on any machine.
• PassReport::released_bytes' doc says a pinned arena keeps those pages, and HostNeed::arena's doc
  (crates/placement/src/placement/workstation.rs) counts an arena of either kind. The unsafe ratchet takes three
  blocks in gpu (1743 to 1746: ArenaMem::slice and the devmap tests' DtoH copy and flags read) and two in
  engram (8 to 10: the pread and read_span's view of its bytes), each with its SAFETY line.
• The second commit updates the docs this change makes stale:
  • docs/research/tierengine.md, the opening: S7 has landed, and the contributor holds its card leg.
  • (d): nvme_tier picks the arena by the lever and the device's integrated bit, the same bit that decides
    write-combining; what a dontneed does on a pinned arena, per gpu: a small pinned arena from cuMemHostAlloc can start off a 4 KiB boundary, and S7's D1 mutant is red only on write-combined pages #109, and D1 required red on an integrated device
    only. Its line references point at this head.
  • (h) item 1: device_slot(id, part) -> Option<u64>. Item 2: one allocation for the tier, counted in
    HostNeed.arena until the card leg, allocated without WRITECOMBINED on a discrete card.
  • The S7 row: the default, the devmap hw tests with D1's verdict per device, and its status. §5's order line:
    the contributor holds S7's card leg.
  • docs/contrib/dgx-spark.md §1 (the P4 row and the file list), §8 (its opening and "The owner"), §10 (the P4
    item) and the state lines under §11's answers 12 and 19: S7 is in, the arena on a Spark is pinned by default, and
    the card leg is left.
  • Left as recorded: the decision table in tierengine.md §8 (Q5's answer, which the default follows), tierengine.md
    §7 (dated 2026-10-11), and the wording of dgx-spark.md §11's answers.

Points for review:

1. auto is pinned on an integrated device and anon on a discrete card. This is the answer tierengine.md §8
   records for Q5/Q7 (2026-10-11), "anonymous memory on a discrete card; auto is integrated only", and the S7 row's
   "default auto = integrated only". Q5's recommendation waited for S7's clause and a measured registration time;
   the clause is now the two devmap tests (point 4), and the allocation time is in point 3. On the DGX Spark the
   pinned arena lowered decode p50 by 9.5 % (181.39 ms to 164.15 ms) and added 1.37 s to the load at a 16 GiB arena.
   That run read a cold arena: every arm filled 3068 slots and evicted none in its 64 tokens. The fills' wall
   (fill_ns) was 0.98 s lower with pinned (3.42 s against 4.40 s), close to the 0.90 s that the 63 timed decode
   steps saved (10.74 s against 11.64 s) [derived, the means of three arms]. Most of the decode gain is therefore the
   first touch of anonymous pages, which the pinned arena pays at load instead: the load, the prompt and the 63 steps
   together took 0.45 s longer with pinned (22.16 s against 21.71 s) [derived]. Once an arena has filled, an
   eviction hands its slot over with its pages, so I expect that term to go to zero on both kinds; I did not measure
   an arena that evicts on the DGX Spark (see "Not run" below). On the RTX 5090 the pinned arena lowered decode p50 by
   1.6 % (84.87 ms against 83.49 ms, three arms each, where the 95 % interval is about ±1.4 % [derived, at the 0.6 %
   SD of AGENTS.md]) and made the load 2.6 s longer (6.6 s against 4.0 s). I kept the recorded answer for a discrete
   card. If a discrete card should pin too, the switch is the Auto arm of NvTierMem::pinned, and a longer A/B on a
   discrete card would show whether the 1.6 % holds. A pinned arena never gives pages back, but the plan already
   counts them: HostNeed::bytes adds the arena's bytes whatever its kind, so a load plans the same room under
   either. The pinned allocation asks 4096 B (DIRECT_ALIGN) more than the carve, for its alignment; when the carve
   fills the granted bytes exactly, those 4096 B are outside what the plan counts. BLOOMERY_NVTIER_MEM=anon keeps
   today's arena on any device.
2. One allocation for the tier. (h) item 2 planned one cuMemHostAlloc per LayerArena. This PR keeps one
   allocation for the tier, as AnonMap does today, since §6.2 row 2's 64 GiB allocation succeeded, and keeps the
   arena in HostNeed.arena. (h) item 2 counted it in HostNeed.cards / pool_card_bytes; I would move it there
   with the card leg, as my offer said. The docs commit writes both into (h) item 2.
3. Allocation time at the planned arena size. I did not time the cuMemHostAlloc call alone. At a 16 GiB arena
   the DGX Spark's load wall is 8.70 s for anon and 10.07 s for pinned (the means of three arms), 1.37 s more
   [derived]. §6.2 row 2's linear cost, 1.59 s at 19.91 GiB, gives about 1.28 s at 16 GiB [derived].
4. Q18's clause is the two devmap hw tests, not a gate_nvtier clause. gate_nvtier plans with gate_card,
   which opens only an A6000 or a 3090, so a clause there could not run or be shown to fail on my cards.
   just gate-gpu-lib runs both tests on any card. On those two cards auto is anon, so with the lever unset
   gate_nvtier builds the same anonymous arena as before; its only change is the ArenaKind::Anon argument in
   refuse_clauses.

Order: I would land this after #100. Today no serving default on a Spark builds an arena (Qwen3.8 loads whole, and
GLM and V4.1 are refused at the floor), so GLM's and V4.1's paged loads after #100 are the first to use the pinned
default.

Files that overlap the maintainer's train: crates/gpu/src/host/nvtier.rs, crates/gpu/src/graph.rs, crates/gpu/src/model.rs,
crates/engram/src/direct.rs, crates/gpu-gates/src/bin/gate_nvtier.rs (one line),
crates/levers/src/{lib,registry,tests}.rs, crates/model/src/moe.rs, crates/placement/src/placement/workstation.rs
(one doc comment), tools/unsafe-ratchet.txt, docs/research/tierengine.md, docs/contrib/dgx-spark.md, and one
call line each in the four bodies that build a host run. #112 (the LES plan) rewrites the same fill path in
nvtier.rs in its T4 to T6 (read_pick, fill, PickRead, Inflight); this PR keeps their signatures, and its
ArenaMem::read_into and window are the door those fills write through.

Code shape

The review reads these first. A box left unticked needs one line of why.

• No second copy. One cuMemHostAlloc site (MappedHost::alloc) for every flag set; one reading of a part's
  place in the mapping (LayerArena::span_of) for the host's bytes, the card's address, the lane's slot read and
  the audit; one read loop (engram::direct::read_all) for initialized and unwritten buffers; one bounds check
  (ArenaMem::window) for reads and fills; one integrated bit (raw_device_integrated) for both the default and
  write-combining.
• Model code holds only architecture facts. The four bodies only pass gpu.context() to the common
  host_run / nvme_tier.
• Dead code is gone. Nothing this PR made unused is left; MappedHost::new keeps its zero fill on top of
  MappedHost::alloc.
• No silent failure. ArenaMem::pinned starts the arena on a DIRECT_ALIGN boundary it computes, and an
  assert names the address if that ever fails; a failed pinned allocation names BLOOMERY_NVTIER_MEM=anon;
  device_slot gives None, never an address, for an id the pin does not hold or on an anonymous arena; a
  BLOOMERY_NVTIER_MEM value outside its three words is refused by name.
• Comments state what is true now. tools/check-comments.sh is green.
• No relaxed gate. The unsafe ratchet rises for new code only, with its PIN(2026-10-13): lines;
  tools/check-unsafe.sh is green at the new pins.
• cargo fmt and cargo clippy are clean on what I touched; the lint count did not go up (41 warnings,
  equal to tools/lint-ratchet.txt).

Duplication found or removed (path:line), or "none": at bd29e9c9 a part's place in the mapping was computed
three times, in TierSlots::slot for NvTier (crates/gpu/src/host/nvtier.rs:2671-2673), in NvTier::staged_slot
(:1754-1755) and in NvTier::audit_slots (:2134-2135). All three, and device_slot, now call
LayerArena::span_of.

What I ran

At this head (905e7ac5), on a DGX Spark in tools/box.sh local mode (BLOOMERY_BOX_LOCAL=1) in the CUDA 13.3.1 dev
container, --arch sm_86, with BLOOMERY_NVTIER_MEM unset:

$ ./tools/box.sh 'bash tools/gate.sh --oxide -p bloomery-gpu --release --lib -- --include-ignored nvtier'
test result: ok. 42 passed; 0 failed
  host::nvtier::tests::hw_devmap_a_pinned_slot_reads_on_the_card_as_on_the_host_across_a_release ... ok
  host::nvtier::tests::hw_devmap_a_direct_fill_lands_in_a_pinned_slot ... ok
$ just gate-gpu-lib
test result: ok. 1 passed ... test result: ok. 229 passed; 0 failed ... test result: ok. 4 passed; 0 failed
$ just gate-levers
test result: ok. 40 passed; 0 failed
$ ./tools/box.sh 'bash tools/gate.sh --release -p bloomery-engram --lib --test engram'
test result: ok. 18 passed; 0 failed ... test result: ok. 0 passed; 0 failed; 11 ignored
$ ./tools/box.sh 'bash tools/gate.sh --release -p bloomery-engram --test engram -- --include-ignored hw_engram_direct_file_reads_aligned_spans'
test hw_engram_direct_file_reads_aligned_spans ... ok

FAIL-first for the devmap fixture at this head: I applied one mutant at a time, rebuilt bloomery-gpu, ran the
nvtier hw tests, then reverted, rebuilt and saw them green again. Each mutant is red at both devmap tests:

Mutant	Message at both devmap tests (DGX Spark)
NvTier::carve leaving a layer's card address unset	a pinned slot's address
ArenaKind::of resolving auto by the other bit	auto pins on an integrated device alone
device_slot dropping the layer base	round 0 part 0: the card's read of the slot
ArenaMem::pinned starting the arena at the allocation's first byte	the pinned arena's first byte 0x32f000400 is not 4096-aligned

On a Mac mini (Apple M4 Pro): cargo fmt --check, tools/mac-check.sh check (0 warnings), the lint (41 warnings,
equal to tools/lint-ratchet.txt), tools/check-comments.sh and tools/check-unsafe.sh are green.

Not run on a GB10: a load whose arena evicts (every arm in the A/B below evicted 0), a prompt longer than 7 tokens,
pp at P = 512 and 4096, and bloomery-serve. The prompt path is the one to watch. After the load,
Gpu::alloc_room reads the host's available memory on an integrated device to size a prompt batch
(crates/gpu-deepseek41/src/chain/attn/batch.rs, crates/gpu-glm5next/src/prefill.rs,
crates/gpu-mimo2/src/prefill.rs). A pinned arena takes its whole allocation at load, while an anonymous one leaves
its unfilled pages in the available memory, so the same prompt can find less room under pinned.

Selected gates: python3 tools/recipes.py affected upstream/main --no-box selects 144 of 156 gate recipes and two
weekly recipes (weekly-gpu-ds41-cand, weekly-gpu-glm5next-e2e-long), most of them because crates/gpu and
crates/levers are in their closure. I ran gate-gpu-lib and gate-levers on both cards. I did not run
gate-gpu-nvtier (point 4), the other tier gates (gate-gpu-ds41-nvtier, gate-gpu-glm5next-nvtier,
gate-gpu-qwen38-residency, gate-gpu-swap) or the rest of the list. With the lever unset they build the same
anonymous arena as before on a discrete card; on a GB10 they would build a pinned one, which the probe loads below
check on Qwen3.8.

The rest ran on earlier versions of this change. The mutants of part_at, D1, in_place_flags, the RTX 5090 runs
and the A/B ran on two versions I pushed as branches of my fork so the commits can be read:
31a73dd2 on b2080a5a
(the DGX Spark, and the RTX 5090's first gate-gpu-lib) and
da88ce4e, the same change
on f382b474 (the RTX 5090's mutants, repeated gates and A/B). The Mac's mac-static and mac-test and the DGX
Spark's probe loads ran on 00b71de7 on d90151ed, the head before the review fixes above. Since 31a73dd2 the
gate_nvtier clause that no card could run is taken out (point 4), and with it its unsafe block in gpu-gates, and
auto follows the device (point 1); every A/B arm set BLOOMERY_NVTIER_MEM to anon or pinned, so this does not
change what they measured. Since 00b71de7 the fill reads into unwritten bytes through read_span_uninit, a part's
place has one helper, the carve is its own constructor, atomic_u32 checks its address, and the devmap fixture
goes through the tier's carve, pin and TierSlots::slot; the mutant table above is the devmap run at this head.
Of this PR's files, the main commits between b2080a5a and bd29e9c9 change two: crates/levers/src/registry.rs
(d24cb1a4, another row) and tools/unsafe-ratchet.txt (a49e1391, the qdot pin).

On an Apple M5 MacBook, with the pinned nightly, on 00b71de7: just mac-static upstream/main is green in all 14
steps (rc 0, 651 s; fmt-check, lint with 41 warnings, combos with 27 shapes run and none red, then the eleven check
scripts including check-levers, check-unsafe and check-comments). just mac-test is green too: 12 crates,
1139 tests passed, 0 failed, 177 ignored, in 550 s. FAIL-first for the default: with NvTierMem::Auto resolving to
!integrated, cargo test -p bloomery-levers on a Mac mini (Apple M4 Pro) fails in accessors_read_their_rows on
assertion left == right failed: auto (left: (false, true), right: (true, false)), and passes again reverted
(40 passed).

On the DGX Spark, in tools/box.sh local mode (BLOOMERY_BOX_LOCAL=1) in a CUDA 13.3.1 dev container, built with
--arch sm_86, on 00b71de7 with BLOOMERY_NVTIER_MEM unset: just gate-gpu-lib green (1 + 229 + 4 passed, both
devmap tests ok), then one load per lever state with the A/B's command below at -n 32, reading the process's
/dev/nvidia* mappings from /proc/<pid>/maps once the load line printed (pinned host memory the card maps shows
there; an anonymous arena does not):

BLOOMERY_NVTIER_MEM	/dev/nvidia* mapped (B)	arena granted (B)
unset	17156665344	17179869184
anon	25268224	17179869184
pinned	17156665344	17179869184

The three loads printed the same token ids.

On 31a73dd2:

$ just gate-gpu-lib        (base b2080a5a)
test result: ok. 1 passed ... test result: ok. 226 passed; 0 failed ... test result: ok. 4 passed; 0 failed
$ just gate-gpu-lib        (31a73dd2, three runs, each)
test result: ok. 1 passed ... test result: ok. 229 passed; 0 failed ... test result: ok. 4 passed; 0 failed
  host::nvtier::tests::hw_devmap_a_pinned_slot_reads_on_the_card_as_on_the_host_across_a_release ... ok
  host::nvtier::tests::hw_devmap_a_direct_fill_lands_in_a_pinned_slot ... ok
$ ./tools/box.sh 'bash tools/gate.sh --oxide -p bloomery-gpu --release --lib -- --include-ignored nvtier'
test result: ok. 42 passed; 0 failed
$ just gate-levers
test result: ok. 40 passed; 0 failed

On the RTX 5090, in local mode on the host, built with --arch sm_86: on 31a73dd2, the same nvtier run (42
passed, both devmap tests ok) and just gate-gpu-lib (1 + 229 + 4 passed); on da88ce4e, just gate-gpu-lib
three times (1 + 229 + 4 each, base f382b474 1 + 226 + 4) and just gate-levers (40 passed).

FAIL-first on 31a73dd2 (DGX Spark) and da88ce4e (RTX 5090), with the devmap fixture of that version: each mutant
applied alone, bloomery-gpu rebuilt, the devmap tests (and for the two in_place_flags mutants the flags unit
test) run; then reverted, rebuilt and green again on both cards. part_at then computed the part's place itself;
at this head that computation is span_of.

Mutant	DGX Spark	RTX 5090
D1: give_back advising pinned pages with MADV_DONTNEED	both devmap red at the refill: round 1 slot 1: the slot's fill: NvTier::fill: .../row0: a direct read of 4096 B at 0: Bad address (os error 14)	green: both devmap ok, the pages keep their bytes (#109)
part_at dropping the skew	both devmap: round 0 part 0: the card's read of the slot	the same
part_at dropping the slot's offset	the same	the same
part_at reading window 0	the same	the same
part_at taking the part for the row	the same	the same
device_slot dropping the layer base	the same	the same
ArenaMem::pinned starting the arena at the allocation's first byte	both devmap: the pinned arena's first byte 0x32f000400 is not 4096-aligned (and 0x32f00d400)	both devmap: 0x70ab97200200 and 0x70ab9720d400
in_place_flags: write-combined on every device	unit test only: a card across PCIe (devmap green: integrated here)	unit test and both devmap: the arena is write-combined on an integrated device alone
in_place_flags: write-combined on no device	unit test and both devmap: an integrated device, the arena is write-combined on an integrated device alone	unit test only: an integrated device

D1 ran in two forms on the DGX Spark (a failed advice as a panic, and as the give-back's error) and in the second on
the RTX 5090. The advice itself returned 0 on both cards. On the RTX 5090 the arena is not write-combined and the
pages keep their bytes, so D1 cannot turn red there (#109). In the alignment mutant the arena lands inside the shared
chunk of the fixture's one-byte allocation, off a page boundary.

Timing. generate_qwen3moe built from the same tree (cargo oxide build --arch sm_86 -- -p bloomery-gpu-gates --features gpu --release --bin generate_qwen3moe), run directly, one machine lock held for all arms, each arm started
once load1 was under 1.00, a discarded anon warm-up, then anon pinned pinned anon anon pinned. Command:

env BLOOMERY_QWEN38_EXPERTS=host BLOOMERY_HOST_ROOM=<room> BLOOMERY_NVTIER_BYTES=16G BLOOMERY_NVTIER_MEM=<anon|pinned> \
  generate_qwen3moe --prompt 'The history of the printing press begins' -n 64 --place a --time

Model: Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf (unsloth/Qwen3.8-Flash-Next-GGUF). Every arm granted the
17179869184 B arena it asked for, and every arm of a run printed the same token ids. The prompt is 7 tokens.

DGX Spark (NVIDIA GB10, 595.84, stock clocks, SM 208 MHz before each arm and 2411 MHz after it), room 72G, on
31a73dd2:

Arena	decode p50 per arm (ms)	mean of p50	mean step (ms)	load wall per arm (s)	fills, evictions per arm	fill_ns per arm (s)
anon	180.51, 181.36, 182.29	181.39	184.73	8.8, 8.6, 8.7	3068, 0	4.371, 4.413, 4.407
pinned	164.58, 166.41, 161.45	164.15	170.45	9.9, 10.3, 10.0	3068, 0	3.421, 3.406, 3.424

Decode p50 9.5 % lower and mean step 7.7 % lower with pinned; load 1.37 s longer [derived, from the means]. Point 1
reads the fill_ns column against the steps.

RTX 5090 (NVIDIA GeForce RTX 5090, 595.99.02, CPU governor performance), room 40G, on da88ce4e:

Arena	decode p50 per arm (ms)	mean of p50	mean step (ms)	load wall per arm (s)
anon	84.32, 85.12, 85.18	84.87	87.81	3.9, 4.0, 4.2
pinned	83.54, 83.12, 83.81	83.49	85.45	6.0, 6.1, 7.7

I did not run the A/B again at this head. The fill makes the same pread as before, and the other fixes move no
step-time work.

What I ran into

This does not block this PR.

On an RTX 5090, a write-combined arena makes decode about 8 times slower. The first build of this change set
WRITECOMBINED on every device, and the host leg reads every slot on the CPU. The CUDA driver API documents
WRITECOMBINED memory as "cannot be read efficiently by most CPUs". A scratch build that only dropped the flag
brought the pinned arm back to anon's speed. In one session, at the settings above (40G, 16G, -n 64), the CPU
governor at performance, two arms each, interleaved, equal tokens over all of them:

Arena	decode p50 (ms)	load wall (s)
anon	86.64, 87.12	4.2, 3.9
pinned, WRITECOMBINED	681.06, 686.57	6.3, 5.9
pinned, no extra flag	83.87, 83.42	16.1, 6.3

The 16.1 s load was the first arm of the scratch build after it was built; I did not find its cause, and the
second arm of that build loaded in 6.3 s.

So this PR sets WRITECOMBINED on an integrated device only, and the devmap tests check the allocation's flags
on both kinds of device. On the DGX Spark the same flag did not slow the host leg down (the table above). I did not
check whether the driver maps those pages write-combined for the CPU on a GB10, so "WC reads are fast there" and "WC
is not applied there" are not told apart.

To see it: on a discrete card, apply in_place_flags returning CU_MEMHOSTALLOC_WRITECOMBINED for both answers,
build generate_qwen3moe and run the command above with BLOOMERY_NVTIER_MEM=anon and =pinned on a paged
Qwen3.8 plan with an arena.

A small pinned arena can start off a 4 KiB boundary, and D1 is red only on write-combined pages. Both are in
#109 with the probes behind them; the alignment fix and the fixture's one-byte allocation above come from the first,
and the tierengine.md lines from the second.

Hardware (if the change is for a machine)

Machine	Card, driver	CPU, RAM, OS	What ran
DGX Spark	NVIDIA GB10, 595.84, stock clocks	20 Arm cores (Cortex-X925 and Cortex-A725), 128 GB unified, Ubuntu 24.04.5, kernel 6.17.0-1032-nvidia	At this head: the nvtier hw tests, gate-gpu-lib, gate-levers, bloomery-engram's tests and its direct-read hw test, the four devmap mutants. On 00b71de7: gate-gpu-lib and the three probe loads. On 31a73dd2: gate-gpu-lib (base once, the change three times), the nvtier hw tests, gate-levers, the mutants, the A/B. Built in a CUDA 13.3.1 container, --arch sm_86, cargo-oxide 87bc9518, rustc 1.100.0-nightly (2026-08-27)
RTX 5090	NVIDIA GeForce RTX 5090, 595.99.02	AMD Ryzen 9 9950X (16 cores, 32 threads), 64 GB, Ubuntu 26.04.1, kernel 7.0.0-38-generic	on 31a73dd2: the nvtier hw tests, gate-gpu-lib. On da88ce4e: gate-gpu-lib (base once, the change three times), gate-levers, the mutants, the A/B. The write-combined runs in "What I ran into" on the first build. CUDA 13.3, rustc 1.100.0-nightly (2026-08-27)
Mac	MacBook, Apple M5; Mac mini, Apple M4 Pro	Apple silicon, macOS	at this head on the Mac mini: fmt-check, check, lint, check-comments, check-unsafe. On 00b71de7: just mac-static upstream/main and just mac-test on the M5, the levers FAIL-first on the Mac mini

AI help

AI 코딩 에이전트가 코드를 작성한 후 설명을 썼고, 테스트를 마쳤습니다. diff 리뷰와 PR 초안 리뷰는 제가 직접 했습니다.

[github-actions]

Verdict: CLEAN: No must-fix finding remains; the two suggestions concern validation and load diagnostics.

Non-blocking

⚠ Validate the pinned default under memory pressure: crates/gpu/src/host/nvtier.rs:179.
The allocation pins the whole arena at load. On integrated devices, Gpu::alloc_room reads available host memory; crates/gpu-deepseek41/src/chain/attn/batch.rs:670 uses it to admit prompt buffers. Unfilled pinned pages therefore consume room anonymous pages leave available.
The PR reports seven-token GB10 runs with no evictions. Add P=512 and P=4096 runs and an arena that evicts; record token ids, prompt room and fill/eviction counts.

⚠ Preserve the alignment probe: crates/gpu/src/host/nvtier.rs:187.
The alignment fix and its fixture are present, but NvTier::stats() exposes neither allocation kind nor the offset from the driver's allocation to the aligned arena.
Keep a standalone probe under tools/, or expose those fields through stats or a record, so a load can report the driver's allocation behavior without editing the fixture.

Checked

Duplicates: MappedHost::alloc owns the allocation call; host and device slot offsets share LayerArena::span_of.
Complexity: bloomery_levers::read adds one lever-membership condition.
Silent failure: read_pick rejects incomplete part reads; missing addresses return None; pinned give-back retains pages. The ignored result removes test scratch files.
Tests, static mutation analysis: WC and Auto inversions fail flags/accessor assertions. Both devmap gates compare host/device reads with file bytes across three rounds; dropped skew, row, slot or layer offsets fail comparisons. Removing the pin-membership check fails the unheld-id assertion. GPU runs and FAIL-first results are reported by the PR.
Gates: unsafe ratchets carry PIN(2026-10-13) and new-block reasons; changed gate code retains its assertions.
Performance: the fill read loop uses pread, retrying short reads and EINTR.
Escape catalog: C1–C4 have no matching changed script code. No ledger lessons are in force.

_{Card check: gpu-evidence. This pull request changes crates/gpu-deepseek41/src/body.rs, crates/gpu-gates/src/bin/gate_nvtier.rs, crates/gpu-glm5next/src/body.rs … and carries a gate run on the author's card, which counts as verified.}

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[juliankang4]

The review is clean and the contribution check passed. Following the merge order the maintainer set (#94, #95, #100, then this one), I'll merge this after #100 lands.