Skip to content

placement: plan_nodes checks its inputs and decides each node in helpers, and nic_not_max pins a window's one rounding - #163

Merged
juliankang4 merged 2 commits into
midagedev:mainfrom
juliankang4:dgx-spark-nodes-window-pin
Oct 14, 2026
Merged

juliankang4 merged 2 commits into
midagedev:mainfrom
juliankang4:dgx-spark-nodes-window-pin

Conversation

@juliankang4

Copy link
Copy Markdown
Collaborator

What and why

This follows up the two non-blocking points of the review on #95 (now in main as 33fb317b). It is two commits on main baeedd73, a move and a test, kept apart (R22):

  1. d51ec11f splits plan_nodes (R12). The input checks move to checked_id_bytes and the decision on each node to idle_why. Behavior is unchanged.
  2. 8d2ac015 adds a paged case to nic_not_max. Before it, no test turned red when a window's read, rows and fill rounded down one by one instead of together.

Code shape

The review reads these first. A box left unticked needs one line of why.

  • No second copy. The helpers hold code that was inline in plan_nodes, moved once.
  • Model code holds only architecture facts. No model code is touched.
  • Dead code is gone. Nothing is left unused.
  • No silent failure. The checks and their errors are unchanged, in the same order.
  • Comments state what is true now.
  • No relaxed gate. No pin moved; nic_not_max gains one case.
  • cargo fmt and cargo clippy are clean on what I touched; the lint count did not go up (41, the ratchet).

Duplication found or removed (path:line), or "none": none.

1. The split of plan_nodes (R12)

plan_nodes checked its inputs, built the candidate plans, decided each node and named the idle nodes in one function of 131 lines. Now:

  • checked_id_bytes(nodes, terms, budgets, workload) runs the checks in their old order (fabric, unified pools, budgets, the undefined inputs, overflow) and returns one id's bytes over every routed layer. The refusals stay listed on plan_nodes, the public function.
  • idle_why(now, used, next, j) returns why the plan leaves node j idle, or None where it takes it. A node is taken where the pools of the nodes before it cannot hold every id and its own has room, and otherwise only where the link stays within a tenth of the step and the step gains twice the ruler. The comparisons are the same expressions.
  • plan_nodes keeps the candidate plans, the loop over the nodes with the pool check, the pool bound and the idle list, in 57 lines.

Both helpers are private. Compared as a multiset of lines, the old plan_nodes and the three functions differ in the helpers' signatures, calls and returns, in let n = nodes.machine.nodes.len();, which both plan_nodes and checked_id_bytes now read, and in three lines: the needed-node test reads now.room(used) and now.terms.experts, the ruler reads now.nodes.ruler_permille, and the decision is Ok(if ...) in place of why[j] = if .... One path changes shape. A needed node used to be taken on the spot, with continue; idle_why now returns None for it, and the loop's if why[j].is_none() takes it, as it takes a node that passes the link and gain lines. Either way why[j] stays None and the node joins used. The placement tests pass 165 of 165 at both commits (155 at #95's merge; main has since added ten ctx tests), and each mutant of #95's table that changes plan_nodes turns the same tests red as it did there (below).

2. A window's one rounding

layer_ps sums a window's read, its rows through the NIC and its fill over one denominator, so the window rounds down to whole picoseconds once. The mutant that rounds each term down on its own reads a window up to 1 ps lower where it has two of the terms, and 2 ps where it has all three [derived]. The link, gain and NIC cases all held every id, so no window had a fill, and the mutant stayed green.

The new case pages, so each window carries a read and a fill, both with a fraction of a picosecond:

  • Forty ids of 649,500,000 B at top-k 1, read at 10¹⁹ B/s: a read is 64.95 ps a slot, and the serial term (the head, 4 × 649,500,000 B) 259.8 ps.
  • Two nodes with room for one id each (with_room), a hit rate of 999 ‰ and a miss's fill of 1,000 ps. The split is [20, 20], so each node pages 19 of its ids, a fill of 19/20 ps a slot.
  • The step is 259.8 + 64.95 + 19/20 = 325.7 ps [derived]. The predictor reads 259 + 65 = 324 ps, because each window rounds down once. With one routed layer a step can read less than 2 ps low (the L + 1 ps bound described on placement: a model's plan over several unified nodes (dgx-spark.md §9), and nodes_scale on one to four Sparks #95).
  • A row of 647 values (2,588 B) at a NIC of 8 × 10¹² B/s: node 0 takes half a row a token, 1,294 B, so its decode NIC time is 161.75 ps, half of 323.5 ps.

The line at 323.5 ps sits 2.2 ps under the step, more than the 2 ps the step can read low, so the predictor's verdict is the real one: nic_bound() is None. The mutant reads each window as 64 + 0 = 64 ps, a step of 323 ps, which is under the line, so it names node 0. The test asserts the split and nic_bound() only. The permille it reports moves about 1.5 ‰ for each picosecond of step here, and the contract does not fix it to that precision.

What I ran

On a Mac (Apple M5) through just, with main baeedd73 as the scope base. The placement crate is pure and runs natively. just mac-test ran at each commit; its summary line, and the placement library's result:

d51ec11f: mac-check: test rc 101; 12 crates (crate=rc): bloomery-decision=0 bloomery-hf=0 bloomery-jinja=0 bloomery-levers=0 bloomery-models=0 bloomery-placement=0 bloomery-refset=101 bloomery-runtime=0 bloomery-sampler=0 bloomery-serve=0 bloomery-tokenizer=0 bloomery-vision=0; 1196 passed, 1 failed, 179 ignored in 43 test binaries
          bloomery-placement lib: test result: ok. 165 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 61.47s
8d2ac015: mac-check: test rc 101; 12 crates (crate=rc): bloomery-decision=0 bloomery-hf=0 bloomery-jinja=0 bloomery-levers=0 bloomery-models=0 bloomery-placement=0 bloomery-refset=101 bloomery-runtime=0 bloomery-sampler=0 bloomery-serve=0 bloomery-tokenizer=0 bloomery-vision=0; 1196 passed, 1 failed, 179 ignored in 43 test binaries
          bloomery-placement lib: test result: ok. 165 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 63.19s

The one failure at both commits is bloomery-refset's arch::draft::tests::a_plain_target_with_no_draft_beside_is_the_family_path, which compares a /var/folders temporary path with its /private/var/folders form. It fails the same way on main, and this PR does not touch that crate.

just mac-static upstream/main at 8d2ac015 (upstream/main = baeedd73):

mac-static: fmt-check rc 0 in 15 s — rustfmt diff blocks: 0
mac-static: lint rc 0 in 51 s — mac-check: lint ^warning: 41 = the ratchet 41 in tools/lint-ratchet.txt; target <worktree>/target, 25 bloomery crates checked or compiled
mac-static: combos rc 0 in 73 s — mac-check: combos rc 0; 21 shapes run, 0 red, 28 skipped (scoped), in 73 s (recipes.py combos: 21 of 42 commands to run, 212 distinct (package, target, mode, features) tuples, 10 green in the ledger, 11 with no input changed since baeedd73a, 7 left out by name)
mac-static: check-arch rc 0 in 24 s
mac-static: check-comments rc 0 in 25 s
mac-static: check-defaults rc 0 in 2 s
mac-static: check-levers rc 0 in 21 s
mac-static: check-loads rc 0 in 3 s
mac-static: check-ptx-names rc 0 in 1 s
mac-static: check-records rc 0 in 15 s
mac-static: check-recipes rc 0 in 129 s
mac-static: check-rustflags rc 0 in 0 s
mac-static: check-unsafe rc 0 in 2 s
mac-static: check-waits rc 0 in 1 s
mac-static: total 129 s, base upstream/main, ledger <home>/.cache/bloomery/mac-static-ledger.tsv
mac-static: ok

The move needs no ptx-scan or timed A/B: the placement crate links no kernel, nothing outside it calls the node plans yet, and the planner runs when a plan is made, not in the step.

The mutants ran one at a time on this branch's tree before the rebase onto baeedd73 (then on 33fb317b; nodes.rs and nodes_scale.rs are byte for byte the files at 8d2ac015, and main changed no other file those tests read), each followed by cargo test -p bloomery-placement --lib nodes_scale. They are #95's table with the window row now red. Six of them changed a line that moved into a helper, so their text follows the line: the ruler fixed at 20 and at 10 ‰, the link share and the gain read in whole nanoseconds, the link share read on the plan without the node, and the two rules for a needed node. Each makes the same change as before.

Every mutant built. The rows for plan_nodes, the move's evidence:

Mutant (its exact form) Red Against #95
M2, only nodes 0 and 1 planned (for j in 1..n.min(2)) fills_balance, link_share, messages_o_n, never_slower, nic_not_max, pool_per_node same
M7, ruler fixed at 20 ‰ (2 * 20u128) never_slower same
M7, ruler fixed at 10 ‰, the box's (2 * 10u128) never_slower same
the link share and the gain read in whole nanoseconds (/ 1000 * 1000 on both steps and the link in idle_why) link_share, never_slower same
a node needed where a pool is over, the old rule (if !now.over()?.is_empty()) fills_balance, nic_not_max, pool_per_node, read_balance adds nic_not_max (the new case)
a node needed whatever its own room (drop && now.room_ids(j) > 0) pool_per_node same
a node whose own budget passes its pool is tried (drop the fixed > pool check in plan_nodes) pool_per_node same
idle list from the loop only (no fallback reason for a node the final split empties) pool_per_node same
a prompt rate of 0 accepted (drop the prompt_rate == 0 refusal in checked_id_bytes) named_refusals same
the link share read on the plan without the node (now.exposed_ps(a) and > a for next's and b) link_share same

Besides the old rule for a needed node above, two rows turn more tests red, each through the new case:

Mutant (its exact form) Red Against #95
a window's reads, rows and fills rounded down apart (each over its own rate in layer_ps) nic_not_max was none
M6, a post on node 0's window, N = 1 included (+ POST_NS * 1000 on node 0's base) fills_balance, link_share, n1_is_today, never_slower, nic_not_max, read_balance adds nic_not_max

The other 26 rows match #95's table. Of these, 25 turn the same tests red, and one stays green, as on #95: the optimum's share sum in u64, which cannot pass u64 in a test, since each share is at most 1000 × experts and the split walks every id.

Hardware (if the change is for a machine)

None. Every number above is the predictor's [derived]; nothing was timed. The Mac checks ran on an Apple M5 (32 GB) with rustc 1.100.0-nightly (e457a7b0d 2026-08-27).

AI help

AI 코딩 에이전트가 코드를 작성한 후 설명을 썼고, 테스트를 마쳤습니다. diff 리뷰와 PR 초안 리뷰는 제가 직접 했습니다.

…ers of its own

plan_nodes checked its inputs, built the candidate plans, decided each node
and named the idle nodes in one function. The input checks move to
checked_id_bytes, which returns one id's bytes over every routed layer, and
the decision on a node to idle_why, which returns why the plan leaves the
node idle or None where it takes it. Both are private; plan_nodes keeps the
candidate plans, the loop over the nodes, the pool bound and the idle list.

The checks keep their order and the decision its comparisons: a node is
taken where the pools before it cannot hold every id and its own has room,
and otherwise where the link stays within a tenth of the step and the step
gains twice the ruler. The lines the move changes are the helpers'
signatures, their calls and returns, and nodes, terms and used read through
the plan (now.nodes, now.terms, used as a slice).
… in one rounding

layer_ps divides a window's reads, rows and fills over one denominator, so
the window rounds down once. No test turned red when the three terms each
rounded down on their own: the link, gain and NIC cases hold every id, so
no window carries a fill.

nic_not_max gains a paged case on two nodes of one id's room each: forty
ids of 649,500,000 B at top-k 1 read at 10^19 B/s, a hit rate of 999
permille and a miss's fill of 1,000 ps, split [20, 20]. Each window is a
read of 64.95 ps and a fill of 19/20 ps, so the step is 325.7 ps and the
predictor reads 324 ps. Node 0's NIC time is 161.75 ps, half of 323.5 ps:
the line sits 2.2 ps under the step, more than the 2 ps a one-layer step
can read low, so no node is named. Rounding the read and the fill apart
reads each window 1 ps lower, a step of 323 ps, and names node 0.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CLEAN: The extraction preserves validation and node selection, and the added case rejects separate window rounding.

Checked

  • Duplicates: 2 new functions checked against existing placement owners; none duplicated. Dryer reported 0 pairs.
  • Complexity: idle_why is new at 18; its decision branches moved unchanged from plan_nodes.
  • Silent failure: 3 sites followed. Time conversions return named overflow errors; the idle fallback supplies a reason before filtering.
  • Tests: nic_not_max protects the combined division at crates/placement/src/placement/nodes.rs:1191. Separately rounding read and fill changes the derived step from 324 ps to 323 ps and returns Some; the assertion requires None.
  • Gates: no pins, thresholds or cases removed or relaxed. The PR supplies mutation evidence for the added case.
  • Adversarial checks: empty nodes and mismatched budgets fail before indexing; needed-node acceptance and ruler short-circuiting match the base.
  • Performance: planning code only; all plan_nodes callers are tests.
  • Escape catalog: no patterns matched.

Notes

Rust tests were not rerun. The PR reports 165 placement tests passing at each commit.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

@juliankang4

Copy link
Copy Markdown
Collaborator Author

The review is done and the contribution check passed, so I'm merging this.

@juliankang4
juliankang4 merged commit cb5688c into midagedev:main Oct 14, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

placement Placement planning: crates/placement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant