placement, tools: nodes_scale holds the node plans to cells tools/flow/nodes_scale.py derives apart from the planner

[juliankang4]

What and why

Follows #95, now merged. This PR is one commit, 4e480715, on main at 84f77b01. The runs below are at a515bc44 on 54ceee74; the rebase onto 84f77b01 left the patch as it was (git range-diff), and main changed no file under crates/ in between.

docs/contrib/dgx-spark.md §9.9 asks for the gate's expected sets to come from a derivation script under tools/ with its own self-test, so the pins have one derivation, and §10 asks that script to recompute the [derived] tables of §9.7 and §9.8. #95 pinned the predictor's own cells and left the script open (its open question 4). This PR adds tools/flow/nodes_scale.py, which derives the cells from §9.7's model and §9.8's criteria with no code or method shared with nodes.rs, and nodes_scale holds every plan to them. It recomputes §9.8's table and §9.7's (a) rows at the gate's inputs. The rest of §9.7 needs inputs the script does not have, so that part of the §10 item stays open (below). Part of #13 (P5).

What the script derives, and what it takes as given

Derived for each synthetic shape (V4.1, GLM, Qwen3.8; A low and high), N = 1 to 4, λ 25 and 70 µs, at the gate's RULER and S₁ 1,000 and 3,000 tok/s and at a 4 % ruler and S₁ 3,000 (144 cells):

• the nodes the plan uses, and why it leaves each other node idle (pool for criterion 1, link for criterion 2, gain for criterion 7);
• the routed ids the pools cannot hold, in all and per node (criterion 1);
• the decode step;
• the busiest NIC's busy share, and whether criterion 5 fires;
• node 0's messages a layer (criterion 6).

How it stays apart from the planner:

	nodes.rs	nodes_scale.py
split	whole ids, placed one at a time on the slowest window	node 0's share in real numbers by §9.2's rule (node 0's mean window equals a remote's, the remotes equal), found by bisection, each node held to its pool's room
windows	closed-form multinomials over the slot counts, in exact fractions of a picosecond	the count tuples of a token's slots built one slot at a time, each slot moving a tuple's probability up by the node's share; floats with + − × ÷ only, so every host writes the same file
node choice	adds nodes while the predictor says so	§9.8 stated on its own: a node the pools need is used where it has room of its own, a remote whose own bytes pass its pool is idle for it, any other is used where its link share stays within a tenth and it gains twice the ruler
NIC	the plan's own id counts	the real shares: the rows of the slots on the remotes, node 0 taking them all in

Three definitions are shared on purpose, because they are what the gate pins, not a method:

Definition	Source sentence
the prompt batch: S₁ times the one-node token's bytes over A and the slowest node's FFN at the split	§9.7: the Amdahl term 1/(1−φ+φ/N), which this is at an even split with no Sh
node 0's messages a layer: 2 × min(k, remotes used)	§9.8 criterion 6: "at most 2·min(k, N−1) for (a)"
a shape's bytes, each term rounded down to whole Q4_K rows	§9.9 "Models": "each term rounded down to whole Q4_K rows"; the script's shape() is the gate's Shape::bytes, because the shapes are the gate's inputs

Taken as given, from crates/placement/tests/nodes_scale_inputs.tsv: the machine (BW 208.8 GB/s, NIC 23.7 GB/s, a node's pool, λ, the hop, the post, the ruler, S₁), §9.7's model terms, and each node's bytes beside the routed ids (fixed0, fixed1). The last are what today's card plan sets for the shape, which the script cannot rebuild without repeating the card planner. Every row carries its kind (measured, derived or assumed) and its source. The script refuses an input outside its domain by name (top-k outside 1 to the experts, a ruler outside 1 to 1000 ‰, a zero rate, pool or row size, node 0's bytes past its pool, and so on). The gate first holds its own constants, shapes and budgets to these rows (expected_sets), so the inputs cannot drift from the gate.

Tolerances, edges and pins

The planner splits whole ids, and the script splits real shares. So every value carries a tolerance: the largest change one id moved makes, between node 0 and any remote either way and between two remotes, within each node's bounds. Each band also carries two whole units (SLACK) for rounding: the script truncates the band and rounds its value to the nearest unit, and the gate reads the planner's value rounded down, which together move the two under three units apart. The gate holds the planner's step and busiest NIC to that band, the nodes used and idle exactly, and the paged ids in all exactly and per node to one id.

Where a band holds a line (the NIC's 50 %, the link share's tenth, twice the ruler), the cells file names an edge: nic, link:<node>:<ppm>:<tol ppm> or gain:<node>:<ppm>:<tol ppm>. The derivation cannot decide such a cell, so the gate pins its side by hand in SIDES, the sides #95 already pinned, and also holds the planner's value to the band. An edge with no pin, or a derived side that differs from its pin, panics by name. Four rows are edges, two cells each at two (ruler, S₁) keys:

• V4.1, N = 2, λ 25, A low at S₁ 3,000, at RULER and at 4 × RULER: the busiest NIC is 501 ± 5 ‰ against the 500 ‰ line, and criterion 5 fires.
• Qwen3.8, N = 3, λ 70, A low at RULER, S₁ 1,000 and 3,000: node 2's link share is 99,956 ± 78 ppm against 100,000, and the node is used.

No cell has a gain edge at these inputs.

What changed

• tools/flow/nodes_scale.py (stdlib only): prints the cells, --writes crates/placement/tests/nodes_scale_cells.tsv, --checks it and the two doc tables, prints §9.8's and §9.7's tables (--table), and has a --self-test (below). Registered in tools/check-recipes.sh.
• crates/placement/src/placement/nodes_scale.rs: a new clause, expected_sets, holds the inputs and then every plan to its cell, as above. The exact used and idle sets that clauses 4 and 7 listed move into expected_sets, and both clauses' doc comments say so; clause 4 keeps Qwen3.8 at N = 4, λ 70, A low among its link-idle cells, and clause 7 keeps the 4 % ruler case. Clause 5 takes its firing cells and clause 6 node 0's messages from the cells. Clause 4 adds that a node dropped for its link share would pass the tenth, read on the NodeIdle whole nanoseconds: the link is under one more nanosecond and the step at least its own, so placement: a model's plan over several unified nodes (dgx-spark.md §9), and nodes_scale on one to four Sparks #95's case of 10 of 106 ns still counts. Clause 5 adds the line itself: a NIC share of exactly one half reads 500 ‰ and does not fire, and one just past it also reads 500 ‰ and fires. nodes.rs is unchanged.
• docs/contrib/dgx-spark.md, as additions to the maintainer's text: §9.7 gains the (a) rows at the gate's inputs and a paragraph naming what is not recomputed; §9.8 gains "where each fires at the gate's inputs" and the edge paragraph; §9.9 names the script, the cells file and the edge rule; §10 says what the script recomputes and what stays open. §9.7's capacity rows for GLM at N = 1 and 2 and for V4.1 at N = 1 are struck through and corrected from the real files, and §9.7 says that the node plans refuse the V4.1 file (below).

What stays open in §10

The script recomputes §9.8's "where each fires" table and §9.7's (a) rows at the gate's inputs. §9.7's one-node and NVMe rows, (b), (c) and N requests in flight keep their first inputs: they need ρ, the NVMe rate, a pipeline model, the union R(2) and a batched step, none of which is an input of the script. The §10 item stays open for those. §10 also gains an open item: a fill term in the node plans for rows read from the NVMe tier, which the V4.1 file needs (below).

V4.1 and GLM on two Sparks, from the files' headers

plan_nodes on node 0's card plan of each real file, a 115 GB pool a node, at 4,096 and 32,768 positions, with the test below. The runs were at 8cb76e03, an earlier version of this commit with the same nodes.rs (What I ran, below). The GLM rows ran on a machine that holds that file. They give the same owned, room and paged counts, idle sets and pools as a first read at d29e9aec; only a predicted step or link time differs, by 1 ns in some cells (#95's later change to exact picosecond sums, rounded down).

The machine does not hold the V4.1 file, and the test reads only headers, so the V4.1 read used the file's header bytes in files of the full size. The node plans refuse that file at every N, before plan_nodes runs. Node 0's card plan leaves V4.1's two engram_embd tables (84.5 GB) on the NVMe tier, a token reads 5,280 B of them, and Terms::of refuses a row read from the NVMe tier by name (blk.1.engram_embd.weight). That refusal is #95's fix for #127, for the reason #95's section "Row tables on the NVMe tier" gives. The first read at d29e9aec came before it and counted those reads as no bytes, so its V4.1 plans at N = 2 and 3 left a gathered NVMe read a token out of the step, and this PR does not use them. §9.7 now says that the node plans refuse the file, and §10 lists the table fill term that would replace the refusal. NodeBudget::of does not go through Terms::of. It gives node 0 a room of 154 ids at 4,096 positions and 153 at 32,768, so 230 and 231 of the 384 ids do not fit one Spark. That is the only V4.1 figure §9.7 takes from the real file.

File	N = 1	N = 2	N = 3
GLM-5.3-Flash UD-Q4_K_XL (288 ids)	pages 128 ids, 82.4 GB	pages 0: room for 160 and 176 ids, split [119, 169] at λ 25 and [130, 158] at λ 70	pages 0
V4.1 Q3_K_M (384 ids)	refused by Terms::of; node 0's budget has room for 154 ids (4,096) and 153 (32,768), so 230 ids, 155.0 GB, and 231, 155.7 GB, do not fit	refused by Terms::of	refused by Terms::of

The synthetic shapes agree for GLM, which pages 0 at N = 2. The synthetic V4.1 shape has no engram tables, so the gate still plans it: §9.9's clause 2 and §9.7's (a) rows, which page 62 to 67 ids at N = 2.

The test that read the real files (not in this PR)

A throwaway integration test in crates/model/tests/, run with SCRATCH_V41 or SCRATCH_GLM set to the first shard and cargo test --locked -p bloomery-model --release --test scratch_nodes_real -- --nocapture. It builds node 0's card plan with plan_with on a GB10 spec, a remote's from routed_only, reads NodeBudget::of and Terms::of from them, and prints plan_nodes at N = 1 to 4, λ 25 and 70 µs, paged and whole. It asserts nothing. The V4.1 run stops at Terms::of, as above. A second throwaway test, with the same SCRATCH_V41, prints how node 0's plan places each tensor role and reads NodeBudget::of; it gave the 84.5 GB, the 5,280 B and the rooms named above.

//! Scratch, not in the tree: the node plans of the real GLM-5.3-Flash UD-Q4_K_XL and
//! DeepSeek-V4.1-Flash Q3_K_M files on synthetic GB10s, through #95's plan_nodes. Prints; asserts nothing.
//! Inputs: SCRATCH_GLM and SCRATCH_V41, each a first shard.

use model::arch::{deepseek41, glm5next};
use bloomery_placement::placement::{
    KvBytes, Machine, ModelTensors, Node, Plan, Role, plan_with,
    devices::{DeviceInfo, spec_of_device},
    nodes::{Fabric, LinkCost, NodeBudget, Nodes, Terms, Workload, plan_nodes, routed_only},
    workstation::{host_of, plan_on, unified_of},
};
use gguf::Split;

// The gate's GB10 figures (workstation.rs, test-only there).
const GB10_POOL: u64 = 115_000_000_000;
const GB10_READ: u64 = 208_800_000_000;
const GB10_NIC: u64 = 23_700_000_000;
const RULER: u64 = 10;

fn gb10() -> bloomery_placement::placement::workstation::CardSpec {
    spec_of_device(&DeviceInfo {
        ordinal: 0,
        name: "NVIDIA GB10".to_string(),
        total_bytes: 130_661_761_024,
        free_bytes: GB10_POOL,
        uuid: [0x10; 16],
        pci_bus: "000f:01:00.0".to_string(),
        held_by: None,
        integrated: true,
    })
}

fn nodes(n: usize, layers: usize, lambda: u64) -> Nodes {
    let spec = gb10();
    let mut machine = plan_on(spec, layers);
    machine.nodes.extend((1..n).map(|_| Node { host: host_of(&spec), unified: unified_of(&spec) }));
    let link = LinkCost { round_trip_ns: lambda };
    let fabric = match n {
        1 => Fabric::Alone,
        2 => Fabric::Pair { link },
        3 => Fabric::Ring { link },
        _ => Fabric::Switch { link, hop_ns: 1_500 },
    };
    Nodes { machine, fabric, read: GB10_READ, nic: GB10_NIC, ruler_permille: RULER }
}

struct NoKv;
impl KvBytes for NoKv {
    fn layer_bytes(&self, _: usize, _: u64) -> u64 { 0 }
}

fn report(name: &str, model: &ModelTensors, n_embd: u64, kv: &dyn KvBytes, ctx: u64) {
    let machine: Machine = plan_on(gb10(), model.layers);
    let plan: Plan<'_> = plan_with(model, &machine, ctx, kv, None).expect("today's plan");
    let remote = routed_only(model);
    let rplan = plan_with(&remote, &machine, ctx, &NoKv, None).expect("remote plan");
    let need = plan.pool_need().expect("pool need");
    let b0 = NodeBudget::of(&plan).expect("budget 0");
    let b1 = NodeBudget::of(&rplan).expect("budget 1");
    let terms = Terms::of(&plan, n_embd).expect("terms");
    let id: u64 = terms.layers.iter().map(|l| l.expert).sum();
    let routed: u64 = model.tensors.iter().filter(|t| t.role == Role::RoutedExperts).map(|t| t.file_bytes).sum();
    let mut by_role = std::collections::BTreeMap::<String, u64>::new();
    for t in &model.tensors { *by_role.entry(format!("{:?}", t.role)).or_default() += t.file_bytes; }
    println!("== {name} ctx {ctx}: layers {} routed layers {} experts {} k {} n_embd {n_embd}", model.layers, terms.layers.len(), model.experts, model.experts_used);
    println!("   file by role: {by_role:?}");
    println!("   routed file {routed} B, id over routed layers {id} B, ids x id {}", id * model.experts);
    println!("   terms: serial {} B, shared {} B, expert per layer min {} max {}", terms.serial,
        terms.layers.iter().map(|l| l.shared).sum::<u64>(),
        terms.layers.iter().map(|l| l.expert).min().unwrap(), terms.layers.iter().map(|l| l.expert).max().unwrap());
    println!("   node 0 plan: pool need cards {} host {} pool {}; violations {:?}", need.cards, need.host, need.pool, plan.violations());
    println!("   budgets: node 0 fixed {} pool {} room {} ids; remote fixed {} pool {} room {} ids",
        b0.fixed(), b0.pool(), (b0.pool().saturating_sub(b0.fixed())) / id,
        b1.fixed(), b1.pool(), (b1.pool().saturating_sub(b1.fixed())) / id);
    for n in 1..=4usize {
        for lambda in [25_000u64, 70_000] {
            let nd = nodes(n, model.layers, lambda);
            let budgets: Vec<NodeBudget> = (0..n).map(|j| if j == 0 { b0.clone() } else { b1.clone() }).collect();
            for whole in [false, true] {
                let w = Workload { prompt_rate: 3_000, whole, fills: None };
                match plan_nodes(&nd, &terms, &budgets, w) {
                    Ok(p) => {
                        let rooms: Vec<u64> = budgets.iter().map(|b| b.pool().saturating_sub(b.fixed()) / id).collect();
                        let paged: Vec<u64> = p.owned().iter().zip(&rooms).map(|(o, r)| if whole { 0 } else { o.saturating_sub(*r) }).collect();
                        let pr = p.predict().expect("predict");
                        println!("   N {n} lambda {} whole {whole}: owned {:?} rooms {:?} paged {:?} ({} ids, {} B) idle {:?} step {} ns link {} ns pools {:?}",
                            lambda / 1000, p.owned(), rooms, paged, paged.iter().sum::<u64>(), paged.iter().sum::<u64>() * id,
                            p.idle(), pr.step_ns, pr.link_ns, pr.per_node.iter().map(|l| l.pool).collect::<Vec<_>>());
                    }
                    Err(e) => println!("   N {n} lambda {} whole {whole}: refused: {e}", lambda / 1000),
                }
            }
        }
    }
}

#[test]
fn scratch_nodes_real() {
    let room = (124_000_000_000u64, bloomery_placement::placement::workstation::HostRead::Given);
    if let Ok(p) = std::env::var("SCRATCH_GLM") {
        let split = Split::open(&p).expect("GLM split");
        let inputs = glm5next::place::PlanInputs::describe_in(&split, room.clone()).expect("GLM inputs");
        let terms = inputs.seq_terms(&inputs.kv, None);
        for ctx in [4096u64, 32768] {
            report("GLM-5.3-Flash UD-Q4_K_XL", &inputs.model, inputs.hp.n_embd as u64, &terms.slots_of(1), ctx);
        }
    }
    if let Ok(p) = std::env::var("SCRATCH_V41") {
        let split = Split::open(&p).expect("V4.1 split");
        let inputs = deepseek41::place::PlanInputs::describe_in(&split, room.clone()).expect("V4.1 inputs");
        for ctx in [4096u64, 32768] {
            report("DeepSeek-V4.1-Flash Q3_K_M", &inputs.model, inputs.hp.n_embd as u64, &inputs.kv, ctx);
        }
    }
}

What the derived table says

At the gate's inputs (--table, now in §9.8):

Model	Decode step (ms), N = 1 → 2 → 3 → 4, λ 25 / 70, A low	Criterion 2	Criterion 7	Criterion 5, S₁ 3,000
V4.1	38.4 → 30.6 / 31.4 → 27.7 / 29.0 → 26.3 / 27.7	never	never	N = 2, λ 25, A low; N = 3; N = 4
GLM	67.4 → 55.2 / 56.2 → 50.9 / 52.3 → 48.7 / 50.3	never	never	N = 2; N = 3; N = 4
Qwen3.8	24.2 → 21.8 / 22.8 → 21.0 / 22.3 → 21.0 / 22.8	N = 4, λ 70, A low	N = 3, λ 70, A high; N = 4, λ 25, A low and high; N = 4, λ 70, A high	never

Nothing fires at S₁ 1,000. At a 4 % ruler, Qwen3.8 leaves node 1 idle at N = 2 at λ 70 only. §9.9's M7 names no λ; at λ 25 node 1 gains 10.1 % at A low and 8.4 % at A high, past the 8 % line, and is used.

What I ran into

None of this blocks the PR.

• A continuous derivation cannot decide a cell on a line. Qwen3.8 at N = 3, λ 70, A low has its link share 44 ppm under the tenth, with a one-id band of 78 ppm. Which side the planner's integer split lands on depends on which id moves. So the cell is an edge, its side stays placement: a model's plan over several unified nodes (dgx-spark.md §9), and nodes_scale on one to four Sparks #95's pin, and the band is an extra check.
• The budgets are inputs, not derived. fixed0 and fixed1 are today's card plan of each shape. Deriving them would mean a second card planner. If the card plan's terms move (the KV, a reserve), expected_sets goes red on the input row first and names it, before any cell moves.
• §9.8's first table moved in criterion 5. It assumed an even split of the NIC's rows. With node 0's real share, GLM fires at N = 2, and V4.1 fires at N = 2 only on the line.

Code shape

• No second copy. The script is the second derivation that §9.9 asks for, and it shares no code with nodes.rs; the three definitions it shares on purpose are named above. The gate reads its constants and budgets once and holds them to the input rows.
• Model code holds only architecture facts. No model file changes.
• Dead code is gone. The hard-coded cell lists in clauses 4, 5 and 7 are replaced by the cells and expected_sets.
• No silent failure. The script refuses by name a malformed input row, a missing, unknown or repeated row, a kind outside the three, a value not in ASCII digits, and an input outside its domain (exit 2, in every mode, --self-test among them). The gate panics by name on a missing input row, a cells file of another header or row width, a cell it cannot find, and an edge with no pin.
• Comments state what is true now.
• No relaxed gate. placement: a model's plan over several unified nodes (dgx-spark.md §9), and nodes_scale on one to four Sparks #95's pins stay: the edge cells keep their sides in SIDES, clause 5 still says "exactly", and the bands are checks added beside them. No clause lost a check.
• cargo fmt and cargo clippy are clean on what I touched; the lint count did not go up.

Duplication found or removed (path:line), or "none": none.

What I ran

On an Apple M5 (32 GB), just mac-static 54ceee74 and just mac-test at a515bc44, on main at 54ceee74:

$ just mac-static 54ceee74
mac-static: fmt-check rc 0 in 11 s
mac-static: lint rc 0 in 30 s (warnings 41 = the ratchet 41)
mac-static: combos rc 0 in 2 s (0 of 42 commands to run: no input changed since 54ceee74c)
mac-static: check-arch rc 0 in 19 s, check-comments rc 0 in 20 s, check-defaults, check-levers,
            check-loads, check-ptx-names, check-records, check-rustflags, check-unsafe, check-waits: rc 0
mac-static: check-recipes rc 0 in 126 s
mac-static: ok
$ just mac-test
test result: ok. 168 passed; 0 failed; 0 ignored (bloomery-placement)
mac-check: test rc 101; 12 crates (crate=rc): bloomery-decision=0 bloomery-hf=0 bloomery-jinja=0 bloomery-levers=0 bloomery-models=0 bloomery-placement=0 bloomery-refset=101 bloomery-runtime=0 bloomery-sampler=0 bloomery-serve=0 bloomery-tokenizer=0 bloomery-vision=0; 1201 passed, 1 failed, 179 ignored in 43 test binaries

check-recipes ran three blocks at a time on this Mac (its roughly 35 blocks started at once overload four performance cores and miss the timing bounds inside several self-tests). An earlier run at 5ba09875 on a second Mac, with every block started at once, was red in four: the depth-serve runner's stub tests, the gate-batch self-test, the gpu-gate queue self-test and the review runner's self-test. Each passed when run alone on the same commit, and the run above is green in all of them.

The one red is bloomery-refset's a_plain_target_with_no_draft_beside_is_the_family_path: the temporary directory comes back once with and once without macOS's /private prefix. Neither this PR nor #95 touches crates/refset, and on earlier runs the same test was red on the same Mac at main's acfb220a, 5c98cc70, ce422577 and baeedd73.

The script at a515bc44 (check-recipes above also runs its --self-test):

$ python3 tools/flow/nodes_scale.py --self-test
self-test: ok
$ python3 tools/flow/nodes_scale.py --check

FAIL-first, the script. One mutant at a time in a copy of nodes_scale.py, then --self-test. All 31 go red on a hand-worked case; most also on --check:

Mutant	Red on the hand case
no further post	a ring's layer
no rows ahead of a second remote	a ring's layer
λ = 0	the switch's hop at N = 4, Qwen3.8 idle for the link on four
no switch hop at N = 4	the switch's hop at N = 4
an even split, no balance	plan()'s balance
node 0 without its shared read	a pair's layer, plan()'s balance
the mean window for the slowest	a pair's and a ring's layer, the gain edge
no paging bound	the paging bounds, a room-0 remote
a slot overwrites the tuple's probability	the three tuple cases, both layers
the NIC on decode only	the prompt-dominated NIC, the NIC edge
the NIC on the prompt only	the decode-dominated NIC
node 0's NIC without the remotes' rows	node 0 takes every remote's rows
the prompt without node 0's Sh	the prompt's slowest FFN with Sh
the link line at 20 %	the link edge, Qwen3.8 idle for the link on four
the gain line at one ruler	a gain between the ruler and twice it
a room-0 remote taken without its gain	a remote of room 0 must gain twice the ruler
no remote idle for its pool	a remote past its pool
no refusal of node 0 past its pool	node 0 past its pool, by hand and in the inputs
no remote-to-remote move in the tolerance	the one-id moves
no link edge	the link edge
no gain edge	the gain edge
no NIC edge	the NIC edge
one message a remote, not two	V4.1's messages on four Sparks
a node's paged ids without its room	per-node paged ids sum to the total
no duplicate check, no kind check, digits of any script, top-k unchecked, ruler unchecked, a zero rate unchecked	the matching refusal (each fixture changes one condition of the real inputs)
the doc's tables unchecked	a doc table with a cell changed, a row more, or a row less

FAIL-first, the gate. One mutant at a time in the tree at a515bc44, then the nodes_scale clauses (cargo test -p bloomery-placement --lib nodes_scale) on the same Mac. Every mutant goes red; with none, all ten clauses pass.

Mutant	Red
clause 4's check on a dropped node read as 10 * link_ns > step_ns	link_share, on #95's case of 19,199 B ids (10 of 106 ns)
the script: criterion 2's line at 10.3 %, cells rewritten (--check rc 1 first)	expected_sets (Qwen3.8 at N = 4, λ 70, A low: the nodes used and idle), messages_o_n
the script: criterion 5 fires from 50.2 %, cells rewritten (--check rc 1 first)	expected_sets (V4.1 at N = 2, λ 25, A low: the derived NIC side is not the pinned one), nic_not_max
a cell: V4.1 at N = 2, λ 25 pages [21, 41] per node, not [11, 51]	expected_sets (paged per node)
nodes.rs: after the split, one id from the first remote to node 0	expected_sets (V4.1's step out of its band), pool_per_node, read_balance, link_share, nic_not_max, never_slower, fills_balance
nodes.rs: after the split, one id from the second remote to the first	expected_sets, pool_per_node, messages_o_n, fills_balance
nodes.rs: the link line at 20 % (link.times(5) > b)	expected_sets (Qwen3.8 at N = 4, λ 70, A low), link_share, messages_o_n
nodes.rs: the gain line at one ruler	expected_sets (Qwen3.8 at N = 4, λ 25, A low), never_slower, nic_not_max, messages_o_n
nodes.rs: no further post in the step	expected_sets (V4.1's step at N = 3 out of its band), nic_not_max, messages_o_n
nodes.rs: nic_bound fires on the floored permille (nic_permille(0, step) > 500)	nic_not_max
nodes.rs: node 0 reports no messages	messages_o_n (node 0's messages against the cell's)

Hardware (if the change is for a machine)

No term was measured for this PR; the inputs are §9.7's and §9.1's, with their kinds in the inputs file. The checks ran on an Apple M5 (32 GB). The real-file reads ran on one DGX Spark (GB10, driver 595.84, Ubuntu 24.04.5, kernel 6.17.0-1032-nvidia, 128 GB unified, nightly-2026-08-28) and used GLM-5.3-Flash UD-Q4_K_XL (six shards, 199.7 GB, sha256 checked) and the headers of V4.1-Flash Q3_K_M (nine shards, header bytes only, in files of the full size).

AI help

AI 코딩 에이전트가 코드를 작성한 후 설명을 썼고, 테스트를 마쳤습니다. diff 리뷰와 PR 초안 리뷰는 제가 직접 했습니다.

[github-actions]

Verdict: MUST_FIX: the oracle uses a remote with zero room and zero gain, and stores an unused field.

Must fix

1. Apply link and gain checks when a remote has zero room

tools/flow/nodes_scale.py:359 bypasses both objectives whenever the existing pools page, even when the remote adds no room. The fixture at tools/flow/nodes_scale.py:584 has four experts, top-k 1 and rooms [3, 0]. Both steps are 10 seconds, yet cell() uses the remote and the self-test requires it. This is a reachable kind A failure: the oracle accepts a zero-gain node contrary to criterion 7. The planner requires positive remote room for its capacity exception at crates/placement/src/placement/nodes.rs:710.

Correct this before the wrong self-test expectation becomes the accepted contract. Replace the capacity branch with:

        if c.room(m) < E and c.rooms[1] > 0:
            m, now = m + 1, nxt
            continue

Replace the checks at tools/flow/nodes_scale.py:585 with:

    expect("a room-0 remote has paging bounds and fits its fixed-byte budget",
           c3.rooms == (3, 0) and not c3.pool_idle and c3.limits(1) == [(0.75, 1.0), (0.0, 1.0)])
    _, m, idle, _, _ = cell(mc, dict(sh, fixed1=95), 2, 0, 10)
    expect("a remote of room 0 must gain twice the ruler", m == 0 and idle == [(1, "gain")])

2. Remove the unused node-count field

tools/flow/nodes_scale.py:236 stores self.n, but it has no reader. The constructor uses the local n to choose the hop. This is an unneeded element, kind B. Deleting it requires one line and no caller changes:

        self.mc, self.sh = mc, sh

Checked

Duplicates: the base contract §9.9 requires a separate derivation; no duplicate finding retained.
Complexity: the changed Rust module is test-only; no runtime hot path changes.
Silent failure: zero room and fixed bytes beyond a pool have explicit branches; the zero-room branch above skips the objectives.
Tests: the existing fixture reproduces the zero-gain selection. With the proposed fixes applied in memory, the full Python self-test and generated-table check pass. Without the room guard, the corrected fixture fails.
Gates: the fixes preserve the generated cells and both checked doc tables. All three inline suggestions target added lines and compile together.
Escape catalog: no pattern matched. No ledger lessons are in force.

Notes

Rust tests were not rerun during validation.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[github-actions]

A remote with zero expert room adds no resident capacity and must pass the link and gain objectives; also correct the room-zero self-test expectation.

Suggested change

	if c.room(m) < E:
	if c.room(m) < E and c.rooms[1] > 0:

[github-actions]

This fixture has equal one-node and two-node steps, so a remote with zero room must be idle for gain after adding the room guard.

Suggested change

	expect("a remote of room 0 the pools need is used", m == 1 and idle == [])
	expect("a remote of room 0 must gain twice the ruler", m == 0 and idle == [(1, "gain")])

[github-actions]

The stored node count has no reader; the constructor only needs the local n.

Suggested change

	self.mc, self.sh, self.n = mc, sh, n
	self.mc, self.sh = mc, sh

[juliankang4]

Both findings hold, and both are fixed in c7eaa014, as suggested.

1. The capacity branch in cell() now takes a remote only where the pools of the nodes before it cannot hold every id and the remote has room of its own (c.rooms[1] > 0), the rule idle_why keeps in nodes.rs. A remote of room 0 goes through the link and gain lines like any other. The self-test case now expects the room-0 remote of that fixture to be idle for its gain, and with the guard removed the case turns red.
2. Case no longer stores self.n.

tools/flow/nodes_scale.py --self-test and --check pass at c7eaa014, so the cells file and the doc's tables are unchanged and the nodes_scale gate reads the same cells. The mutant table in the body has the new row.

[github-actions]

Verdict: MUST_FIX: Remove two added parameters that no caller varies.

Must fix

1. Remove the unused path and ruler parameters

tools/flow/nodes_scale.py:166 and tools/flow/nodes_scale.py:435: The three load() calls at lines 514, 643 and 732 use INPUTS. The four where() calls at lines 454–457 use the enclosing r; only rate varies. The two parameters add configuration the change does not use. This is kind B under “fewest elements”.
Remove them while every caller already uses the fixed values. No caller needs an edit.

Replace load with:

def load():
    try:
        with open(INPUTS, encoding="utf-8") as f:
            text = f.read()
    except OSError as e:
        raise Refused(f"{INPUTS}: {e.strerror}") from None
    return inputs(text, INPUTS)

Replace the nested where in blocks with:

    def where(model, pick, rate=s1_hi):
        """The cells `pick` names, by N; an N alone is every λ and A at it."""
        hits = []
        for n in range(1, 5):
            each = [(lam, [a for a in ("low", "high") if pick(at[(model, a, n, lam, r, rate)])]) for lam in (lo, hi)]
            if all(len(ends) == 2 for _, ends in each):
                hits.append(f"N = {n}")
                continue
            hits += [f"N = {n}, λ {lam}, A {' and '.join(ends)}" for lam, ends in each if ends]
        return "; ".join(hits) or "never"

Checked

Scope: both parameters are added in this full review. The revert and no-merge mirror exclusions do not apply.
Contract: base §9.9 requires the independent derivation. Removing the parameters preserves it.
Silent failure: load converts an OSError into a named Refused error; both replacements preserve input validation.
Tests: the current script and the exact replacements compile and pass all 55 self-test checks, including the cells fixture and documentation --check. The replacements were tested in memory.
Escape catalog: C1–C4 do not match the parameter finding. No ledger lessons are in force.

Notes

Rust tests were not rerun locally.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[juliankang4]

Fixed in 974c99de, as suggested. load() reads INPUTS and the nested where() reads the enclosing ruler, so neither takes a parameter no caller sets. No caller changed.

tools/flow/nodes_scale.py --self-test (55 checks) and --check pass at 974c99de, so the cells file and the doc's tables are the same as before.

[github-actions]

Verdict: MUST_FIX: one unneeded element in the gate (fires is d.fires under another name); no behavior defect found.

Must fix

1. fires() adds a branch that changes nothing

nodes_scale.rs:611 fires(d) returns side(d, Edge::Nic) on an edge cell and d.fires otherwise. side returns the derived d.fires, after it asserts that the derived side equals the pin (L573). So fires(d) == d.fires on every input. expected_sets already calls side on every edge (L754) and compares the pinned set to the derived edges, so the pin check does not need a second caller.
Cost: a reader takes fires for the place where the pinned side wins over the derivation. It is not. The bool return of side exists only for this helper.
Fix: delete fires, make side return () (drop the fires binding's let result, end with the assert_eq!), and at L1367 write:

if derived_at(cell, RULER, rate).fires {

Non-blocking

⚠ Clauses 4 and 7 lost their local exactness: nodes_scale.rs:1590 never_slower now collects only Qwen3.8 at N = 2 and asserts it is empty, and link_share uses fired.contains(..) instead of equality. A planner change that adds a gain-idle or link-idle cell is still caught, but only by expected_sets. Say so in the two doc comments, or compare fired with the derived idle set, so each clause names its own failure.

⚠ + 2 in step_tol, nic_tol and the edge tol (nodes_scale.py:366, L407) is a band widening with no stated reason. Name it as the rounding slack of the picosecond sums in one comment line.

⚠ self_test calls check() (nodes_scale.py:718) outside the try in main. A bad inputs file ends --self-test with a traceback and exit 1, not the named refusal and exit 2 of the other modes. Catch Refused in main around self_test() too.

Checked

• Duplicates: 14 new items checked (input, Derived, Edge, side, fires, past_tenth, idle_of, nic_bytes, nic_prompt, the script's functions). Dryer: 0 pairs. The TSV split in qwen38_cards.rs:154 reads a different file; no shared owner worth a helper. One redundant element: finding 1.
• Complexity: crapper flagged nothing.
• Silent failure: 6 sites followed (input parse, cells header and width, unknown shape, unknown edge, missing derived cell, SIDES without a pin). Each panics by name. The script refuses bad inputs with exit 2.
• Tests: mutation check on nic_not_max's new half-line case. A > to >= flip in nic_bound turns at(line) red; dropping the at(line - 1) branch turns it red. expected_sets goes red on a one-id move in the split (per the PR's own mutant table, not rerun here).
• Gates: no band, tolerance or KNOWN_DIVERGENCE value moved. The new tolerances are checks beside the pins. The removed explicit lists in clauses 4, 5 and 7 moved into the cells file.
• Hot path: none (test and tool code).
• Escape catalog: none matched (no shell added).
• Mac-only run: the placement tests and --self-test ran at 9e5429cf, not at the head; the PR body says so.

Notes

• The PR body has no gate run on a card per CONTRIBUTING.md (no just gate-* output with both commit hashes). This does not change the verdict.
• The gate mutants were not rerun at the head 974c99de. The author states this.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[juliankang4]

Fixed in a515bc44 on main 54ceee74, now rebased onto main 84f77b01 as 4e480715 (the same patch in git range-diff; main changed no file under crates/ in between). The rebase onto 54ceee74 had one conflict, in §10 of docs/contrib/dgx-spark.md: the sentence on clause 1 keeps main's reason (plan_in_pool and node 0's terms taken before that split) beside this PR's words on the derived cells.

• fires is gone and side returns (). nic_not_max reads derived_at(cell, RULER, rate).fires, and expected_sets alone holds an edge's derived side to its pin; clause 5's doc comment now says so.
• Clauses 4 and 7 keep their checks, and their doc comments now say that expected_sets holds the link-idle and gain-idle nodes to the derived cells. A comparison of fired with the derived idle set in each clause would pin again part of what expected_sets pins in every cell (idle_of(plan) == d.idle, reasons included), so I left it out.
• The + 2 is one constant, SLACK, with its reason: the band truncated to a whole unit (under 1), the value rounded to the nearest (½), and the planner's value rounded down from sums of whole picoseconds (under 1 and a fraction), under 3 in all, so 2 whole units. No cell moved.
• main catches Refused around --self-test too. With the inputs' read row set to 2.5, --self-test at 974c99de ended in a traceback and rc 1; at a515bc44 it prints the named refusal and returns 2.

At a515bc44, on an Apple M5: just mac-static 54ceee74 passes, and just mac-test passes except bloomery-refset's /private path test, which is red on main too; the 168 placement tests pass. tools/flow/nodes_scale.py --self-test (55 checks) and --check pass. The gate mutants ran again at this head, and each goes red as the table in the body says.

[github-actions]

Verdict: CLEAN: No must-fix findings remain. The independent derivation preserves the existing cell pins and adds prediction-band and boundary checks.

Checked

Duplicates: §9.9 explicitly requires the independent verifier.
Silent failure: input parsing and domain checks produce named failures; missing Rust input rows, cells and edge pins panic by name.
Tests: Python --self-test passed, including the cells and documentation checks. Four in-memory mutants each returned 1: dropped remote post, overwritten probability, one-ruler gain threshold, and decode-only NIC demand.
Rust assertions inspected: expected_sets checks used/idle sets and prediction bands; link_share checks dropped-link witnesses; nic_not_max pins exactly half and just past half; messages_o_n checks node 0; never_slower rejects slower plans.
Gates: the embedded cells retain the base link, gain and NIC firing sets. Edge sides remain pinned in SIDES.
Escape catalog: no C1–C4 pattern matched the added code.
Performance: nodes.rs is unchanged; the diff changes tests, tools and documentation.

Notes

Rust tests were not rerun here. The PR reports 168 placement tests passing on an earlier commit and one refset failure. The original item counts and dryer/crapper results were not independently verified. No NVIDIA gate run with the required evidence is supplied.

_{🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)}

[juliankang4]

The review is done and the contribution check passed, so I'm merging this.