Validate each input signature once instead of every port on every call

[martyn]

Summary

GraphModule._execute() used to check every port of every node, and replay the registered-state walk, on every forward call. That was a fixed ~6 µs per node on top of the modules themselves. This PR checks everything once per input signature, keeps the hot path close to a plain loop of module calls, and makes failures readable.

What is checked when

When	What runs
First call for a signature: each external input's (shape, dtype, device), plus training and autocast state	The full checked program: every port's shape, dtype and device before and after its node, then the registered-state check (same _state_matches replay as c78a43e). The signature is remembered (up to 64 per model, then the cache is cleared).
Later calls with a remembered signature	Build the signature tuple (~0.6 µs), a dict lookup, one integer compare of the watch generation, then the modules from a slot-indexed program. Single-input nodes call module(values[i]) directly.
Any registration hook fires during such a call	The state check runs at the end of that call, as before.

What invalidates it

• _apply (.to(), .cuda(), .double(), .half(), ...) clears every validated signature and forces a full state re-check before the next call.
• Registering state on any module of the graph. register_parameter, register_buffer, add_module/register_module, and attribute assignment all go through torch's global registration hooks (register_module_{parameter,buffer,module}_registration_hook). A hook filtered to modules of built graphs advances a watch generation, so unrelated modules constructed elsewhere cost nothing. On the next call the graph re-checks state against the build, clears its signatures (a parameter replaced under the same name can change any shape), and revalidates every port.
• deepcopy gives the clone a fresh cache and watches the clone's modules. load_state_dict needs nothing: it copies in place, and assign=True goes through setattr, so the hooks fire.

What the state check protects against (after reading c78a43e / 5b880dd)

c78a43e restored detection of: filling a declared-None parameter/buffer/submodule slot (the lazy-init bug), self.weight = None, and replacing or grafting a submodule. In this PR:

• During forward on the first call for a signature, all of these are caught exactly as before, with the same _state_matches walk. That's where lazy init happens.
• During a later forward, or between calls, anything that goes through a registration hook is caught: filling slots, grafting or replacing submodules, and None over a buffer or submodule. test_every_kind_of_state_mutation_during_forward_is_rejected passes unchanged.
• The gap (please review): PyTorch runs no hook for del module.x, for module.param = None (register_parameter(name, None) skips hooks), or for writes straight to _parameters/_buffers/_modules. If one of these happens after a signature has been validated, it is reported at the next full check (new signature, .to()/cast, any hooked registration) or as soon as a node then raises. The failure path always re-runs the state check, so del model["head"].weight gives a readable "parameter 'nodes.n_head.weight' was removed" error rather than a bare AttributeError. The only silent case is param = None on a module that tolerates it, e.g. linear.bias = None, which runs bias-free until the next revalidation. test_a_parameter_set_to_none_is_reported_at_the_next_revalidation pins this down. Closing it would need a per-call sweep of store.get(key) is not None over every registered tensor, about 50 ns each. I left that out to keep the hot path a straight loop, but it's a small follow-up if you want it.

Errors

Port mismatch after a validated call:

E_RUNTIME: input 'x': expected shape [B=3, 4], got [3, 5]
  expected [B=3, 4]:float32 on cpu
  got      [3, 5]:float32 on cpu

An exception raised inside a node propagates as itself, with the same type, message and traceback, so except RuntimeError and OOM handling keep working. It gains one add_note(), which the traceback prints under the message. The note is built only in the except path, so there's no per-call cost:

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2x4 and 3x3)
HNDL: raised inside node 'odd' (flaky, line 2, column 1)
  input 'x': got [2, 4]:float32 on cpu; contract [B=2, 4]:float32 on cpu

A deleted parameter (no hook fires) explains itself in the note:

AttributeError: 'Linear' object has no attribute 'weight'
HNDL: raised inside node 'head' (linear, line 3, column 1)
  registered state no longer matches the build: parameter 'nodes.n_head.weight' was removed
  input 'x': got [3, 8]:float32 on cpu; contract [B=3, 8]:float32 on cpu

An upstream node that changes its output after validation is named at the port it breaks, in the note on the downstream node's error (linear 'head' input 'x' (from node:d/out): expected shape [B=2, 4], got [2, 2] plus an explanation). An exception passing out through nested graphs (a built graph used as a node of another) keeps only the innermost node's note. Errors that come from dynamo or inductor get a line saying so. HNDLError/E_RUNTIME is kept for what HNDL's own checks find: first-call validation and registered-state mismatches.
Registered state changed:

E_RUNTIME (node n0; line 1, column 1): registered state of lazy 'n0' changed during forward: submodule 'nodes.n_n0.inner.proj' was filled in with Linear (it was None at build)
  HNDL fixes every parameter, buffer and submodule when it builds a plan, so state added or removed later is not trained, seeded or saved as the plan describes; create it in __init__, or resolve and build a new plan to change the architecture

Other details:

• A second input that disagrees on batch gets the hint B=3 is this call's batch size, read from input 'x'.

Behavior changes to review

1. Exception types are unchanged for anything raised inside a node; HNDL only adds a note. (The first revision of this PR wrapped them in HNDLError. That broke except RuntimeError callers and was replaced.)
2. Dtype spelling in messages: expected dtype float64, got float32 (HNDL notation) instead of torch.float64. The dtype assertions in tests/test_torch.py were updated for this.
3. Per-call guarantees: after the first call for a signature, a node whose output shape depends on data is not re-checked on every call. It's caught when a later node raises, and reported at the port. External outputs are not re-checked per call either.
4. Autocast: it's part of the signature, so the first autocast call is validated (and, as before, fails the dtype check). A signature validated without autocast is never reused under it.

Tests

• test_deepcopy_keeps_lookup_moves_and_the_state_consistency_check used to tamper with _state_program after a successful call and expect the next call to notice. With per-call checking gone that no longer holds, so I rewrote it to do a real mutation on the clone (clone["head"].ghost = nn.Parameter(...)). It now asserts that the clone reports it, that the original keeps running, and that the original still catches its own mutations.
• Dtype message assertions: torch.float64 became float64 (see above).
• New tests: once-per-signature caching and its invalidation (batch, training mode, .to(), and that unrelated module construction does not invalidate); a broken input after the first call (exact message); a second input's batch hint; a torch error inside a node on the validating call and on the fast path (type(...) is RuntimeError, exact note); OOM keeping its type; an HNDLError inside a node keeping its message and gaining a note; one note only through nested graphs; upstream output drift named in the note; five mutations after the first call (grafted submodule, added parameter and buffer set to None → HNDLError; deleted parameter → AttributeError with a state note; wrong-shaped parameter → RuntimeError), each reported on every call; the param = None gap; state registered during a later forward; and torch.compile(backend="eager", fullgraph=True) tracing the checked program with no graph breaks.
• Full suite after rebasing onto Keep 0.6.0 saved plans loading when operators gain arguments #81 and Relax strictness that costs more than it protects #82: PYTHONPATH=src python -m pytest -q -m "not network and not benchmark" gives 3586 passed, 80 skipped.

Benchmarks

CPU, torch.set_num_threads(1), torch 2.14. The machine was shared (load average 16–25), so each row is the best of 9 interleaved hndl/hand-written measurements, with ranges over 3 alternating old/new process runs. "Before" is origin/develop at 7f7cd3d.

Case	Before: hndl / hand	Before: ratio	After: hndl / hand	After: ratio	Overhead before → after
MLP (5 nodes), batch 1	53 / 23 µs	2.28–2.35x	28 / 22 µs	1.21–1.30x	30 → 5–7 µs
MLP, batch 32	80–92 / 46–49 µs	1.73–1.89x	51–54 / 44–47 µs	1.15–1.21x	33–43 → 7–9 µs
MLP, batch 256	207–210 / 173–175 µs	1.19–1.21x	183–185 / 172–177 µs	1.04–1.06x	33–37 → 7–11 µs
transformer_block, batch 8	269–279 / 205–210 µs	1.29–1.33x	243–247 / 207–210 µs	1.16–1.18x	61–69 → 34–38 µs
20× relu, batch 1	102–109 / 45–48 µs	2.27–2.29x	51–54 / 44–46 µs	1.17–1.21x	57–61 → 7–9 µs
conv_stack, batch 16	~5 ms	0.96–1.10x	~5 ms	0.97–1.6x (noise)	lost in ms-level noise

The fixed cost is now ~2 µs per call (the signature, a dict lookup and the output dict) plus ~0.35 µs per node. The ~35 µs left on transformer_block sits inside that single operator's own implementation compared with nn.MultiheadAttention, not in per-node validation.

pytest -m benchmark tests/benchmark -q -s passes before and after (28 passed, 1 skipped). Parity ratios from one run each: mlp 1.22x → 1.09–1.14x, conv_stack 0.74x → 1.06–1.07x (noise), transformer_block 1.13x → 1.08–1.09x. All compile-compat networks compile as before. moe_transformer is skipped in both runs by an inductor CppCompileError, which is environmental.

TOLERANCE: I did not change it. The MLP now runs ~1.2x at batch 32, so the parity suite could time the MLP at batch 32, the case this issue was about, and TOLERANCE could come down from 2.0 to about 1.5. I haven't done that because of the shared-machine noise on CI hosts. Your call.

Other notes

• The global registration hooks are installed once at import hndl.torch, for the whole process. Each is a WeakValueDictionary lookup that runs only on registrations, never in forward. Returning None means they never alter a registration.
• Under torch.compiler.is_compiling(), _execute traces the full checked program and state check exactly as before (dynamo turns them into guards), and skips the signature cache.
• The docs changes are the CHANGELOG bullet under ## Unreleased, README, IMPLEMENTATION.md, SPEC.md §3/§8/§11/§13, and the parity benchmark docstring.

🤖 Generated with Claude Code