Skip to content

Hook ABI v5 goldens for units, anatomy, mid and the S lens; population artifacts, bf16 norm fix, --dtype - #70

Merged
a9lim merged 14 commits into
mainfrom
browser/units-goldens-v5
Nov 2, 2026
Merged

a9lim merged 14 commits into
mainfrom
browser/units-goldens-v5

Conversation

@a9lim

@a9lim a9lim commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Python goldens for drowse.ai's hook ABI v5 (units, token anatomy, the mid site and the S lens on WebGPU), plus the small contract and engine changes the browser parity plan needs.

Goldens

tests/support/browser_units.py owns the units-tiny-models-v1 fixture: gemma3_text and qwen3, 4 layers, d = 64, d_ff = 96, 4/2 heads × 16. Every weight comes from a portable 32-bit integer generator, so drowse.ai can rebuild the tensors bit for bit, and every linear and the embedding is q4f32_1-exact (power-of-two group scales with a ±7 in each group). The module has one build_<golden> per file and write_goldens, which enforces the validator and the size caps. Regenerate with PYTHONPATH=. python -m tests.support.browser_units --write.

New files under tests/fixtures/browser/. There is one model file, plus one file per arch for each of: block-sites, unit-stats, unit-ablation, unit-probes, unit-activity, lens-reads, slens, token-anatomy, attention-patterns, unit-reads, unit-artifacts and neuron-push. That is 25 files, 1.85 MB in total, the largest 191 KB. The caps are 256 KiB per file and 2 MiB overall. No existing golden changes.

unit-probes records one entry per forward (prefill and decode steps) from the unit tap's ring, with a gated case on a gate-only unit and the gate scalars the decode loop saw. unit-activity covers generated, sink, sink-population, authored (a user turn's prompt capture), two-spans, all-excluded, excluded-and-zero and segmented. lens-reads includes S direction reads (uncentred) and a session with no S lens. slens records the lowered 0.3 slens/<word> push, and token-anatomy carries each case's intermediates.

Every value comes from drowse's own machinery: tap_token, the real steering scope and decode loop, UnitHook/UnitOp, NodeReplayer (unit_activity, token_anatomy, _activity for the hand-built forwards), UnitReader, the lens instrument's reads and the S embedding statistics.

test_units_golden_matches_python rebuilds each golden from a fresh fixture and compares it within the file's own tolerance block. The model file must regenerate byte for byte, and its q4 round trip is asserted.

The validator refuses non-finite values, a sink row that moves under unit ablation, an ablation that moves no logit, a saturated attention row, and every top-k boundary without a 1e-3 margin. Each ranked list (lens reads, unit reads, unitTop hits, footprint top-k, the anatomy's unit lists and pattern heads) is read again one deeper: a changed returned set or a k-th entry that ties the best excluded refuses, and a read moves to another k (recorded in the file) until its boundaries clear. Probability lists rank by ln p, the logit gap their order rests on. unitFind records each greedy pick with its runner-up and refuses a pick without the margin. Every greedy decode pick must be decisive, generated nodes decode distinct tokens (greedy with a presence penalty), and unit activity refuses a listed or traced unit whose peak row has no margin (outside the declared excluded-and-zero case) and a traced neuron that keeps one sign.

Goldens hold across platforms: the python tolerance is the library's (1e-5 + 1e-4|x|). When writing, every margin also clears twice that tolerance, and the parity test rebuilds on the file's recorded choices (each read's k, a case's top, the anatomy's knobs, the gate thresholds; build_golden(..., replay=)), so a rebuild on another platform never re-decides near a threshold.

A near tie inside a list does not refuse: it is a run in declared_ties ({path, key, groups}). The comparison matches a run's entries by identity and compares every field (ids, strings, values), with order kept between runs. Unit-activity peak rows without a margin are skipped by index, and float16 populations allow one ulp.

Engine and contract

  • contracts.ts: StructuredHookProfileId gains standard-v4 and standard-v4-residual. standard-v1–v3 stay until drowse.ai's libraries are recompiled. CatalogInstrumentPack.kind is now core | jlens | slens | units | sae.
  • UnitReader.population_artifacts(layer) returns, per layer, what the browser ships offline: the population scalars (R rho/lse/lse_neg, S norm/footprint_std), the contrast statistics (S fires pooled), the heads' OV contrast, the head-norm mean derivatives and the whitened write norms. null_basis() and head_kept_ids() are now public, so drowse.ai's pack builder doesn't reach into privates.
  • _norm_spec now judges each convention relative to its gain. When both fit (every gain large), the module itself, called with a zero weight through torch.func.functional_call, answers the offset; the closer median is only the fallback. Before this, a bf16 module with gains of several hundred (gemma-3-270m's MLP output norms, up to about 1,650) failed the absolute offset gap even though 1 + w was clearly the better fit. The regression tests (bf16 gemma3_text 1 + w and gemma4_text plain, large gains, and twelve all-large draws per arch) fail before this change.
  • --dtype float32|bfloat16|float16 on every model-loading verb. It reaches from_pretrained(dtype=) and the model_source_fingerprint that the lens-fit and manifold no-op proofs key on, so a bf16 fit is never proven as an fp32 one. The loaded model's stamp already carried the parameter dtype.

Checks

  • pytest -q -m "not gpu and not perf" -n 8: 6022 passed, 7 skipped
  • ruff check .: clean
  • pyright: 0 errors (the two remaining warnings are pre-existing)
  • scripts/generate_webui_types.py --check: current
  • cd webui && npm ci && npm run check && npm run build: Node 24.22.1, passes, no dist drift
  • GitHub CI (Linux, Python 3.11 to 3.13): green

a9lim added 14 commits November 1, 2026 22:20
…tention-patterns and unit-ablation golden builders
…ties by identity

The validator reads each ranked list one deeper and refuses a returned set
or a k-th entry without the margin to the best excluded (lens reads, unit
reads, unitTop, footprints, anatomy units and pattern heads); reads move to
another k and record it. Probability lists rank by ln p. Near-tie runs are
declared as {path, key, groups} and compared entry by entry, matched by
identity, with order kept between runs; unit-activity peak rows without a
margin are skipped by index, float16 populations allow one ulp.

unitFind records each greedy pick against its runner-up. Generated nodes
decode distinct greedy tokens with every pick decisive; unit-activity refuses
unmargined listed peak rows outside the tie case and one-signed traced
neurons, and gains the authored case. unit-probes records per forward from
the tap ring, with a gated case on a gate-only unit and the gate scalars.
lens-reads adds S direction reads and a session without an S lens; slens
records the lowered 0.3 slens/<word> push; token-anatomy carries every
case's intermediates. tests/units pins the top shortcut in all four cases.
The python tolerance becomes the library's (1e-5 + 1e-4|x|): Linux CI
differed from the M5 by up to 1e-5 on sites and logits. Writing, every
margin also clears twice that tolerance, and the parity test rebuilds on the
file's recorded choices (each read's k, a case's top, the anatomy's knobs,
the gate thresholds), so a rebuild elsewhere never re-decides near a
threshold. Unit-activity row margins compare at their peak's scale, and a
tuned top also skips listing an unmargined peak row.
# Conflicts:
#	AGENTS.md
#	tests/browser/test_browser_parity.py
@a9lim
a9lim merged commit 0966d4e into main Nov 2, 2026
12 checks passed
@a9lim
a9lim deleted the browser/units-goldens-v5 branch November 2, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant