Development

Language: concise to write, simple to read, consistent. A design rule is a plan on trial: discuss it when applying it makes a useful expression harder to read, removes something expressible or adds a special case; APL familiarity and compatibility aren't costs there. API change: change callers; no compatibility layers. Each kind of thing takes the general path: a system name resolves to a function, compiles as a glyph does.

Speed and simplicity before edge cases no real user meets. Follow IEEE, Rust and CPU behaviour where reasonable. Precision: rustymath DEV.md's "Overview".

A form with a reasonable meaning gives a result, not an error; a silent result that is a bug in some programs is fine if it buys useful behaviour or speed. Settled: negative positions count from the end in every selection (Python), though x.[i-1] at i=0 then reads the last item. ⌈ beside NaN: NaN or the other argument, whichever its kernel gives.

Dynamic kinds (default): the APL REPL experience most users want most of the time: machine integers that widen within machine widths and wrap at i64, explicit unbounded ᵣ integers/rationals, tolerant comparison. Static kinds: speed-sensitive work. Supporting both costs complexity, not static speed. Large work runs in notebook cells and scripts, not the REPL.

Typed programs and the JIT must end in fairly simple, very fast code; otherwise the approach is in question. Static kinds: integer programs should fuse, compile and run array-wise, by the simplest implementation that allows → one integer regime: ordinary integer = fixed integer of int width; overflow should wrap on every route (per-item checks and error paths out of fused or compiled loops cost speed and code); every integer operation should compute at its type's width. Each width's kernels cost compile time, binary size, code: add a kernel family only where its benefit outweighs that.

Big-data analysis runs keyed arrays of functions over keyed arrays of data → keyed and named arrays should fuse and compile as plain ones do; missing-item and fill semantics can change to make that fast and reasonably simple.

Not worth optimizing: laziness between cells; chains that rebind a name so earlier lines compute nothing. Pending arrays fuse large calls in code no trace holds. A view (window, transpose) stays a view until a call needs its items in memory.

Bare number = float (3; integer 3ₓ): a language rule, not an implementation one; an internal route may run a whole float as an integer where the result can't change.

Required first reading

Read nbs/llms.txt in full. Its published links may omit unreleased changes: use the corresponding sources under nbs/. Before writing BPL examples or tests, read nbs/composition.qmd. Check a very small non-trivial expression's linear form, result and shape through fresh before running it at full size.

The current plan is meta/eager-release-plan.md. Release 1 is eager basedpl, with its engine taken from basedpl 96ea793, running on current rustymath. The release's language and results are its default, plus the settled changes the plan lists, with one home per fact. The release's behaviour, edge cases and errors included, is a default rather than a requirement: where keeping a behaviour would add substantive complexity and a reasonable simpler design exists, discuss it with Jeremy first. meta/owners.md describes subsystem interfaces. meta/ records reasoning and evidence, not extra instructions or file leases. AGENTS.md holds the founding principles. nbs/target.qmd and nbs/foundations.qmd record the parked graph design and constrain nothing. Rustymath and basedpl are co-designed: use one mechanism per job and one home per fact, changing either project where that makes the combined system simpler.

Source, builds and both projects' meta/ directories are on Syd1 under /home/ubuntu/aai-ws/{basedpl,rustymath}. meta/ isn't in git. Mac copies are snapshots: never sync them over newer Syd1 work. meta/open.md lists open work and links the plan. Tag graph-parked in both repos holds the graph-repair tree. Reference worktrees live outside ~/aai-ws, because the uv workspace takes every sibling directory as a member and the generated cargo patches point at siblings:

/home/ubuntu/git/graph-refs/{basedpl,rustymath}: tag graph-parked.
/home/ubuntu/git/pre-rewrite-refs/{basedpl,rustymath}: basedpl 96ea793 and rustymath d39f693, the baseline. With json5 1.3.1 they pass all 8,035 reference cases. json5 1.3.7 parses integers past u64 as floats. Results: meta/pre-rewrite-baseline.md. Per-case costs: meta/pre-rewrite-costs.tsv.
/home/ubuntu/git/release-refs/{basedpl,rustymath}: v0.1.82 and v0.1.54.

Main coordinates connected work and publishes interface changes to consumers. One coordinator captures a coherent source pair for an infrequent useful build; writers and reviewers continue on the live trees. There are no file leases, global editing freezes or continuous rebuild watchers. The frozen pair lives in /home/ubuntu/git/integration/{basedpl,rustymath}, with basedpl's cargo config pointing at that rustymath. Use the documented snapshot/build helpers: snapshots include untracked files without moving live HEAD or the real index. The coordinator owns cargo develop, native checks and integration validation. Pin checks to the captured artifacts and fixture version in meta/release-validation.md; source review is not runtime proof. The plan's Process section records the current procedure.

Kernels use host="syd1"; each fork creates its own kernel and uses main's shared SSH forward. Review edits with journal(os.environ['EXHASH_JOURNAL'], author=<agent id>). Older kernels use /tmp/exhash-journal/mcp/<kernel id>.jsonl; list_kernels(host="syd1") maps their identities. Keep commits local unless the current shipping instructions request a push.

Reference evaluations keep their two-second deadline (reference::TIMEOUT). The worker's separate 30-second watchdog bounds work that doesn't poll cancellation. Every asserted value, kind, error and output gates, including cases that call •explain, •memory or •storage. Route-directive violations remain separately reported obligations; they never waive a case's assertions. Retain exhash inputs in a kernel variable. After a conflict, refresh addresses and preserve semantic intent. Use build= to pin checks to an exact build, and record its source snapshot and fixture version. Never shrink cases, introduce skips, relax budgets or alter expected mathematics to clear failures without Jeremy's agreement.

The reference below remains applicable. Read the relevant part when using its tools or changing its concern: accuracy and dynamic kinds for numerical work; build/test, routes and budgets for validation; cancellation and timing locks for jobs; browser and machine code for those targets. Do not reread unrelated tool catalogues at every handoff.

Development reference (on demand)

Internal guidance; no commitments. Founding principles: AGENTS.md; details: "Accuracy" and "Dynamic kinds" below.

First read nbs/llms.txt in full. BPL differs substantially from Dyalog APL in grouping, names, numbers and operators: write BPL, not assumed APL idioms. llms.txt links point at the published site, which lacks unreleased changes: read sources under nbs/ (nbs/glyphs/at.qmd for glyphs/at.html.md, nbs/indexing.ipynb for indexing.html.md). Before writing BPL for tests or examples, read nbs/composition.qmd.

Rustymath and basedpl have no outside users: treat neither as published. Machinery serving both basedpl and rustymath's Python API is shared; changing rustymath to share it is encouraged. Separate basedpl and rustymath machinery for one task is an error unless Jeremy asked for them to differ there.

The design and meta/ notes for work ahead:

eager-release-plan.md: release 1's plan. Its aims: about the same speed as the previous release, or better, with less code, clearer responsibilities and a better foundation for static types. Speed is measured against the baseline costs in meta/pre-rewrite-costs.tsv. owners.md: pieces, their files and the pinned interfaces between them. nbs/target.qmd: the parked graph design, constraining nothing. Its “Numerical algorithms” part records the approved numerical direction and the validation needed before documenting changed accuracy bounds.
open.md: every open item, and the checks for the closing pass.
foundation-reset.md: removals, released algorithms to reuse, deferred ideas, and the "Deletion checklist" of old code that the target's "Not" lists name and the tree still holds.
crate-split-2026-10-31.md: whether to split basedpl into crates: no split now; rebuild time by build step; engine-to-language references, recounted at the closing pass.

meta/archive/: notes the target replaces: the one-graph plan (one-graph-2026-10-31.md: reasons, decisions with their history, evidence) and its build record (one-graph-build-record.md), programs.md, jit-plan.md, graphreview.md, stage0-design.md, program-layer-2026-10-30.md, graph-ir.md, recorded.md, linear.md, binder.md, the stage, index-notation and design-audit notes; the stopped build's re-plan (gaps-2026-11-01.md) and design notes (design/). Other meta/ notes (build notes of old features, e.g. PRD.md): history only. closed.md: archived decisions and finished work, for revisiting an old decision.

Accuracy: nbs/numbers.qmd's "Accuracy": floats within 4 ulps of correctly rounded, except those rustymath's accuracy page lists; some results exact. Unpromised: last bits, behaviour of infinities and NaNs, zero's sign, and which value a write keeps at a repeated position. Last bits can differ by thread count, array size, platform, build, run, route and spelling (each can reorder a fold such as +/). FMA and reassociation can overflow differently near the range limits. A few ulps traded for real speed or simpler code become an exception documented on rustymath's accuracy page. Agents chasing identical bits or tighter-than-documented bounds have cost many rewrites. Never spend speed, memory or code on agreement between results, reproducibility, or accuracy past documented bounds, unless Jeremy agrees a real, common use needs it:

Don't make a fused, compiled or batched route copy another's fold order, block structure or kernel to match its bits; each route may regroup folds for speed.
Don't add reference cases, harness runs or tests comparing floats bit for bit across routes, thread counts or array sizes, or checking behaviour of infinities and NaNs or zero's sign; compare within the Numbers page's accuracy.
Don't count identical bits as a design benefit, or differing last bits as a regression.

Non-commitments don't license breakage: keep the exact results the Numbers page lists. Remove code existing only for agreement between results or accuracy past documented bounds, where removal cuts lines or adds speed without substantive complexity and breaks nothing else.

Dynamic kinds: autodiff and fast code target static kinds, branching and updating with approaches like Agenda (⍚), Under (f@g, f@[p]) or selective assignment (x.[p]←), not with dfn guards on traced values. Dynamic kinds serve interactive analysis: tacit code over dataset-sized arrays, and simple dfns over values such as dates and strings. A call settles when its kernel can't fail an item at its operands' ranges (plan::ranges: a held constant's own value, nonnegative where the operand is known never negative, otherwise any), as rustymath's Kernel::may_raise_on decides; that check is the fact's one home. Only settled calls defer and fuse. Integer arithmetic settles: it computes at the narrowest width that holds every result (u8+u8 → i16) and wraps at i64. √ * ⍟ ○ settle only where the ranges meet the domain. A call that doesn't settle runs eagerly as one whole-array call and never defers; on a complex result, the whole call runs again, complex. The ₐ forms are the fast path for real-domain math. Dfn guards on traced values run correctly and aren't speed targets. The speed tests keep dynamic-kinds cases, so costs stay known and new slowdowns fail. A slow event from a value-dependent type under dynamic kinds: allows (known cost).

Build and test:

cargo develop: rebuilds and installs the native extension and bpl with the release profile, which development, tests and timings all use. fresh, check_file and pytest load the installed build. Cargo tests build missing code but install nothing.
cargo cs checks every target in its workspace, tests included, without codegen. Rustymath's tests need cargo cs in rustymath.
cargo test --release: correctness checks while working; one test binary for focused checks. Bare cargo test builds the test profile, which has release's settings but compiles everything again into target/debug/. Both skip the timing suite ([[test]] speed, test = false in Cargo.toml): BASEDPL_COSTS=1 cargo test --release --test speed runs it, logging every cost. Kernel: fastws.cargo.run_tests(root, *cargo_options, test_args=(), timeout=30) defaults to release, builds native test executables once, with no build limit, then runs each executable directly within its own timeout. A binary that runs out of time is killed and counts as a failed binary, with its suite's timed_out set. Filters and libtest flags go in test_args. Results describe that compiled build, not later working-tree edits. It accepts seconds or None, not worker.Limit; waits inside a test binary count. Doctests are disabled; meaningful example assertions belong in ordinary Rust tests.
pytest -q: at the end of a piece of work. Before reporting a pytest failure in untouched code, rebuild with the same profile and rerun.
Builds have no execution timeout; Cargo serializes them per target directory. Worker checks and timings stop after 30 s of work (basedpl.worker.LIMIT), excluding builds and the worker's wait for the timing lock. timeout= takes seconds, None, or a worker.Limit that calls share. A check stopped at its limit gives each unfinished case as unchecked; other tools raise TimeoutError. cargo_costs shares that budget but counts all time inside its native test binary, including lock waits. A plain shell cargo test has no limit: use run_tests. Pytest's separate timeout = 30 watchdog bounds each test's setup, call and teardown together; it is not a suite execution budget.
Unix cancellation: one process-group owner per job (fastws.procs.bounded, or each standalone prerelease step). Nested commands inherit the marked job's group; let their timeout/error end the job so its owner cleans up descendants. Do not detach commands. Worker is a protocol transport, not a supervisor: it inherits its caller's group and close() stops only the worker. Standalone callers needing descendant cleanup put the call inside an owned job.
Full checks, in the release's iteration stage and before a release (CI's, plus targets this Mac's daily builds skip): cargo develop, cargo test --release, cargo test --release --test speed (the timing suite), pytest -q, bpl-nb, nbdev-test, node tests/input.mjs, node tests/highlighting.mjs; then bpl-tests target_builds /tmp/<prefix>_wasm (basedpl's and rustymath's Linux x86-64 libraries plus the browser package); then wasm_test(DIR) and wasm_test(DIR, speed=True) on it. A build stuck on one core for minutes: see "Codegen and rebuilds". Build locations and profiles follow the current integration process in meta/process.md; documented browser and x86 diagnostics below remain available. Do not use maturin develop: its configuration does not reuse the normal local build. The paired build worktrees isolate compiles from unfinished live edits; do not wait for the shared live tree to become quiet.
Running Python processes keep their loaded extension. Kernel: from basedpl.worker import fresh; fresh(code) checks the installed bpl in a fresh worker (no kernel restart); BPL errors raise BplError; fresh(code, build=BASE) runs another build (baseline: below); a list runs each expression in its own worker, one row each (output or error); prefs= runs first in each worker: fresh(cases, prefs='•prefs ["kinds":"static"]'). Check one expression and read its result before building on it.
fresh, time_cases, and check_file/check_page failures show each input's linear form (basedpl.syntax.linear): its grouping, bracketed. basedpl.bpltests.check_file, check_page: check reference files and docs through the worker, no Rust build; build= checks another build without installing it.

Work allocation follows meta/owners.md: each piece owns whole files. Main keeps independent work moving, samples design quality and rebalances boundaries. Use verified edits to preserve others' changes.

Main reviews coherent milestones against the plan and owners.md: architectural departures, duplicated mechanisms, deletion of old routes, interface fan-out, net source lines and substantive costs. The owning fork decides routine names and implementation details. Delegate mechanical inspection when it resolves meaningful uncertainty; do not require another broad review of every note or API proposal. Main can also cut code and reassign connected work when repeated coordination shows a poor boundary.

Builds and integration follow the integrator process in the required section above. Source owners keep writing useful independent work while the integrator builds an exact paired snapshot. Compile failures may justify small compiler-driven corrections; a substantive interface issue goes back to the connected owner rather than growing an adapter. Release 1's aims: about the same speed as the previous release, or better, with less code, clearer responsibilities and a better foundation for static types. Speed is measured against the baseline costs in meta/pre-rewrite-costs.tsv.

Costs in copies: BASEDPL_COSTS=1 cargo test --release --test speed, basedpl.bpltests.time_cases(cases), or time_cases(cases, section='models.bpl:static') with a speed file's setup and a section's preferences. A copy: time to write the data's item count of negated floats to a second buffer. Timings use the installed build, or a kept build passed through build=.

rustymath::timing::COPIES: stored copy times per speed-test machine, by CPU, item size and item count; the slow end of normal; for copies whose source plus destination fit in cache, 64 MiB (CACHED_BYTES). Stored because a 65,536-item copy timed beside each case varied over 3× on CI's runner (source in L3 shared with other VMs) while functions varied at most 1.18×. Larger copies run at memory speed, varying up to 2× between runs on a shared runner, as do those cases: measured once per run. A cached size with no entry: measured once, printing a line to add to COPIES. Browser build: own entries (copy under the JavaScript engine); it can't read the CPU name, so tests/wasm-pkg.mjs passes Node's.
Logs: each case's line ends with its time per item (logged_costs: per row); one copy line per data size (copies).
Footprint: data plus peak bytes held (rustymath::memory::peak_held). Over 1 MiB (rustymath::timing::CORE_CACHE_BYTES, one core's L2 on CI's runner), a case also times a copy of its footprint beside it, appending footprint and that copy's time per item to its line (logged_costs: footprints). Past L2, data reaches L3 and memory shared with other VMs: cases from about 2 MB swung up to 4× between runs, a different few each run; up to 1.5 MB held. Costs still divide by the data's copy; the footprint copy shows whether a case's time follows it.
Budgets are single-thread. thread_scaling() checks threads: a large product, 512×512 SVD and Hermitian eigendecomposition, a stack of QR decompositions, a product of a convolution's windows, a copy of those windows, a large elementwise function, each on 1 and t = min(4, physical cores) threads. Serial fraction s (share of one-thread time that doesn't split) gives expected speedup S = 1/(s + (1 − s)/t) (Amdahl). s: first principles for our kernels; measured for Accelerate on macOS (product, SVD, eigendecomposition, product of windows) and for the copy of windows (memory bandwidth limits copies). The product of windows fails if its rows stop splitting across threads; the copy of windows if a copy's items stop splitting. Failure: below 1 + (S − 1)/2 (2.5 on 4 cores, 1.5 on 2, for full splitting), or below 0.9 for s = 1. Accelerate's SVD and eigendecomposition: s = 1 (GESDD, SYEVD run on one thread whatever its threading setting). Run thread_scaling() on the authoritative host. Linux counts one core per set of hardware threads sharing a core; a VM not reporting the sets counts each vCPU.
speed_costs(): cases in tests/speed/*.bpl; cases= expressions; wasm=DIR browser build; build= another build (e.g. a kept build's id); native=True times rustymath's own linear algebra, which dev builds compile in; prefs= BPL run before each section header (•prefs ["kinds":"static"]). logged_costs(log): timing output, text or file. cargo_costs(path): either project's Rust speed test; build=EXE a saved copy of its executable. pytest_costs(path): rustymath's Python speed test.

Instruction counts: code layout alone moves timings more than a few percent, hiding per-call changes in copies; instructions(cases) counts instructions retired, which load and layout don't move. instructions, profile and call_profile run loops as ordinary code (BASEDPL_LOOPS=ordinary): counts measure a call's interpretation, which a case's own ⍣N loop would otherwise replay. replays=True lets loops replay their recordings, as real runs do: a case that names N, such as {0.5×⍵+⌽⍵-(+/⍵)÷4}⍣N 1 2 3 4, then counts its loop's replayed steps (22,473 instructions a step, against 26,825 as ordinary code, on 1 November 2026).

A case is an expression (k(⌈/⊢)⌸a) that each step evaluates at top level and forces in full, as the speed suite's timings do (timing::time), through the worker's steps request. A case naming N makes its own calls ({⍵}⍣N 0: one dfn call a step) and must make one per step: a map like f¨N⍴0 calls its dfn once on all cells, giving no per-call cost.
Fresh single-threaded bpl processes at four N, three each (runs=), lowest count per N; each successive pair's slope, Δinstructions/ΔN, cancels startup and setup=; the median of the three slopes drops a one-off cost starting past some size (≢•ucs N⍴"ab" "cde": about 5×10⁶ more from about 131,350 items). All N lie past warm-up, from rough counts at 1 step and, for a step under 10⁸ instructions, at enough steps to retire about 10⁸: allocating cases retire up to about 5×10⁶ extra, varying by process, while the allocator settles; successive N 5×10⁸ instructions apart keep that under 1% of a slope. counts= sets two or more N. Reads /usr/bin/time -l (macOS), perf stat (Linux).
build= another build; a list (build=[base, None]) counts each at the same N, side by side, with the first-to-last change.
Counts run bpl --worker with a steps request. Builds that predate it, the release's included, can't be counted or profiled; compare them with timings.
tests/instructions.txt: instructions per step per case, macOS on Apple silicon. Most cases make one call per step ({⍵}⍣N 0 to fib); three map over the same 64 cells each step via Each, Rank's general path, and a system function's item map. instruction_budgets() counts each case through instructions: over if more than 3% above its count, stale if more than 3% below; tests/test_instructions.py fails on either, skips other platforms. In the change that makes a call cheaper, lower each count it moves. A new compiler version can move counts by inlining differently: then set them again from bpl-tests instruction_budgets. CI runs no performance-budget suites.
profile(case): estimated instructions by function. A fresh bpl --worker process runs setup and one warm step, replies, then repeats the step until it is killed. macOS sample or Linux perf samples its main thread's call stacks. Each function's self-sample share times the build's instructions count estimates its instructions per step, assuming one retirement rate for all functions. profile_diff(case, base) compares those estimates for two pinned builds; inlining can move a function's count into its caller. For costly cases, pass counts=[1,2], runs=1 to skip calibration. callers=N groups each leaf with up to N caller frames. Use these before reading machine code.
call_profile(case, function) (macOS, Apple silicon): exact instructions one call of function retires, callees included, by function. lldb runs the case once per step in a fresh process, lets skip= calls pass, single-steps the next to its return. function: regex on a demangled name (binder::derived), which must have code of its own. base=: a build shown beside build. Allocator functions and callees share one row (heap-state dependent); a second total excludes it. profile's time-based estimates can misattribute: find the changed function with profile_diff, settle it with call_profile. Limit: calls of 100,000 instructions, at about 450 a second.

On Linux, install perf and binutils, then allow user events instructions:u and cycles:u. Check the selected perf with profile('1+1', build=PIN, seconds=1, callers=4) before an investigation. kernel.perf_event_paranoid=1 permits those events on Syd1; changing it requires the machine owner's approval. Missing tools, failed commands, missing instruction counts, empty samples and unparseable samples raise errors. Unknown samples remain in the total; limiting displayed rows never renormalizes their shares. Report a tooling gap immediately with its owner and next check.

Syd1 selects the installed perf 6.11.11 through ~/.local/bin/perf; its HWE tools package is marked manually installed. Distro perf 7.0.14/libdw applied an incorrect 0x1000 ELF load bias to native BPL frames. Readable symbols alone did not establish valid callers. Matching libc6-dbg resolves libc internals. The selected perf and the shared c++filt step have been verified on native and formatter call chains without a BPL rebuild.

The helper enables Cranelift's existing perf-map output in its child and removes the map when that child stops. Generated kernels appear as kernel. Their samples have no caller-unwind metadata; native caller support does not imply unwinding through a JIT kernel.

bpl-tests NAME ARGUMENT... runs one of these from the shell, printing its result: check_file, check_page, time_cases, instructions, instruction_budgets, profile, profile_diff, call_profile, disasm, jit_disasm, speed_costs, thread_scaling, cargo_costs, pytest_costs, run_pytest, compare_costs, bisect, explain_diff, wasm_test, wasm_stack, target_builds, prerelease, module_edges (references from the engine's modules into the language's, meta/crate-split-2026-10-31.md), dead_functions (functions whose name appears nowhere else in either repository). bpl-tests time_cases '+/v' --threads 4: on four threads. time_cases, instructions, speed_costs take positional cases; put them after -- when one starts with - (bpl-tests speed_costs -- '-.×m' '+/v'). bpl-tests NAME -h: arguments. In a kernel they return Python objects (e.g. rows for compare_costs).

Syd1 is authoritative for x86 performance budgets; Jeremy's laptop is authoritative for macOS/ARM. Run the existing speed suites and thread_scaling() on those hosts, outside CI. bpl --costs PATH times a speed file or directory and logs costs without failing budgets. logged_costs reads those logs and their CPU/copy records. Timings warm recording and compilation before measurement; recurring dispatch/replay work remains included. Do not subtract a universal fixed-call constant. Compare baseline and candidate on the same host before interpreting a change; the old CI thresholds are historical reference values until reviewed.

Reviewing a run: find costs over budget and costs changed more than 20% since the previous run. compare_costs(before, after) lists both, from two runs' logs or rows from speed_costs and the other timing functions, comparing times per item where both have them (so COPIES changes don't show). Study a slowdown only if it repeats on the same machine or across similar cases.

Budgets:

Budget = expected cost × multiplier: 2 under 50 copies, 1.5 for 50-500, 1.3 over 500 (cheap cases vary most). Expected cost: first-principles analysis of what the case should cost; a later one comes from fuller understanding (a platform difference, a missed cost), recorded in the case's comment. The comment above each case or group gives the expected cost and reason, and any platform's own cost and reason. One expected cost serves every platform until a specific platform difference is found (e.g. macOS runs matrix products and decompositions on Accelerate, Syd1 on rustymath's kernels).
A BPL speed file's ⍝⍝: footer: rule budgets (expected × multiplier) for macOS on Apple silicon, Linux on Syd1, then (basedpl only) wasm under Node on Syd1.
A case over its rule budget gets a directive holding the column: ⍝⍝⍝ held linux 15.6 wasm 18.9: REASON (a code gap; each held column with its held cost) or ⍝⍝⍝ busy linux 2.7: REASON (data past the 1 MiB footprint threshold, which busy hosts slow in some runs). The multiplier applies to the held cost; the case's line shows both (budget 32 held from 16). A gap's hold goes stale at 0.9× its rule budget or less (inside the 10% margin a review holds a column at); runs print it stale, and its held line can go. Busy holds never go stale.
A case within 10% of its budget on a checking platform gets a dated held line raising that column 10% as a margin against timing noise: held cost = the one whose budget, rounded up as budget rounds, is at least 10% higher (⍝⍝⍝ held wasm 17.6: 32 → 36); stale by the 0.9× rule.
A platform cost no run has measured (a Linux or wasm estimate) keeps twice the estimate until the authoritative host measures it, marked ~ in held lines (⍝⍝⍝ held linux ~107: REASON). Rustymath's tables still sit at twice expected cost (old rule).
rustymath::timing: the multiplier (budget); reads (hold) and applies (Budget) these lines for every speed test. Rustymath's budget files write them as # held ..., # busy ... above a case's line.
Over budget: study it, finding a speed bug or something the analysis missed. Avoid raising a budget to match a run without knowing why it costs that.
rustymath::timing::BUDGET_CPU selects the development Mac for macOS/ARM and Syd1 for Linux/x86 and wasm. Only that CPU fails a speed test on costs still over budget after retries. Other CPUs print over without failing. Existing Linux/wasm thresholds came from the old CI machine and need review against matched Syd1 measurements; changing the authoritative host does not approve new thresholds.
set_budgets(expected) applies the multiplier and writes rule budgets for expected's cases (dict: case code → expected cost), leaving held and busy lines. builds=: one platform's column; estimate=True: twice the cost, for estimates; sections=: named sections only (a case can be in several, e.g. copies, widths); path=: a BPL speed file or directory, or rustymath's tests/speed.txt, tests/speed_python.txt.

Routes:

tests/explain.rs's speed_cases_take_their_routes captures every speed case after warm-up at normal setup and size, forcing results outside timing (timing::routes), then each reference case with route directives once under ordinary preferences. Plain cargo test runs it. check_file checks a file's route directives likewise, on one thread. Route checks decide which calls wait with the Mac's costs on every platform (fusion::Waits::Reference, set in timing::route), so a directive observed on the Mac holds on CI; real runs and timings use each platform's own costs. ⍝⍝⍝ route NAME above a case requires that fast route. A permitted slow event never excuses a missing required route.
Unlisted slow events fail: stored (pending items computed outside a fused group), forced (a call computing pending items it reads as items; rows with reason read as values, an argument such as a count, or an effect, the pending arrays that an effect computes first, aren't slow), fallbacks (batched fallback), mixed (a write leaving mixed storage), itemwise, stopped (stopped trace). Name them as •explain does: function (forced), code (mixed), route (itemwise), reason (the rest). ⍝⍝⍝ allows RECORD "EVENT": principle permits work (a matrix product reading its arguments whole); ⍝⍝⍝ gap RECORD "EVENT": what should happen names a fast-path gap. A stopped trace takes allows only for a by-design stop (dynamic kinds' value-dependent types), else gap. declined (a considered route not taken) is informational; a gap or allows can name its route and reason (fusion: an exact argument). Every annotation must match an observed event; remove it when the event disappears. cargo test -- --nocapture prints remaining gaps.
Native selectors (reference, explain, speed): BASEDPL_SOURCE matches a source name, with or without .bpl; BASEDPL_SECTION matches a section prefix; BASEDPL_CASE matches an ID or code prefix. A selector matching no cases fails. Filtering precedes setup and case execution. Explain stories use source explain, with sections idioms, records, read-back, accumulation, reads, declines, stops, memory.
tests/skipped.txt: full-size cases explicitly set aside, one per line: test name, tab, section (* for every section), tab, case key, tab, the open item that records why. Coverage and unit suites use the case's code as its key. Reference-file rows use reference:<file>.bpl, the exact section and the existing case ID, or the first comment line when the ID is empty. reference::mark_skipped defines that key and rejects ambiguous or stale entries against the full file before selection. Rust checks and worker inventories share the matcher. check_file retains skipped identities, reasons and modes; skips are unresolved, not passes. Remove an entry when its case is ready to run again.
Events: Rust in both repositories reports routes taken, declines and stopped traces via rustymath::events; basedpl records itemwise routes in •explain's rows. Uncaptured reporting costs one atomic load. The route test compares observed successes, declines and fixed itemwise routes with the declared IDs (idiom cases and annotated references included); the inventory includes warm-up, route contracts only the warmed run. It checks event-name coverage, not every branch at an emission site. Study an unexpected event before annotating it, as for a case over budget.

Retries: speed tests and thread_scaling retime each failing case after the rest, after each pause in rustymath::timing::RETRY_PAUSES (0.1, 1, 10 s; 10 s outlasts longer load bursts from other VMs on CI's runner), failing only cases that fail every time. Before each retry, speed tests and bpl --costs keep the thread busy 0.1 s (rustymath::timing::warm): idle cores start slow (on this Mac, small cases timed straight after the 10 s pause cost up to 3× without it). Retries print old and new costs (speedups in thread_scaling), even in passing runs; logged_costs keeps a retried case's lowest. Study a case that needs retries in most runs, as a slowdown.

Before and after: for a change meant to speed something up, or to code every call goes through, time affected cases before and after. Other changes need only the speed test, which fails only past budget, missing smaller slowdowns.

cargo develop keeps each installed build read-only under an id hashing its inputs and files, installing by repointing one link (fastws.store). fastws.store.builds(root): the newest builds of the project at root, as records: id, when and by whom kept, input files changed since the previous build, inputs possibly modified since build started (mtime diagnostics, not proof of what the compiler read), * on the installed one. bisect(case, good, bad, by=): the adjacent kept builds where a case's verdict changes, by check (a reference case passes), value (an expression prints what it prints on good), instructions or time (cost nearer good's than bad's), with the input files changed between them and every session's journal edits to those files (a shell has no journal, and bisect says so); explain_diff(case, base): how •explain's records of case differ between two builds, each captured as the route test captures it (the worker's route request, timing::warmed), with listing diffs; a good newer than bad finds a fix. Note the installed id before changing anything. Don't restore old code in the shared tree for a baseline.
Every timing tool's build=: a kept build's id prefix, a bpl path, or None (installed; installed from the shell), resolved once at start, so another agent's cargo develop can't change a run. Time both builds in one call: kernel time_cases(cases, build=[base, None]) (base: the noted id); shell bpl-tests time_cases CASE --build BASE NEW. Builds run in fresh workers taking turns in alternating order, rounds times; the result holds each case's fastest time per item per build, in picoseconds. compare_costs(*result) lists cases that moved.
compile= and defer= on fresh, time_cases, instructions, speed_costs and cargo_costs set the environment of the processes they start, with native= (worker.environment, which each takes through @delegates). compile= sets BASEDPL_COMPILE: always compiles every block stage on its first run, never compiles none, None keeps the compile policy (settings::Compiles). defer= sets BASEDPL_DEFER: all makes every call wait, as fusion at any size runs; reference takes the Mac's costs; None, this platform's (settings::waits). Shell: --compile never, --defer all.
Timing sessions, and the processes instructions, profile and call_profile start, seed random generators at 0: every process and build draws the same data. Builds from before a rustymath::timing::COPIES change divide by a different copy.
On this Mac a fused case on a large input can cost up to about 15% more in one process than another (depending on whether malloc starts the input on a 32 KB boundary, where every block of a block stage then ends): retime with more rounds before calling a lone difference that size a regression.
pytest_costs, run_pytest: build= takes a rustymath build id, for rustymath Python API changes. macOS native timings of matrix products and decompositions mostly measure Accelerate; rustymath's DEV.md says how to time its own kernels.
Compare static with dynamic kinds, or programs with the interpreter, only on code written for static kinds, with explicit sizes on literals and data: dynamic-kinds code can be wrong under static kinds, and its timings don't show static cost.

Edit journal: Claude Code sessions record every exhash edit in /tmp/exhash-journal/<session id>, by agent (kernel startup sets EXHASH_JOURNAL from the session id, EXHASH_AUTHOR from the RUSTYGATE_AGENT rustygate gives each subagent's kernel). A fork reports its code lines with from exhash import journal and from fastws.integrate import code_count, then journal(os.environ['EXHASH_JOURNAL'], author=os.environ['EXHASH_AUTHOR']).stat(count=code_count), reading only the .rs rows (a kernel's exhash is a function without journal). Main reviews a fork's edits with .diff() on its agent id's journal: git can't separate forks' uncommitted edits.

Timing lock /tmp/aai-ws-timing.lock:

Exclusive: timings. Shared: every rustc call (via .cargo/rustc-lock, the wrapper ws-sync writes into the workspace's Cargo config, so builds wait behind timings however started); reference and explain tests, pytest, check_file, check_page, bpl-nb, instructions, profile, wasm_test without speed.
/tmp/aai-ws-timing.gate: held by a waiting timing; shared holds pass it first → no new build or test starts while a timing waits; the timing starts once current holds end.
Rust takes it via rustymath::timing, Python via fastws.timelock. A kernel's time_cases or native speed_costs: the bpl worker takes it, not the kernel. Node can't lock files: wasm_test and speed_costs(wasm=DIR) hold it in Python while Node runs.
Every acquisition fails after 5 minutes rather than proceeding unlocked. After 30 s an exclusive wait prints processes with the lock file open (possibly mere waiters). Same-thread shared nesting reuses its hold; any nesting involving an exclusive hold is rejected immediately.
A process holding the lock shared must not start a build or timing: it would wait behind a timing waiting on that hold, for the full 5 minutes. Never take the lock yourself in a kernel process. Never coordinate timings by hand: the lock schedules builds and timings. A hold outlasting its call blocks timings later calls launch.
Native timings: one thread, user-interactive macOS QoS (favours performance cores; not CPU affinity).

Browser: during development, run cargo wasm only for browser work. Agents use cargo wasm --out DIR (not replacing a module another agent uses), then basedpl.bpltests.wasm_test(DIR) (tests/wasm.mjs) and wasm_test(DIR, speed=True) (tests/wasm-speed.mjs; on this Mac it prints every cost and marks cases over wasm budget without failing: wasm budget authority is Syd1). Other edits under cfg(web) or host_web.rs: check with cargo cs -p basedpl-wasm --target wasm32-unknown-unknown (browser wrapper crate plus basedpl's browser code); plain cargo cs compiles neither.

Machine code

bpl-tests disasm PATTERN: machine code (objdump, installed bpl) of the function whose demangled name matches regex PATTERN; --build BUILD: another build; several matches → lists names. Always-inlined functions have no code of their own: read their callers.

JIT kernels exist only while a process runs. BASEDPL_JIT_DISASM set → bpl prints each kernel it compiles to stderr: a description line, then machine code. bpl-tests jit_disasm CASE --setup SETUP --prefs PREFS runs a case so: bpl-tests jit_disasm '+/ux>127ₓ₁' --setup 'ux←(¿(2*22)⍴256)ₓ₁' --prefs '•prefs ["kinds":"static"]'. A stage compiles on its first run only where that run repays compiling (blocks::Blocks::repays): its items × the passes its parts save × SAVED_NS exceed its parts' compile cost, which is higher for a fold or a comparison.

--x86: the x86-64 code CI and Linux run. First builds basedpl's library and its linked rustymath library for Linux x86-64 into /tmp/aai-ws-x86: minutes after a change → only for an x86-code question. Unfinished edit stops basedpl compiling → reads rustymath's library alone, saying so first. Linux: macOS builds omit rustymath's native linear algebra kernels. Libraries only: the Python extension crates can't build for Linux on this Mac. Their release profile matches CI's test profile in x86-64 flags and optimization. The build sets CC_x86_64_unknown_linux_gnu=clang itself; don't move it into .cargo/config.toml (CI's Linux release builds read that file in a manylinux container without clang). Don't use --emit asm: it splits code into other codegen units and inlines differently.

Workspace, builds and CI

Crate	Responsibility
`basedpl`	Python-free interpreter; public Rust API
`cli/` (`basedpl-cli`)	`bpl`, REPL, Jupyter kernel, JSON-lines worker
`py/` (`basedpl-py`)	unpublished PyO3 extension `basedpl._core`
`wasm/` (`basedpl-wasm`)	browser bindings
`xtask/`	`cargo wasm`: builds the browser package

Default workspace members: core, CLI, Python; wasm and xtask build only on request. Frontends use the core's public API. Rust tests: tests/ (core), cli/tests/ (CLI). Libraries set test = false (no separate cfg(test) builds) → no build runs #[cfg(test)] code → libraries hold no inline tests or test-only counters. Internal behaviour worth a test (e.g. whether an expression fuses) is worth knowing for every user: tests/ checks it through the public API (•explain, •storage, •memory). PyO3: its own crate, not an optional core feature. The core uses serde_json for the process protocol and data functions.

Profiles: Rust 1.98 or later (algebraic float operations). test inherits release: opt-level 3, incremental, thin LTO across each crate's codegen units; no debug information, overflow checks or debug assertions → a debug_assert! never runs: write a check that matters as an always-on test. .cargo/config.toml: -Zshare-generics on every target and x86-64-v3 on x86, in target entries Cargo joins. dist: no incremental compilation; strips symbols. Keep these aligned with rustymath's.

Packaging: fastws's cargo develop builds the project-selected profile (quick by default), installs the extension, stages bpl into the venv. py/: PyO3's extension-module, link arguments without libpython. Editable installs: fastws.build_backend installs metadata and a path to python/, no Rust compile → after a standalone uv sync or editable pip install, run cargo develop (ws-sync does after syncing). Wheels: maturin for the extension, cargo stage for the Python-free CLI; Linux staging inside the manylinux container.

Features: basedpl's crates pass on rustymath's default native feature (rustymath's DEV.md): each depends on the crate below with default-features = false and makes native its own default. Release staging and wheel builds pass --no-default-features → released Mac builds omit rustymath's own linear algebra. The Python binding inherits the CLI's pure-Rust frontend dependencies → identical shared dependency features in release builds. flate2: the system zlib on Apple targets; elsewhere its Rust backend, miniz_oxide.

Dependencies: published ranges, with the lower bounds the APIs in use require. Cargo.toml names a published rustymath range; the ~/aai-ws Cargo patch selects the sibling checkout locally; CI resolves published crates → release rustymath changes before basedpl CI relies on them. Don't edit fastws-generated Cargo patches, .git/fastws-cargo-key or workspace configuration; don't commit the workspace-generated Cargo.lock. meta/: ignored planning material, not contributor documentation. [tool.uv] cache-keys tracks packaging manifests, not Rust sources.

CI: Test runs correctness checks on pushes to main and pull requests. Native checks include editable installation, cargo develop, cargo test --release, pytest, bpl-nb and nbdev-test; Python tests use the installed release build. Wasm checks build with cargo wasm --strip and run node --test tests/wasm.mjs. Native and wasm jobs use ubicloud-standard-4-ubuntu-2604. A lockfile is generated before restoring Swatinem/rust-cache so dependency versions participate in its key. CI runs no performance-budget suites or release-cost logging. Reference deadlines and execution watchdogs remain enabled.

Release: on v* tags, separately, on GitHub's runners. Builds dist wheels, sdist, standalone CLI artifacts and the Windows keyboard archive; publishes Python artifacts and the GitHub release. npm job: publishes the stripped browser package by trusted publishing; purges jsDelivr's package files (stale unversioned URLs). Runs no tests; doesn't wait for Test.

Codegen and rebuilds

cargo develop and fastws.cargo.run_tests(root, ...) build the release profile, selected by [tool.fastws] dev-profile = "release" in pyproject.toml, so development and routine tests share one build in target/release/. It uses optimization level 3, local ThinLTO (lto = false) and incremental compilation. Bare cargo test still uses Cargo's built-in test profile, which has release's settings but builds again in target/debug/; pass --release when not using run_tests.

The project Cargo configs enable cross-crate generic sharing with -Zshare-generics=yes. They set RUSTC_BOOTSTRAP=1 for Cargo's compiler processes, without changing the shell or requiring nightly. The tested toolchain is rustc 1.99.1 (58cf8f9c35b355199727627d07ae34a4e2846d67), LLVM 23.1.1. Keep the existing Rust minimum and toolchain-selection policy. The unstable switch should be checked when upgrading the compiler.

The dist profile owns authoritative x86 speed gates: O3, explicit cross-crate lto = "thin", 16 codegen units, nonincremental. Routine release/test stays local ThinLTO. Run cargo test --profile dist --test speed in either repo; use ship-rs-build --profile dist for the Python wheel and run its existing speed test; build Node budgets with cargo wasm --profile dist --out DIR, then speed_costs(wasm=DIR). Syd1 targets were recalibrated from actual BPL v0.1.82 (shipped rustymath v0.1.53) and standalone rustymath v0.1.54 with this profile and the existing budget multipliers; macOS targets and holds are unchanged. LTO changes runtime code as well as build time. Keep the profile, optimization level, features, target, compiler and LLVM version fixed within a comparison. Rustymath's DEV.md "Loops and codegen" covers runtime inlining and vectorization.

Measure a representative rebuild

Use the normal cargo develop workflow for development. A compiler investigation can time its underlying build without capture or installation. In an isolated, warm paired worktree, the normal release workload is:

/usr/bin/time -p cargo build --lib --bins --release --timings --verbose

Use the profile of the actual slow workflow. Keep source and target paths stable. Never overlap raw Cargo with cargo develop capture against the same outputs. Run the probe in a quiet compiler interval. Live writers can continue outside the frozen pair. Compilation has no runner timeout. Test execution keeps its documented 30-second budget.

Record the command, source revisions, effective profile/features, rustc -Vv, target flags and relevant Python environment. Verify dependency manifest paths with cargo metadata --format-version 1: both rustymath and buffer must resolve to the intended pair. Use historically compatible tags for a released baseline. ws-sync owns workspace patches and the rustc wrapper. fastws.core.cargo_setup(root) regenerates them for an isolated pair without pulling or building.
Build dependencies and consumers once. Run the unchanged command again. An unexpected rustc invocation or artifact change needs an explanation before the edit probe. Retain lock waits. A new target directory, changed feature set or changed PYO3_* setting is a different workload.
Edit one used numerical body. A same-line, same-length edit avoids source-location changes elsewhere. Confirm the function is instantiated in the measured consumer through monomorphization listings, linked symbols or machine code. Verify that object content or linked instructions changed after the edit. A generic source edit can change metadata without generating any machine code.
Restore the exact source and rebuild before tests or installation. Keep warmup, no-change, edited and restored timings separate. Record wall/user/sys time and the source/configuration of each build. Change one proposed cause per experiment.

For example, reversing the operands of the used real Subtract body in BPL v0.1.82/RM v0.1.53 rebuilt on an M5 Max in 31.15 seconds. Reversed fsub operands proved regeneration. A current agreement::paired edit took 15.30 seconds but regenerated no objects: its generic caller chain was unused. Neither number predicts a broad source update, another host or another profile. Raw experiments belong in meta/.

Locate the critical path

Start with target/cargo-timings/, Cargo's verbose invocation log, fingerprints and artifact timestamps. Separate dependency warmup, frontend/metadata work, LLVM work, linking and fastws capture/install. Cargo starts consumers when dependency metadata is ready. Overlapping crate times are not additive. In one measured rebuild rustymath took 195 seconds while BPL's overlapping 103 seconds finished much earlier; final CLI/Python links took under a second.

A phase labelled codegen does not prove that objects regenerated. Compare object content and modification times. Incremental session directories are renamed and old generations remain in the cache. Normalize by crate/object basename and distinguish active objects from repeated cached generations. Under ThinLTO, pre-LTO inputs and imported post-LTO objects are different counts. Measure the final linked text section for binary-size claims. Summing object bytes or aliased symbols overstates linked size.

Check the actual process handle, CPU use and lock state during a long build. A quiet output stream does not establish that a process stopped. One busy worker can be a large codegen unit containing thousands of functions. It need not be one pathological function.

Check both compiler caches

Run sccache --show-stats before and after the chosen workload. Read hits, misses, errors and non-cacheable reasons. Zero hits alone is not a configuration failure. Our warm incremental RM/BPL builds and executable/cdylib targets were ineligible; dependency builds produced cache hits. Rust's incremental cache handles the dirty project code in that workflow. Do not disable incremental compilation to manufacture sccache hits.

If requests do not reach sccache, inspect the actual rustc invocation, RUSTC_WRAPPER/RUSTC_WORKSPACE_WRAPPER, generated .cargo/config.toml and .cargo/rustc-lock. The generated wrapper takes the shared timing lock before invoking sccache. Fix the configuration owner with ws-sync or cargo_setup, not a second wrapper or a hand-edited patch file. Diagnose cache errors and changed compiler/flags/features before clearing a cache. Undefined .llvm.<hash> symbols after an interrupted build can come from stale incremental units. Clean only the affected crate/profile with cargo clean -p CRATE --profile PROFILE when that failure identifies the cache.

Find generic expansion before optimizing code

Use cargo llvm-lines to rank unoptimized IR lines and copies. Its --files mode reads existing .ll files without a build. It reports the chosen root crate, not the whole dependency graph. Keep the real profile, features and target when generating IR. Do not enable fat LTO to combine the crates for this measurement.

For monomorphization and codegen-unit ownership, copy the actual rustc invocation into an isolated diagnostic. Preserve its crate/features/cfg/target/optimization and dependency arguments. Use separate temporary output and incremental directories. Emit llvm-bc,metadata, remove the extra LLVM -Cpasses setting, and add:

-Cno-prepopulate-passes -Csave-temps
-Zhuman-readable-cgu-names=yes -Zprint-mono-items=yes
-Zdump-mono-stats=DIAGNOSTIC_DIR -Zdump-mono-stats-format=json

These unstable diagnostics need a compiler that supports them; scope RUSTC_BOOTSTRAP=1 to the diagnostic invocation when using our installed compiler. Preserve the original codegen-unit maximum explicitly. Bitcode-only emission otherwise defaults to one unit. MONO_ITEM ... @@ module[linkage] records ownership; *.no-opt.bc measures preoptimization IR. Mono statistics include inline copies, and MIR-statement counts are estimates. This diagnostic skips LLVM optimization and linking. Its time is not an optimized rebuild time.

A downstream IR capture needs core and buffer metadata from the same diagnostic build. Reusing buffer metadata compiled against another core can give E0463. Emit bitcode and metadata together rather than metadata alone when the consumer needs optimized MIR.

Runtime checks of type-known facts can instantiate impossible generic paths before LLVM removes them. if const { N != 2 } prevented unary objects from instantiating binary fold/scan machinery. Formula::BOOLEAN, generated from the existing fact row, pruned impossible numeric/Boolean map alternatives. Keep each capability fact at its existing owner. IR reductions require an ordinary build measurement before making compile-time or linked-size claims.

Free generic functions normally follow their defining module into codegen units. Methods can follow the characteristic type's module. Moving an impl into a different file does not necessarily move its generated code. Raising codegen-units cannot split an oversized unit arbitrarily. Iterator adapters, closures and inlining can copy large bodies into several units or frontends. Inspect ownership and copies before changing module boundaries or adding inline/inline(never) attributes. Rustc's generic-sharing policy also depends on optimization level and compiler version.

Source line shifts change panic locations and other embedded caller locations. A small edit above generic code can invalidate many instances. A child module can isolate that location churn. Keep the body at the same source location when testing another cause. Do not confuse fewer regenerated units with faster optimization of each unit.

Escalate telemetry and reduce a repro

Sample the busy compiler first (sample on macOS, perf on Linux). A sample identifies its current pass, not the cause of the whole build. Use rustc self-profile for compiler phases or -Zllvm-time-trace for LLVM pass/function timings when the cheaper evidence leaves a specific question. A cross-target build containing C code also needs the target C compiler, such as CC_x86_64_unknown_linux_gnu=clang. cargo wasm compiles its host xtask, including rustymath; an x86 host compilation can delay a wasm job.

Separate exclusive pass costs from inclusive pipeline totals. ModuleInlinerWrapperPass includes work beyond the inliner. Do not add nested timings together. Rank functions and codegen units before assuming one loop-vectorizer failure. Target cost models can produce different arm64/x86 behavior. Retain symbol attribution and sampled CPU evidence with the source/configuration that produced them.

Full query-key self-profiles and LLVM JSON traces can add minutes and tens of gigabytes. Request only the needed telemetry. Parse large traces with streaming or memory-mapped tools rather than loading the whole JSON into Python objects. Measure instrumented overhead separately. Use the matching rustup LLVM tools and existing measureme tools for analysis.

A small repro must retain the concrete types, widths, generic caller chain, inlining, profile and target that generate the expensive code. Keep a consumer that instantiates the body. Remove unrelated code in steps and recheck the compiler cost and generated code. A repro that leaves only an unused generic function or an optimization-equivalent expression no longer demonstrates the problem. Test a proposed source/configuration fix with the representative warm edit and the existing correctness/route checks. Keep numerical semantics and the production profile unchanged unless that change is explicitly agreed.

Code map

Files	Responsibility
`syntax.rs`, `name.rs`, `eval/binder.rs`, `eval/tree.rs`	structural parse; interned names; runtime category binding; trains; dot access, subscripts; dfn statements' compiled recordings
`eval.rs`, `eval/session.rs`, `eval/bindings.rs`, `eval/operators.rs`, `eval/operators/`, `eval/assign.rs`, `eval/dissect.rs`	shared calls, function values; sessions, modules; binding storage; operators; scopes; assignments; expression trees of bound values
`execution.rs`, `error.rs`	cancellation, deadlines, input/output, retained diagnostics
`array.rs`, `array/storage.rs`, `array/text.rs`	values, layouts, compact and mixed storage, gathering, source formatting
`agreement.rs`, `keyed.rs`, `selection.rs`	rustymath's broadcasting `Agreement` with basedpl's layouts and their renderer; reading paired items; named and keyed alignment; selection, writeback
`primitive.rs`, `primitive/`	primitive metadata, ranks, axes, inverses, array operations. Structural glyphs read their arguments, take each result's layout and `Steps` from a plan in rustymath's `structure`, copy by it
`number.rs`, `element.rs`, `pervasive.rs`, `fusion.rs`, `explain.rs`	numeric domains; item operations; compact kernels; dispatch's gate (`fusion::gate`, the cost rule `waits`), nodes for copies, joins, selects and conversions, forcing (`resolved`); events `•explain` records; live nodes `•memory` counts
`graph.rs`	the one lazy graph: interned nodes (`graph::Node`: a step, operands read through rustymath `Mapping`s, a `Desc`), their work, forcing a node through a `program::Program` and keeping its result
`derivative.rs`, `linear.rs`, `optimize.rs`	`∂`: duals, tags, seeding, rustymath's partials through rows, operator rules, loops' JVP and transpose; the transpose and substitution passes over the graph's linear nodes; `⇄` through `rustymath::optimize`
`transform.rs`	core shared by running transformations: one stack of levels; routing to the newest level among a call's arguments; the read point for code that isn't a call
`batch.rs`	batched maps: the batch carrier; `each_pair`'s try, calling the operand of an Each, a Rank or an outer product once on batches of cells while the `batch` pref is on (default); batch rules routing hands calls to. `tests/reference.rs` runs every reference case with it off and on; `bpl-tests check_page PAGE --unbatched` checks a page with it off
`plan.rs`, `program.rs`, `program/`, `jit.rs`	plans giving a call's result kind and layout from its arguments' (`plan.rs`); `Program`, `Step`, `Desc`, `Key` and the program cache: `Programs` on a dfn's definition, `program::tacit` by structure for other functions (`program.rs`); real-run traces, recordings and planning runs (`program/trace.rs`); replay and step execution (`program/run.rs`); the rule table (`program/rules.rs`) and contraction's builders (`program/contraction.rs`); block stages running elementwise steps a block of items at a time (`program/blocks.rs`), on compiled kernels (`jit.rs`) where compiling repays, else the interpreter's kernels; storage orders (`program/layout.rs`); contract and range checks in reference runs (`program/verify.rs`)
`search.rs`, `number_theory.rs`, `polynomial.rs`, `distribution.rs`	array-level adapters to rustymath
`system.rs`, `data.rs`, `files.rs`, `date.rs`, `regex.rs`	system-function metadata, native functions, options, import rules; paths and the filesystem; `•date`; `•r`, `•replace`
`csv.rs`, `json.rs`, `xml.rs`, `image.rs`, `plot.rs`, `canvas.rs`, `display.rs`	formats, rich values, rendering
`protocol.rs`, `reference.rs`, `timing.rs`, `settings.rs`	process encoding, semantic checks, measurements; the settings `•prefs` changes
`host.rs`, `host_web.rs`, `js.rs`	platform services; JavaScript values and functions
`symbols.rs`, `inspection.rs`	shared names, shortcuts, syntax and help metadata, inspection
`cli/src/`, `py/src/lib.rs`, `wasm/src/lib.rs`	frontend boundaries

User-facing language rules: nbs/syntax.qmd, evaluation.qmd, numbers.qmd, arrays.qmd, terms.qmd, glyph and system pages; define terms in terms.qmd before use. Sections below: contributor contracts and ownership, not the language reference.

Parser, evaluator and scopes

The lexer/parser records byte spans and structural syntax without evaluating expressions. parse distinguishes complete input, incomplete input and errors. It reads numbers under the current settings, which are process-wide (settings.rs). Session::parse passes the parser the hanzi_aliases setting. syntax_tree supplies source tools with structural nodes and recovers invalid chunks as Error nodes. highlight shares the lexer.

symbols.rs owns APL, simplified and traditional Hanzi spellings. Its rows also hold each glyph's names: a glyph name for the character, which is also its page's file name in nbs/glyphs/, and a name for each valence, or one name where both valences do the same thing. The glyph reference, named entry and help read them there. Python's exported names read them, with each system function's name, through symbols::builtins. hanzi_aliases enables Hanzi glyph spellings, canonicalizing token decisions while retaining original source and spans. Other Hanzi remain name characters. _ quotes the next non-whitespace character into a name, including a glyph alias. The notation preference affects structured function/operator display, not stored source, serialization or structural APIs; help and editor metadata use the same spelling table.

The binder resolves runtime categories right to left. It owns unfinished strands, trains and bound arguments; shared function nodes are immutable. Grammatical reduction produces application requests instead of calling functions. An array that holds a function as an item has its own binder category, Callable (Category::bound), while Category::of, which assignment's grammar reads, keeps it Value. Binder::push works out each entity's role once, as it pushes it: Category::before resolves Callable against the role of the entity below, which is to its right, to Function where that entity is a subject or an absent ⍺, and to Value otherwise. Rule::get and the strengths don't change, and recordings check Callable as a category of its own. A function-only operand slot converts such an array to FunctionNode::Array in constant_operand, and Function::from_value makes the same conversion where a call by text needs a function. Calls give values: a call's result, a subscript's selection and a scope's result bind as arrays, whatever they hold. Bound::given is the rule's one home: it makes a function result a function atom and gives the Bound the category Value, which Bound::category returns in place of the value's own. Entity::held puts a Bound on the binder's stack with that category, and the recorder's leaf and group take the &Entity, so a recording checks the category the value bound with. A field read from a name is the one exception: Entity::grouped gives that group its field's binding back, as Session::static_category reads it. Dot access is written as a ⊃ call (assign::dot_path), so ("k"⊃T) x calls the field too. Primitive/operator rows own names, operand kinds, ranks, pervasion, axis behaviour and reduction identities. Change those rows instead of duplicating metadata in consumers. A non-extending primitive above its natural ranks runs through Rank; pervasive Each/Outer/Rank paths reuse whole-array kernels.

Dot names rewrite to Pick and dot brackets to selection. Numeric subscripts derive selection on arrays and ⍠ on functions from the bound operand. Writable subscripts normalize to selection calls at assignment. ⱼ, ₑ or ᵣ between two arrays then rewrites to the primitives it stands for, as xⱼy to x+y×0ⱼ1 (syntax::joined_term). Dot parentheses evaluate a namespace: T.(expr) and T⍎text bind each key on the first axis of T to its item or major cell as a local, resolve other names in the caller and keep assignments local. They share the scoped execution route and lexical lifetime checks. Between functions, dot remains inner product.

Bindings (eval/bindings.rs) holds every binding: the frames of running calls and scopes, each module's names, the current frame and the running module. Other code reads and writes bindings only through its methods. A lookup gives where a name is bound, as a Place, with its binding, and reads, writes and removals take a Place. Bindings::here gives the place where a plain assignment binds a name. Parsing interns each name as a Name (name.rs), a number that is the same in every session, and a name becomes text again only where text comes in or goes out. Each frame has slots on one stack that every frame shares: first one for each of the six names that a call binds itself, then one for each of a dfn's locals, the names before an ← in its statements (Definition::locals). A call allocates nothing for its names. A call with no left argument binds ⍺ to an absent argument, so lookup finds the call's own ⍺ and never an outer frame's. A name without a slot, such as one that ⍎ binds or an assignment in a scope binds, goes in its frame's side map. A scope's frame holds its record, and a lookup that the side map misses there reads a key through the record's own index of its keys (Keys::position), so a scope copies nothing and a key's text never becomes a Name. Each module keeps its names as entries that an index finds, and deleting a name empties its entry without moving the others. A compiled read of a name carries an address (Bindings::address): the slots on its lexical chain that can hold it, and its module's entry. It reads through Bindings::load, which looks the name up as an uncompiled read does where a frame on the chain binds names without slots or the running module isn't the recorded one. Lexical frames use non-owning parent indices. Closures see live bindings, not snapshots. A dfn's last read of a local, ⍵ or ⍺ takes the binding out of its slot (syntax::release), and leaves Binding::Released, which stops lookup there with a VALUE error. The binder hands a dfn call the arguments it owns (Argument), so ⍵ and ⍺ hold them without a copy. dispatch gives a dfn the arguments as its caller gave them, and so a composition or fork with an operand that can use them (Function::owns, set when the function is built); it lends them to any other composition or fork, and a compiled statement gives a group's train the arguments that it holds (operators::train_call). An operator hands its operand each value that it made and doesn't read again: ∘'s inner result, a fork's tine results, ⍣'s states, a fold's running result, and the cells and selections that it gathers. It gives its own arguments to its operand's last call, as ∘, ⊘, ↣, ↢, a unit ⍚, ⍣ with one count and a fork's left tine do. It lends them where it reads them after a call, as Under, Rank, Axis, stencils, Over, inner products and ⍣'s other forms do, and it lends any value that it reads again, such as the state that ⍣∞ compares and the batches that a batched try reruns after a stop. A train reads its arguments in its last part and in each fork's left tine, and the last of those reads takes them. Plain assignment is local; modified/selective assignment updates the nearest existing binding. Retain the resolved place and original array across modifier calls rather than looking the name up again. Returns and outer writes reject functions, including those nested in arrays/prototypes, whose frames would not survive. Tail calls retain dependencies from arguments and the called function. No collector is used; recheck this lifetime reasoning before permitting escaping lexical captures.

Module 0 owns session names. Each •load runs in a new module, without the caller's frame and with echo off, and returns its non-private names alphabetically. Modules remain for the session's lifetime. Recursive loading of a canonical path is a DOMAIN error. Paths starting ./ or ../ resolve against the source file; other paths use the working directory. Execute/load do not run the top-level preference header.

Dfns run statements in order and return the last statement's value. Final assignments are silent. An absent left argument propagates through the binder's absent-binding rules; default-left assignment runs only when needed. Guard handlers see bindings as they stood at failure; nothing rolls back. Wildcard ∞:: excludes interrupt, timeout and unsupported errors. Explicit named guards can catch them; catching an interrupt clears its flag.

The binder compiles a dfn statement on the statement's first evaluation (eval/tree.rs, Session::statement_step). Its recorder follows the binder's two stacks over registers and constants, and emits an instruction for each node that needs evaluating and for each reduction. A move between the stacks and the drop of an absent ⍺ compile to nothing, and a derive or a bind on constants compiles to a constant, so an operator bound to a constant operand, as in ⍤1, derives once. Later evaluations run the instructions: they make the same calls in the same order, checking the category of each name, group and call result against the recorded one. Each checked instruction keeps a template of the binder's state at that point (binder::Resume): what its two stacks hold, its cursor, the leaf it had just resolved or the result it had just pushed, and the assignment it was making. Where a category differs, the binder carries on from that state (Binder::resumed). Registers hold the binder's own entities (binder::Entity), in an inline array of 8 (INLINE) or on the heap for a statement that needs more. The recording keeps its constants as entities, and an instruction borrows them, so a call reads a constant's array or function without copying it (array_of, function_of). The parse tree depends on the evaluator, because each syntax::Statement holds its own recording. A change to a reduction that the binder records (a drop, a strand, an attach, a call, a derive, a bound right operand or a train) must change the recorder and the instructions too. strand, attached, split_call, derived, bound_right, train and tails in eval/binder.rs serve both, and the binder tells the recorder its cursor and whether a call is its last reduction. The _ => rc.fail() arm in Binder::reduce stops recording any statement with a reduction that the instructions don't cover, and Recorder::finish then gives no recording. The binder evaluates that statement correctly every time.

Each group or run in the statement is a level, whose instructions sit inline between an Enter and an Exit, so one loop runs the statement. The recorder writes a level's instructions straight into the statement's as the binder evaluates the level, keeping a mirror of the level's binder (tree::Walk) while it runs. A failed check resumes the binder of the level that holds the instruction. Where the group's value keeps its recorded category, the instructions after its Exit carry on. Otherwise the level that holds it resumes, and so on out to the statement. Dot access, subscripts, joiners and runs that start with an operator rewrite a statement before the binder walks it (Session::joined, Session::members_with), and the recording keeps the rewritten nodes. Whether a rewrite applies, and whether a run starts with an operator or ends in a dyadic one, depend on what names hold, which one reader reads without evaluating anything (Session::static_category): a name by its binding, a dot path by its field, and any other group by the operand grammar of assignment targets (Session::trailing_operand). While a statement records, the reader keeps a fact for each name or dot path it read, with the name's address and the category it found (tree::Fact). A recording checks a level's facts again when the level starts, and the binder evaluates the statement or group from its own nodes where one differs. A run that ends in a dyadic operator is a level whose binder starts with the run's right operand on its stack, read where the level that holds it keeps it.

A group or run whose leaves are all literals, primitives, operator glyphs or such groups, and which makes no call, records as the array or function it gave, so a train like (+/÷≢) builds once. The recorder drops such a group's instructions by truncating the statement's back to where the group started (Recorder::truncate). A run that ends in a dyadic operator, whose right operand is constant, records as a constant too. A group whose last step builds its value as a train exits with the train's tines (Term::Tines). A call of that group calls the tines in place (Op::CallTines, operators::train_call). The call builds the train and calls it instead while a transformation or •explain runs, where the last tine isn't a function or two array tines are side by side, and where the train may hold a fast combination (rewrite::may_combine), which only the train's function computes (operators::calls_tines).

An assignment compiles to one instruction, which runs Session::assigned, the step after the target is found that the binder runs too: Session::assign, the one home of every target form, or a store of a plain name straight into the frame's slot for it, which Bindings::here found when the statement compiled. What a target's names hold decides where a target of more than one node starts (Session::assignment_start), so the instruction checks that start again, and the binder carries on with the assignment in progress where it moved. A default argument, an error guard's kinds and a dfn's last statement compile as other statements do.

Each item of a bracket literal is a level whose value goes into the literal at once, as Session::item_result converts it, and Session::bracket builds the literal for resolve and for the compiled Array instruction alike. A literal with a computed key that isn't constant stays with resolve. Each statement keeps the recording of its first evaluation (Statement::recording). A later call whose ⍺ is absent where that evaluation's held an argument, or the reverse, resumes the binder where the statement reads ⍺.

Tail calls reuse frames while preserving lexical dependencies. Handlers disable tail reuse. Evaluation depth is capped at 20,000 natively and at 244 in the browser, whose stack runs out first. Each call of a function other than a primitive counts toward it, and so do each statement of a dfn other than its last, each group, each run that ends in a dyadic operator and each bracket item, compiled or not. A compiled statement or level counts with Session::descend alone, without the interrupt check and stack growth of Session::nested. Calls of primitives other than ⍎ don't count, because a primitive never calls back into BPL. They skip the depth, capture and stack bookkeeping, and on numbers a form that applies a kernel to each pair of items runs that kernel at once (Primitive::scalar_call). A separate 20,000-frame limit bounds retained lexical frames. Late-bound name resolution stops at 128 levels. These are execution limits, not a sandbox against arbitrarily deep source, arrays or function graphs.

Values, layouts and assignment

Value: number, character, function, operator or shared array.

Compact storage: a rustymath buffer::Buffer of one dtype (Booleans, narrow integers, width-specific floats or complex numbers, characters); Items borrows it as a buffer::View. Promotion rules for those dtypes stay in basedpl (Promotion, array/storage.rs); a rustymath buffer keeps the dtype it's given. Ordinary integer array width (array::ordinary_width): narrowest holding its values under dynamic kinds; the int width under static kinds, where storage follows kind.
Mixed storage: Values; memoizes prototype, innermost lexical dependency, whether any item is a function; other properties computed on demand.
Segmented storage (Segments): two or more arrays of one rank and kind, with no keys or names on their axes, held flat as one leaf array, with each item's offset and shape, as DuckDB's list vectors and Arrow's ListView hold lists; an item reads as a view of its range of the leaf, and a view of nested items takes its own first item's prototype (ArrayData::nested_prototype). Value::gathered builds it with no copy where every item is a view of one stored array (Segments::of: Rank's ⊂ cells, a partition's parts, a selection of segmented items). A producer whose items are made together builds its leaf whole (Segments::new), as keyed::text_column does for •csv and •json texts, one range for each distinct text. Items computed apart stay mixed. Each's batched try reads items of one layout and kind as one array (Items::stacked: a view of a segmented leaf where they lie in order, else a copy). Search, classification and grade compare exact cells, a segmented array's as slices of its leaf, through rustymath's search::Exact. A write turns it mixed first (Storage::values_mut).
Layout and storage share one allocation. Shape holds ranks 0-2 inline → small arrays allocate no shape. Constructors take impl Into<Shape>: pass a slice or an array ([n]), not a built Vec.
Shape, key or name change on a shared array → a view of the base's storage, no copy. Part of compact storage (a Rank cell, a partition: Value::part) → a view of a range of the base's items; a view keeps its whole base alive (as NumPy's). Part of mixed storage → a copy (the memo covers all items).
A large result of a fusible float call holds a pending call in place of its storage, computed on first read (fusion.rs).
Value::view, imported_buffer: the public width-preserving frontend boundary; they give and take rustymath's View and Buffer, re-exported as basedpl::buffer.

Atoms, rank-0 arrays and singleton vectors are distinct. Function nodes hold handles, not lexical frames or Python objects. Functions match by parts; dfns and generators by identity. Operators can be values and cross the native Python boundary. Values and function nodes: Send + Sync.

Fill: numeric zero, character blank. Mixed nonempty arrays derive their prototype from the first item on demand; empty arrays need one explicitly. Builders take a value or a lazy prototype function, evaluated only for an empty result. Preserve function handles inside prototypes.

Layout: rustymath's layout::Layout (dimensions, position keys, axis names) plus the renderer. Methods changing only keys or names keep the renderer (as select does); methods building a layout from axes drop it; broadcasting two layouts keeps the renderer they agree on (Layout::paired). Rustymath's agreement pairs: unites keys, broadcasts through index maps; missing positions read each source's prototype. Equal axis names pair first; remaining axes pair positionally unless both have different names. Result assembly drops colliding axis names; public name attachment rejects duplicates. Contracted axes need equal key sets, keep the first order. Nonempty key lists with no named positions become unkeyed; empty record keys survive.

Gather owns copying and widening. Steps (rustymath's gather): strides, ranges and fills, rotations, offset tables. Gather::walk copies compact storage via rustymath::gather::walk. Structural operations should use remap/gathering, not per-item Value reconstruction. Equal-shape cells copy in blocks. Items reads counts, positions and axes from compact storage without boxing items. Shape, tally, positions, predicates: dedicated integer and Boolean builders.

Fill choice: ⍠'s options record or a window specification's fill key (WindowSpec, primitive/structural.rs). Primitive::fill_for validates one scalar item of the data's type (Value::fill_of); nested data takes any BPL value. Native fills travel with rustymath's mappings through pending copies, kernels and traces. copied_fill consumes fills of mixed data eagerly. Keyed pervasive calls attach native fills to their agreements without copying operands first. Values keep the chosen fill; tangents and label ids read their own zero fill. Maps needing two different fills keep separate copies.

Writes: Under with a function, and selective assignment, run the selector on the root while every call builds a node, then walk its result back to the root's positions (selection::selecting). A compact root enters as one input node. Any other root enters through leaves (selection::Leafed): each compact array or atom, at any depth, as a view of the leaf node for its type of item; each nested array rebuilt from its parts; each host item (exact number, function, operator) copied to an address of its own (Value::readdressed).

The walk follows copies, gathers and joins back to the root (root_places). A result that reads the root whole in order → a walk over it (Targets::Walk); otherwise offsets, or paths where the root has nested items. A part found by its address, or by its place and shape in its leaf, is taken whole: its write replaces it at any length. A selecting form whose result is its argument itself gives a copy node (Primitive::call_axis), which selects every position; a form with Facts::passes gives the argument itself, taken whole.
A root with duals on it walks through each value's primal (undualed), and transform::written writes each part. A regular batch of the running batched run: the selector runs once, on a batch over its whole (framed). Any other batch stops its run. A result that no root node holds → Under uses the selector's inverse; in selective assignment it's a DOMAIN ERROR, as is an element from elsewhere.
Function-Under applies its selector to a left argument as a plain value; positions-Under passes the left argument whole. Keyed values align to selected keys; nested paths keep empty-cell structure.

Selective assignment removes the binding before writing; Arc::make_mut copies only when another holder needs the old array. Repeated targets must read values written by earlier occurrences. pervasive::scatter combines each target of + - × ÷ ⌈ ⌊ into compact numbers in place, in turn, at the kind the call's plan computes in (plan::pervasive_on), via rustymath's gather::Writes (combine reads a selection's walk a run of offsets at a time, without listing it). Other pervasive modifiers group repeated targets into rounds of distinct positions. A target at a fill reads the fill; its write is dropped. Append updates: Value::append where layouts permit; rank, key or name changes → ordinary catenation. Modified destructuring calls modifiers left to right; ordinary destructuring assigns right to left. An assignment's RHS runs once.

Each, Rank and outer products call their operand via each_pair (eval/operators/each.rs): one batched call first (batch::tried), then once per cell, assembling results by shared layout and prototype rules. A 0-cell keeps an atom or encloses an array item. Custom operands: empty Each, Rank and outer product calls substitute prototypes for empty arguments, in prototype mode; explicit effects stay observable when a call runs; catchable prototype-call failures give prototype zero; interrupt, timeout and unsupported errors propagate; restore prototype mode on success and error. Stencil: Rank over ↕'s windows, reading its specification through ↕'s WindowSpec (primitive/structural.rs).

Rank tries whole before each_pair. whole, in order:

one operand call where neither argument has a frame;
decomposed: •decompose⍤2 on a stack of matrices factors the whole stack via rustymath's linalg::Stacked, as •decompose factors one;
whole_cells, Rank's whole-argument routes, shared with batched calls (batch::whole_call), in order: primitive::stacked (⌹, -.× on stacks of inexact matrices via linalg::Stacked, as rustymath's Python API runs them); every f.g with a rustymath product kernel, on compact stacks of any item kind, via linalg::Product (an exact product not fitting 64 bits keeps per-cell calls, which compute exactly); one padded call of a pervasive primitive (padded_call); primitive_cells (a primitive with one argument, or with a left argument every cell shares); paired_cells (matching frames, or a right argument every cell shares). An argument with no axes besides its cells' is the same in every cell;
pervasive_cells: a pervasive primitive under nested Rank;
monadic operand: cell_rule, which also takes folds, atops and a bound left argument.

primitive_cells runs a primitive on the whole argument along the cells' axes. A form that extends runs whole at a cell rank at least its own. Other forms follow the cells fact (CellRule) in their Primitive::info() row: ⊖ , ↑ ↓ ⌷ # and rotate-first call their axis forms; ⍉ sends each cell's axes after the frame's; ≢, ⍴ give constants; a reshape keeping a cell's items in order is a view; ⊂ encloses each cell of rank 1 or more via its axis form, whose cells are views of the argument's items. Any other form with a left argument every cell shares takes paired_cells.
paired_cells: joins each pair of cells for , and ⍪; calls a form that extends once where both cells are at least its ranks; first repeats an argument every cell shares over the frame (agreement::spread).
padded_call, paired_cells leave an argument with a named frame axis to the per-cell path, which pairs frames by name (named_frame). Names on cell axes pair in whole calls as per-cell calls pair them.
A pervasive primitive folds or scans a simple array along the cell's axis (folded_cells); f/∘, folds all cell axes at once. An atop applies its inner function's rule, then Rank for its outer function.
Each rule applies only where Rank's mix of the cell results equals the whole-array call's result. Other nested results, and folds by other functions, keep per-cell calls.

Axis nodes select or move axes, reusing Rank where the function has no direct axis meaning. Vector-axis reduction: the general composed-ravel rule, keeping axis order and the whole seed; single-axis reductions keep direct kernels. Layout-only changes share storage. Structural results keep renderers where rank permits; scans and in-place changes copy complete layouts. Match ignores renderers.

Inverse nodes propagate fixed-argument sides through composition. f⁻¹←g, f∂←g replace the binding with a Ruled node: the function plus its hand-written inverse and derivative; calling it calls the function; ⁻¹ and ∂ use the rules; Under counts an attached inverse as keeping every item. Candidate-building primitive inverses validate by applying the primitive again. Inverse scans share forward scan's seed and axis validation. Agenda calls its selector once, then passes the original arguments to the selected branch. Computed operands of Rank, Axis, Stencil and Agenda: shared operand evaluation. Python builds these nodes directly, not by generating source.

Form rewriter (eval/rewrite.rs): the one home of fast combinations.

Rule: a row of BPL text, a pattern over ⍵ and ⍺, beside the hidden Special that computes it (⍟1+⍵ beside Log1p); rules adds the selections, one per order and pick. A rule's functions must be glyphs and its constants literals: a name holding 1 or ⍟ isn't recognised.
The rewriter reads a function and an expression as one tree of calls (Form), replacing each subtree a rule matches with a call of the rule's function. Function::new rewrites a function's monadic and dyadic forms once, keeping the results (rewritten); dispatch calls them in the function's place; inverses, Under and display read the original node. Binder::reduced rewrites each list of nodes after joined and members_with. Every spelling, expressions included, takes the combination from its first call. calls_tines builds a train whose tines may_combine accepts; Function::new then rewrites it.
combined runs the structural replacements (selections, ranks, Key's counts and folds) before any running level. An algorithm unsuited to the arguments, or a running level carrying one → combined calls the functions the combination replaces, which the level then sees.
Each combination has a row under "Fast combinations" on its glyph pages. tests/explain.rs checks via •explain that every documented spelling of a structural combination takes it; core.bpl reference cases check each gives what its functions give in turn. The coverage matrix runs each rule whose function maps its argument (mapping_patterns), respelling any form's own case a rule would capture (rewrites).
A combination chooses an algorithm, which stays in rustymath or the kernel modules. A whole-array path that follows from an operator's own definition (Rank's for pervasive operands) belongs with the operator.

Numbers and backend ownership

number.rs owns scalar kinds, promotion, exactness, and the suffix table spelling numbers (e/ₑ, j/ⱼ, r/ᵣ, ₓ, width marks), read by the lexer, Number::parse and display. Ordinary machine integers and the unbounded ᵣ integer/rational domain remain distinct. Ordinary integer overflow wraps at the maximum machine width; dynamic kinds retain widening within machine widths. Integer ÷ floors. ÷ₐ selects float division and ÷ᵣ selects rational division before computation. A whole ᵣ result never becomes ordinary and prints as 3ᵣ, not 3ᵣ1. Python/JSON conversion policies stay unchanged, including imported unbounded values. Booleans: a distinct kind; arithmetic reads them as 0/1, logical operations preserve them. Match, search, grade compare numeric value; reference checks also distinguish representation and exactness.

Joining compact arrays widens Boolean → integer → float → complex; float widths join at the wider. Item-by-item construction keeps exact and approximate kinds apart, staying mixed when needed; only approximate real and complex items widen together. A write into an existing array follows the items rule (Widening::Items), as a literal of the same items would: exact and approximate numbers stay apart, as do Booleans and other numbers. Rearranging or combining mixed storage finishes in Gather::finish: compact where one dtype holds the items under the items rule; its scan compares each item's variant with the first's → a copy that stays mixed pays little. An in-place write into mixed storage keeps it mixed (a scan per write makes a loop of writes quadratic). Fresh computations choose storage from their result path. Results computed one cell at a time (Each, Rank, Key, power histories, polynomial and number-theory functions) assemble as items via Value::assemble; mix and catenation join arrays. Keep whole-array joining and element assembly distinct.

element::Converts: the one rule converting a compact item to another numeric type; size marks, •float, and programs' Convert steps in replays and block stages use it. Float target: reads the item as rustymath's Source does. Integer target: Boolean → 0 or 1; integer wraps to the width; float → floor within the float width's tolerance, then saturates at the target's range. Value::converted: plain values. derivative::converted: carried values, through the carrier (Carrier::cast); on a pending array, one more pending call (fusion::convert).

Bare numbers, empty literals, NaN: the parsed float width. Top-level evaluate_code validates the whole syntax, runs leading literal-record •prefs statements, then parses the rest at the new width; a syntax error prevents header execution. The default width also covers imported numbers, random data, and float results from exact operands. Time, date and metadata values: f64. Formatting marks non-default widths; serialization marks every float (width-independent readback).

Float operations compute at the wider operand width; exact operands round to the width computed at. f16 operations run on f32 copies when Numeric::halving says: a call with an f16 operand, or any call while the default width is 16 bits. pervasive::planned_halves applies it to whole operations, Number::halves to scalar arithmetic and comparisons; fusion::defer declines those calls. Exact inputs convert once, to f32; float outputs round back to f16. The scalar Number path rounds each f16 step. Complex parts: f32 or f64; f16 becoming complex uses f32 parts. Scalar Complex32 arithmetic computes at Complex64 and rounds; kernels can compute at f32. These paths needn't agree in last bits.

Floats: IEEE NaN, infinity, signed zero. Comparison: NaN unequal; match and search: one value; grade: last. Exact arithmetic dividing by zero can give those floats. Floor/ceiling keep an exact operand beside infinity; beside NaN the result is open: NaN or the other argument, varying by platform. Their folds, scans, windows, products and repeated writes give NaN or the maximum or minimum of the non-NaN items, never another number (rustymath's Folds doc): a loop folding by them passes its running result as the right argument, which rustymath::reduce::maximum keeps beside NaN, or folds through rustymath's Max/Min, as Dyad::extremum does. Under dynamic kinds monadic floor/ceiling/sign return machine integers; float-to-integer conversion floors and saturates at the target range. Static unmarked float forms retain floats. Outside number.rs, use Number::float(value, width) and explicit-width array construction, not implicit f64 conversion.

Static kinds (•prefs ["kinds":"static"]): a result's kind follows from its arguments' kinds alone (meta/kinds.md).

Numeric carries kinds and int. Until the float pref sets a width, Numeric::set gives new floats the kinds' width: 32 bits static, 64 dynamic (Kinds::float_width). Whether the pref set one: settings::float_set, outside Numeric (every call builds and copies a Numeric; a Boolean field there cost about 14 instructions per primitive call, 34 per dfn call).
A pervasive call on arrays plans before computing: plan::pervasive builds a plan::Pervasive from operand kinds (plan::Kind: type, fixed width, weakness) and settings, reading no items. It holds the settings with the operands' fixed width (Numeric::for_kinds), whether 16-bit floats compute in 32 bits, the computing kind (computes_in: rustymath's DType::common after Numeric::kind), its kernel table (computes) and, under static kinds, every result's kind (Pervasive::result). Eager dispatch, fusion's deferral (Pervasive::fuses) and typed programs read the same plan.
Products and folds plan likewise: plan::Product holds g's pervasive plan and the settings f folds its results under (Product::folds); plan::Fold a fold's or scan's settings and operand kinds. Eager products, folds and scans read them, as typed programs do.
Comparisons, ⌈ ⌊ and bitwise functions compute at their arguments' storage width, where results are the same at every width (pervasive::first_kind). A comparison of integers with a whole float standing for an integer computes on integers via Pervasive::computes; so do a residue by such a float and a residue scan from one, whose results then convert to floats (plan::whole_float).
Weak (Number::is_weak): literals, values computed only from literals, Python's scalar arguments. Before a call, Primitive::weakened converts a weak number beside a strong one to the width rustymath's DType::common_weak gives, or to a fixed integer's width where it fits. Weak integers have no fixed width; NEP 50 decides theirs.
Loops keep the failed results keeps names: floats' NaNs, wrapped integers of objects reporting WRAPS. Kernel tables choose float ⌊ ⌈ × and wrapping negation from run.numeric(). Stored::build_at keeps complex results complex.
Primitive::number gives every pervasive result on numbers and each step of a fold of numbers; under static kinds, statically converts it to the plan's kind. A static call on ordinary integers that an integer kernel takes runs it at the int width, as fixed integers run at theirs (Primitive::integer_kernel). Under dynamic kinds a call on two numbers stays outside plans: its result type depends on their values.
Number::wrapped, Value::wrapped: wrap an exact integer result to the integer width: the fixed width Numeric::for_kinds finds among the operands, or int under static kinds. Under static kinds a strong ordinary integer is a fixed integer of the width it computes at, int joined with its storage width (plan::Kind::stored) → every static call on integer arrays computes under a fixed width. Display marks a fixed integer of int width with plain ₓ (Mark::integer).
A fixed kernel failing where a result doesn't fit (lcm, binomial coefficients) gives the exact result wrapped (Primitive::fixed_number), used by statically and by exactly computing functions (⊥, ⌻, the determinant). Where integers wrap, a function with approximate results (⌹, polar parts) computes integers as floats of their float division's width (plan::Kind::approximate); ∂ converts its argument's wrapping integers to those floats before running its function (derivative::differentiate). Number::parse keeps complex literals and checks integer literals against the int width.

Settings: once per process in settings.rs, read from there, not passed through functions, except where a read is slow and on a hot path. Atomics: each setting calls and scalar operations read: numeric settings, hanzi aliases, the batch pref, the verifier flag, the fusion report flag, fusion's sizes (settings::set_sizes), the compile setting. Display settings: behind one lock. Writers: •prefs; the lexer, for a source's leading •prefs header as it reads it. settings::reset restores defaults (]clear, test harnesses). A module •load runs changes settings process-wide; nothing restores them on return. Thread count: held by rustymath, the one setting it reads itself; •prefs writes it there.

Each call reads the store once, at its start: pervasive calls, folds and products as they plan (Primitive::pervasive_mapped); a dfn call as it looks for its program (program::call); Domain::of; the lexer. Values derived from settings within a call stay explicit arguments.
A call's plan holds the settings it computes under → a pending array computes under its calls' plans, whatever settings are in force then. A pending conversion holds the settings giving its floor's tolerance (Step::Convert's numeric). A dfn's programs are found by the settings they were traced under (program::Key); under other settings the dfn traces a new program.
Passed explicitly: loop structs keep the settings (their closures run on rustymath's worker threads); Number's tolerance methods and Primitive::number take them (once per item); the parser's leaves take a width marker's width; matches_within takes ⍣∞'s convergence tolerance.
Imports build complex numbers with Number::complex_in_session: Python's _Session.array, the JSON protocol's request. JavaScript values hold no complex numbers.

Numeric::mathish_at(dtype) decides whether a call takes the Mathish forms: Numeric::mathish_on (preference on, kinds dynamic, no fixed width) and rustymath's DType::mathish (the call computes at f64 or complex128), the one home rustymath's Python shares. Integer kinds take the ordinary forms, which give the same integer results. Kernels select their numerical objects, from rustymath, before the loop. Mathish comparison tolerance 1E-14; f32 and f16 compare exactly. Fixed-point convergence: four ulps at those widths, 1E-14 at f64, independent of Mathish. Index, count, Boolean and character-offset conversion: near_integer with the fixed default tolerance. Test with independent inside- and outside-tolerance cases and exact/mixed controls, not language comparison as its own oracle.

search.rs selects compact exact or tolerant backends.

Classification of 1,024 items or more first groups items sharing one array, before reading any (search::shared_classes). Imports make repeated texts share: keyed::Texts makes each distinct text once per •csv column, •json document and Python conversion, up to 65,536 distinct texts.
Exact integer, Boolean and character storage uses keys; float searches use their width. Integer haystacks take the whole-number shortcut only for exactly whole float needles. Tolerant indexing validates whole-key matches unless the haystack is exactly whole, falling back to buckets on mismatch. Classification chains tolerance through sorted runs by design: it needn't match every member to one representative. Other cells: exact hashes or first-number buckets with Value::matches confirmation; pairwise comparison where indexing is unsuitable. Grade is stable.
Position buffers: u32, becoming integer storage without another pass. ⍷ and monadic Key take each class's positions from search::class_positions, which keeps a class's own buffer for 32-bit positions and copies narrower ones once from the shared buffer. Dyadic Key selects rows through Groups::each slices. Key's group-fold idiom folds through search::numbered's class numbers; where fold_groups declines, Key builds positions from those numbers. Key with operand ≢⊢ (also built by ≢∘⊢) counts classes via search::class_counts, building no positions.
Under Rank and in batched maps, ⍳, ∊ and ⍸ whose haystack differs per cell search each cell's own haystack in one call on the whole arguments (primitive::searched_cells).

Domain, with_field!: select working reals, Complex64 or BigRational for generic numeric algorithms; compact storage selects the domain without scanning items. number::read converts once; Field::array builds compact float or complex results without a Number per item. Each caller lists the domains it supports. Compact complex input doesn't take a real path merely because all imaginary parts are zero; results built via Value::complex can normalize to float storage.

Rustymath owns mathematical algorithms; basedpl owns pervasion, storage, numeric promotion, axes and language policy. Code whose inputs and outputs are only buffers, shapes and plain numbers belongs in rustymath, even if only basedpl calls it. Approximate matrix solve, pseudoinverse and decomposition: rustymath factorizations; on macOS rustymath sends much of it to Accelerate (its DEV.md); no platform-specific adapter here. macOS deployment minimum 15, set by the Cargo configuration (Cargo's builds) and pyproject.toml (wheels). Exact solve, pseudoinverse and determinant: rational field algorithms. Generic permanent and polynomial helpers also run at floating and complex field types. Singular or wide matrix division: least-norm solutions. LU eligibility and SVD cutoffs: rustymath policy, not duplicated here.

Hash maps and sets: rustymath::hashing's HashMap, HashSet and Hashing (rustymath's DEV.md), not another hasher → every lookup in both projects inlines its hash the same way.

•decompose/•decompose⁻¹ share domain reading and Factor dispatch, with options for kind, vectors and complete QR. Polynomial adapters: compact evaluation or generic field operations; gradients differentiate coefficients before evaluating. Number-theory adapters: cancellation checks, result cell assembly. Distribution constructors expose native density, CDF, quantile and sample functions; compact continuous calls use slice objects; discrete samples and finite discrete quantiles are exact integers.

•rand seed: a key and a count of draw calls behind a mutex (rustymath::random::Counter), shared by every copy of its record. ¿, and a distribution's sample without a generator, draw from the session's generator; •prefs ["seed":n] reseeds it. Draw call c reads stream c under the key from its start. Item k of a call reads its packed words, then its own region (rustymath::random::Items), so an item reads the same words on every route. The algorithm is fixed, but range or distribution sampler changes can change draws: the reference sequence detects that. Sampling from a generator record uses its roll function to recover the same generator.

Kernels, fusion and threading

Dispatch selects a primitive before its loop; a pervasive call's plan picks its kernel table (plan::Pervasive::computes). Concrete Kernels tables call rustymath objects: many for slices, one/checked for scalar Numbers. at_widths! writes f32/f64 tables once. Arithmetic, circle-code conventions and scalar policy stay in the numeric modules; no second evaluator.

Mixed item types: Converting where the object loops item by item in rustymath (Results::INLINE), converting each item as read; else Chunked, reading each argument through rustymath's Side a converted run at a time into one homogeneous implementation → no storage-type cross-product of machine code. Compile only operand pairings that can occur. Same-type arguments can reach rustymath in one call; other shapes expand reused chunks or use block mappings, avoiding per-item division.

Custom elementwise closures: rustymath::pairs and its uninitialized append path; function objects use their own slice forms. Keep per-item bodies inlinable across codegen units: a closure passed by reference can add an uninlined Fn shim → pass it by value, or wrap the reference in a new closure. Hot loops: slices and fixed chunks, not iterator helpers that add calls in the incremental build. Avoid captured mutable flags held in memory: they block vectorization.

A failed compact kernel leaves placeholder results and reruns the whole call on a wider, exact, complex or scalar path; complex fallback shares Number policy. Overflow detection belongs to rustymath: its arithmetic objects' integer forms, checked reductions, checked products. Cheap integer multiplication picks an exact width from bounds, with no per-item overflow check. No blanket NaN or signed-zero canonicalization pass; algorithm-specific failure checks and rustymath's SIMD kernels stay inside their algorithms.

Fusion (fusion.rs; its module doc owns eligibility rules and limits: don't copy its thresholds here):

Calls a block stage can run (plan::Pervasive::fuses) defer: Primitive::pervasive_mapped returns a pending array (Data::Node holding a fusion::Pending): a typed Program of its one step, plan fixed at call time, plus the arrays and atoms it reads, each pending one in its own value (fusion::held). Pending arrays survive assignment and link to the pending arrays they read. Node-making code stays out of line from dispatch: inlined, it grows the frame kept on the stack at every recursion level.
A read forces via ArrayData::storage: gathers in the programs of pending arrays it reads in order (Pending::gathered), runs the result through its plan's units. program::planned plans each program structure once, from a copy with inputs for held arrays, reusing the plan for later forces of the same chain on new arrays. Shape, kind and prototype readers (Value::len, Value::environment) don't force. Value::force forces, as timing does.
Fusion and dfn traces both add nodes via program::add, which returns an earlier node making the same value: a held atom of the same identity, the same held array, or the same step on the same values.
A pending array read in order that a name or another array also holds joins as an output (Program::outputs), computed in the same pass; a block stage stores its items with the result's. Other pending arrays the program reads but doesn't compute as steps compute first, alone (Pending::gathered): those not readable in order, and held copies. A fold of a held pending result stores that result as an output of the fold's pass, once for all readers (fusion::fold); a pending copy folds so, held or not.
A block stage runs elementwise steps block by block, in parts (blocks::Part): each run of steps lowering to machine operations that give their kernels' exact results (blocks::lowered_op) through one compiled kernel (jit.rs); other steps through the same pervasive kernels (pervasive::node). A part whose kernel can't take a run's items (e.g. an input a broadcast reads as one item) runs those kernels for that run. A stage compiles all parts or none: where the same kernel compiled or ran before, or where one run of its items repays compiling (blocks::Blocks::compiles, repays). settings::Compiles can compile every stage or none: the reference harness; timings via BASEDPL_COMPILE. A call waits where fusing it saves more than its node costs (fusion::waits: NODE_NS, the allowance per call, against SAVED_NS_PER_BYTE); BASEDPL_DEFER=all makes every call wait, for timing what a node costs.
Each step writes a block to a buffer later steps reuse once its readers ran. A comparison read only by later steps of the stage writes floats 1 and 0 there (Dyad::compared, rustymath's tolerant::Indicator). ⍚ and @[p] run cases that can't fail on every item, selecting by a Boolean mask (selected, eval/operators/agenda.rs); plan::select gives its type and the maps its mask and cases read through; fusion::select defers it where the mask is such a comparison, else runs it at once via rustymath's operation::choose (fusion::chosen). A one-step stage runs only where it reads through a copy, recording no fused row.
A structural copy of one source through rustymath Steps (remapped) large enough to wait (fusion::waits) is a pending copy (fusion::copy), any compact kind; its map is a Mapping::Steps, one entry per result axis, as broadcasting's are. Forcing composes a read of a copy with its map and moves a read with no repeat or fill under the pervasive calls it reaches (fusion::resolved): 2×⌽v+1 reads v backwards in one program. A copying read (rustymath's Mapping::copies) moves only under a one-array call. A call whose two array arguments copy same-layout arrays through one map runs on those arrays, its result copied once (program::rules, copy_lifted). Other arrays: a Copy step, which a block stage reads through without making the copy. A step reads items in order under another shape through its operand's program::Read.
A block stage shrinks blocks whose buffers wouldn't fit in cache. Rustymath's agreement::expanded reads a forward run in place, else copies via gather::walk_plan. A Mapping::Steps keeps the plan Steps::simplify makes on its first walk (agreement::Route) for every later block. A copy no call reads through is walked as remapped walks it.
Fused and ordinary folds share rustymath's reduce::along/fold_into traversal. A block stage of one compiled part folds the last step's items itself, handing reduce::fold_pieces each block's total (reduce::Part): sums, float products, maxima, minima. A float fold keeps as many running results as rustymath's folds (reduce::running). Boolean folds, float ⌈/⌊ folds, and blocks crossing a lane go through the last step's block.
On the pending route, a fold of a windows copy along its own axis slides over the copy's source (pervasive::slid, mapped_fold: rustymath's reduce::fold_route with the copy's source, structure::Slide, reduce::sliding); a fold along two window axes folds both in one pass (sliding_2d). A short fold a stage folds as one step per window position (blocks::short_window) runs as a block stage reading the copy's source at each position through the copies' maps: in a dfn's program, and on the pending route where the copy reads another pending copy (e.g. padding) into a result whose last axis exceeds DIRECT (fusion::staged). Under ∂, fold_rule takes a maximum's or minimum's positions in such windows from the source (pervasive::mapped_picks); under a trace, Step::Picks gives each lane's pick, via the untraced mask's function (derivative::chosen). A traced ⌈/⌊ scan, or picks from sliding windows, stops the trace.
Ordinary pervasive kernels read pending copies through their maps: fusion::source snapshots source and mapping without taking the pending work; rustymath composes that mapping with the call's agreement; pervasive::Input keeps the snapshot alive while existing blocked readers read its storage. Rustymath-reduction folds read pending copies via fusion::source too (pervasive::mapped_fold). reduce::along_mapped folds a copy reading a contiguous source block with reordered axes (per Mapping::block: transposes, reversals, non-overlapping windows) in the block's own order; totals in another order become a copy through along_mapped's tables (pervasive::fold), which the next fold reads as a block again. Other copies fold a run at a time where runs reach reduce::RUN items, uncopied; else a piece at a time, copied. A held copy read in shorter runs (reduce::rereads_cheaply) folds from the copy native_fold computes and keeps for later reads (fusion::held_copy).
A read of a few items goes through fusion::narrow, narrowing the pending work to the positions read without consuming it: a chain of copies → a read of its source, any item kind; an elementwise call, or a fold's leftover program, → one over those positions only, on its first nonempty partial read of at most 1 in 128 of its items, shared or not. Where maps don't compose (a rotation with an overtake carrying source items across the end), the read lists its items' offsets. The original keeps its work; its next full read computes and caches the whole result. Copy unwrapping, empty and all-missing reads don't count. fusion::item serves single items and picks; Gather::walk small copies and gathers via fusion::walk_through. Whole-input value checks (bounds, finiteness) read only items its logical ArrayView selects. Unsupported compositions and dense readers keep the pending array's one cached compact result.

explain.rs: events •explain reports. One check in Function::reported serves •prefs ["fusion":$t] and •explain: a call computing a pending argument writes the pref's line and records forced. Blocks::run records a fused group per block stage of two or more steps, fold included; Idiom::call each fast combination taken, by Idiom::route. Nothing emits rustymath's Accumulation or Mixed events, and nothing counts live record nodes or pending arrays, so •memory's records and pending are always 0 (open.md). •explain and Python rm.explain use the rustymath::events capture: each native module captures its worker threads through one process-wide sink; captures can nest on one thread; the shared Route declaration supplies fast, decline and fixed itemwise IDs and their coverage inventory. Successful routes aggregate call and item counts. Optional declines stay separate from stopped traces, which record a slow fallback on every execution, cached failures included. Outside capture a native event checks one atomic flag. Value::footprint walks a value's allocations for •memory's bytes.

∂ (derivative.rs):

Dual: tag, primal, tangent: a transform::Carrier in Value::Carried (number) or Data::Carried (array); primal and tangent can be pending. A record of linear calls is the other carrier.
Routing (transform::call) sends a call to the newest level among its arguments (Value::newest), reading their outer layer; a newer level inside an argument in mixed storage wins (an array of dual atoms beside an older dual). Newest level a ∂ call → derivative::call, run by the form's facts in Primitive::info() (primitive::Facts): a pervasive form's partial derivatives run as primitive calls, from the recipe of the function object the kernel table picks (partials, pervasive::Dyad::differentiable, rustymath::derivative); ⌻ has its own rule; forms whose results carry no tangent run on primals.
Structural glyphs, Under and selective assignment need no rules: their code moves each part of a carried array through the carrier's moves (Carrier::view, Carrier::at, Carrier::copied; writes: transform::written, which a record answers through its ids). Gather copies a dual's primal and tangent together (derivative::Join).
A glyph copying its right argument through one plan, and a broadcast, copy a record tangent as one Op::Copy (primitive::copied; Form::Moves's planned), holding the plan's Steps as a rustymath Mapping (Mapping::walk), as pending copies and Step::Copy do. Its transpose reads through linear::transposed (rustymath's Walk, Steps): the inverse map for a copy reading each item at most once; a correlation for windows moving by 1. A copy of a copy transposes in one read where the composed copy still transposes by reads (linear::composed); maps compose via rustymath's Mapping::through, as in fusion and traces. The record keeps the chain: recognising windows needs each copy's own shape. A sum along a window axis transposes as a sliding sum (linear::slide); a spread axis as a sum along it. Windows moving by more than 1: rustymath's signal::overlap_add, adding each window back where it read (linear::strided_back). Other copies, and copies of several sources, keep ids and scatter.
Rules: reverse mode's ⌈/⌊ fold: the tangent times a mask of each lane's chosen item, summed; its transpose reads. Mix of a dual with a nested primal mixes primal and tangent apart. fold and scans with + × ⌈ ⌊, +.× and outer products have rules; kernels reading numbers directly leave active arguments to general paths. Folds by primitives, products of primitives, ⌹, ⊥ and stacks of matrices route through transform::call as operations (primitive::Operation): their rules (derivative::solve, decode, product_dual, determinant, fold_dual) run on a &Context, calling Operation::call for needed products, folds and inverses. Rank's whole-argument routes send carried stacks through Pairing::Matrices → ⌹⍤2, +.×⍤2, -.×⍤2 take stacked rules in one call, batch on or off. Attached rules, system functions, and folds and products by non-primitives take the same transform::call, as transform::Op's other variants.
⌹, -.× keep the glyph's own result as primal → same values with or without ∂. Tangents and transposes multiply by R: the inverse numeric::matrix_inverse takes from one factorization via rustymath's Stacked::inverse, or exact row reduction for exact matrices; regular_inverse gives UNSUPPORTED where rustymath's regularity test finds the matrix singular. Only B active: R is ⌹A, pseudoinverse included.
One number in x: f∂ x runs f once with x seeded. More: linearized pushes a record level just below its ∂ call's level, runs f once with each part of x seeded with a record at it (linear.rs): a tangent whose items are never computed, which the same rules extend into a graph of linear calls. Newest level a record level → records the linear map its form's facts name (linear::call). reverse reads the Jacobian from the record: a transpose per result item or a forward evaluation per number of x, whichever fewer.
An f∂←g JVP rule is called with ⍺ a dual of zeros whose tangent is the argument's → in reverse mode it extends the record (derivative::attached). Such a function called dyadically: DOMAIN ERROR wherever ∂ needs a derivative through it (derivative::two_arguments).
Code reading a dual array's storage gets dual atoms (correct, per-item speed). A rule packs an array holding its level's duals, at any depth, into one dual before splitting it (derivative::packed). Readers outside a call use transform.rs's read point: Value::observed (counts, positions, ranks, selectors, ≡, guard predicates) reads a dual's primal; Items' integer readers read carried items through it (Value::observed_number). Its Result's error, transform::Stop, names the level owning a batched run. A stop is ErrorKind::Stop: no guard catches it, even one naming it; catch-all handlers let it pass (ErrorKind::is_control). Display reads via Value::displayed (never inside a batched run). Value::computed (kernels, via primitive::numeric) gives UNSUPPORTED for a dual atom.
Writes to bindings outside a ∂ call are recorded and stripped when it returns.

Typed programs (program.rs, program/), run by pending work and ∂:

∂ of a dfn runs one program holding its whole call, value and derivative (derivative::differentiate). The first call with given argument types, layouts and settings traces it (program/trace.rs): the dfn runs on traced values (type and layout, no items); each call plans its result (plan) and adds a step via program::add. Where the trace stops (nbs/autodiff.qmd, "What stops a trace"), the call runs in the interpreter.
Programs keeps each trace's outcome, program or stop, on a dfn's Definition or another function's structure (program::Tacit, at most STRUCTURES). An outcome serves later calls with the same argument signature whose free values still hold what the trace read (Entry::reread) → cycling free values replay. Free values changing every call stop tracing after CHANGES traces in a row; a function keeps at most KEPT outcomes.
program::call replays the program on the call's values; program/run.rs replays step by step, the reference compiled code matches. Fusion's and traced programs share this code.
Index notation and its gradients: Step::Gather selects by computed positions; Step::At writes or accumulates where a map or selector says; Step::OverlapAdd adds a windows copy's cotangent back to its source; Step::Picks marks each lane's ⌈/⌊ fold pick. Level 2: rustymath's selection planning (structure::joined) reads an index array with affine positions through strides, not an offset table (Table::affine).
program/contraction.rs: a + fold of a × whose operands read sources through affine maps runs as one product (contracted); a fold of a copy repeating items as a fold of its source (fold_of_copy). program/layout.rs: stores maps in the order operands and readers need (layout::ordered). blocks::plan: finds runs of elementwise steps for block stages. program/verify.rs: checks each program's contracts before it runs, in every reference case and page (settings::verifies).
•explain lists programs a call built or replayed, and stopped traces (programs, stopped, declined). ["sites":$t]•explain fills site on forced and stopped rows: the innermost basedpl functions that computed the items or stopped the trace (explain::site, from a Rust backtrace, skipping the event's own bookkeeping, BOOKKEEPING). Use it to place a forced read or a trace stop, rather than a temporary backtrace.

Batched maps (batch.rs):

batch::tried (from each_pair): begins a batched level (transform::Kind::Batched), wraps each framed argument in a Batch carrier at it, calls the operand once, ends the level. A map first lays its arguments over its frame by its own agreement, the pairing its per-cell calls read (batch::Arguments::laid, agreement::laid_cells). One cell of atoms: no batched call (faster).
Routing sends calls whose newest level is the run's to batch::call: primitives, operations primitives name (a fold by a primitive, +.×), operations by dfns. ⍕ and random draws keep their own code; other system functions run per cell, results joined into one batch (batch::system, batch::each_cell). batch::call runs whole arguments by Rank's routes (eval::whole_cells via batch::whole_call; eval::folded_cells for folds) → a primitive batches wherever Rank runs it whole.
Folds and scans folded_cells misses (e.g. by a dfn) run the fold's own loop over the batch's items, one call per stage on item j of every cell (Batch::at); fold_array, primitive_fold join stage batches into one (joined). Every fold and scan route decides structural cases (unit, empty lanes, result shape) in FoldPlan and ends in folded, the only place giving the result its layout, fused folds included. Folds and scans by non-primitives over several lanes try their own batched run: operators::by_lanes moves the fold axis last and runs the derived fold over lanes via batch::tried; operators::by_triples, for f.g with non-primitive g: one batched g call on a frame of rows by contraction by columns, then the f fold along the contraction, per chunk of rows whose triples hold at most batch::CHUNK items (operators::by_rows, shared with products of primitives). A try giving None → the usual loop from the start. A fold keeps its stops on its operand's Definition (Function::remembers).
Stops: batch::tries holds every try's conditions. A call no route covers stops the run (transform::Stop). Constants with axes that, spread over the frame, would exceed both 4× the arguments' items and batch::CHUNK (2²¹) items stop it, naming a chunk of cells whose share is at most CHUNK (oversized, Stop::chunk); tried then runs the operand once per chunk (batch::chunked), each a batched level on a one-axis frame, joining results as the per-cell run would. A chunk stopping for size restarts with its own smaller chunk. One cell's share alone over CHUNK → one call per cell.
Code that must run once per cell asks the levels first (Levels::effect): output, input, ⍕ of a carried value, acting system-function code (Effect::Acts, via Context::acts); a guard or prototype call that would catch an error (Effect::Catches); a write outside the map's frames (Effect::Writes, via Levels::record_write). An observing read (Value::observed: a count, a position) stops the run too.
A guard on a batch of the run splits it (Session::guard via operators::guard): batch::truths gives each cell's Boolean; each side runs once as a guard case on its cells (Levels::case), through ⍚'s Cells; batched bindings of the run's frames read at the side's cells (Session::restricted, via batch::read_cells, operators::cells_at); Cells::assembled joins the sides. In a case, the run's frame is the case's cells, random draws draw for them, and a write to an enclosing frame stops the run.
A map inside the run, the run innermost, widens it (batch::Entry::Widen): a case with no definition, frame = the run's frame then the map's; the run's batches among the map's arguments lay over it (batch::laid_over); bindings made before the widening lay over it when read (Session::read_binding). A widened run that stops, or an Each with array results, falls back to per-cell calls inside the run. A dfn guard already splitting cells in another frame stops the run as recursion (Cause::Recursion); sides differing in layout or kind stop it as a call with no rule.
Each stop carries its cause (transform::Cause), setting how long a dfn operand remembers it (Cause::kept): effect, write, observing read, recursion: always; no covering route: while later calls repeat argument kinds and cell shapes; memory guard: while they also repeat the frame shape; caught error: never (it depends on cell values). Definition::remember holds ALWAYS or an argument signature (batch::signature), each with the cause (batch::holding). A covered later try falls back without calling the operand, with an "an earlier stop" event naming the cause. A stop's fallbacks event names its cause (Cause::reason) and, for an uncovered primitive call, the glyph (Stop::refusing).
Code with no batch rule reading a batch's items gets item carriers (item i of every cell) and records an itemwise row, route "batched items". explain::itemwise: the one helper for item-at-a-time routes; the innermost call claims the counts (Function::reported), as for forced. A kernel then reading a number meets the stop Carrier::computed gives a batch → a missed reader costs speed or a fallback, never a wrong result.
A batched run can also start inside a ∂ call: its level is newer than the ∂ tag → batch rules take each call first; their whole-array calls reach the derivative rules. A batch's whole can carry a dual (kind checks read Value::template_dtype). Stacks of matrices whose wholes carry a dual take derivative rules via Pairing::Matrices (eval::whole_cells): stacked kernels read compact storage. ∂ inside the run differentiates by batches of atoms or arrays: derivative::numbers counts a cell's numbers from the batch's template; all cells share each direction's seed; the Jacobian joins into one batch (batch::joined). A planless move on a dual whose primal is a batch runs on primal and tangent apart (derivative::apart). Adjoints a transpose adds into a batch add as one sum on its whole (batch::scatter_add). An attached rule runs on the whole batch at its argument's ∂ tag (batch::call); batch::split_whole, batch::dual_of_wholes keep the newest level outermost.
Random: a map's draws read streams that the outermost map's count, each map's place, axis by axis, and the draw's place among its cell's draws pick (distribution::map, distribution::cell, rustymath::random::Key::folded); a batched run's cells take their places from its frame. Batched and per-cell runs draw the same numbers, under a splitting guard too; a batched run in chunks that draws falls back to its cells. A batched try outside any map that falls back restores each generator's count (distribution::settle).
The result must be a batch at the run's level, a strand of batches whose kinds join (joined), or a carrier-free atom or compact array as every cell's result; else fallback. A pervasive call stops where its result kind differs from the one kind a route gives every item (pervasive::expected) → each cell keeps its per-cell kind.
reference::check runs each reference case with batch off, on, and on under fusion's small sizes (settings::Sizes::SMALL), comparing storage kinds with the first run's.

Folds: rustymath owns numeric folds, scans, adjacent pairs, grouped reductions and integer bounds; basedpl selects kernels, tolerance, seeds, storage widths and exact fallbacks. Rustymath's facts! table gives, via pervasive::folds and truth_fold, the reduction folding each primitive, the object by whose rule each takes characters, the comparison objects with_comparison! makes, and the product kernel each pair f.g names (pervasive::known_product). Folds by algebraic float operations can regroup. Contiguous subtraction and division folds use Alternating; quotients fall back to item-by-item folding if their products leave the normal range. Boolean scans compose truth-table maps; comparison scans compute both possible totals in bulk and use reduce::chosen. Seeds, axis lanes and scalar fallback keep the language's accumulation direction. Generic reductions stay right-associated, keeping operand call order and effects; exception: monadic f.g memoizes each minor's expansion, and a cached minor repeats no operand calls or effects. Determinant and permanent fast paths use the appropriate rustymath backend.

Threads: the automatic Rayon scheduler serves rustymath's elementwise many and pairs, broadcasts through Kernel::map, whose blocks read each argument through agreement::Read, folds via reduce::along, block stages and matrix products. Scans don't split. Decompositions split in their blocked updates' products, their Hermitian and bidiagonal reductions' passes, and their divide-and-conquer merges.

rank sends dyadic +.× and ⌹, monadic ⌹ and -.×, and •decompose over the 2-cells of an unkeyed compact stack to primitive::stacked and stacked_decompositions, each cell through the rustymath function it uses alone. A pervasive operand, or Rank nested down to one, runs as one call: each argument gets length-1 axes after each level's frame → leading-axis agreement pairs frames with frames, cells with cells. An argument with a named frame axis goes through a batched map instead, whose frame carries no names → names pair only between cells (batch::tried). Rustymath's parallel::stack splits whole cells across threads in rounds, with its interrupt check between rounds.
Context::check reads the clock only on every 64th call → a caller whose checks run far apart calls check_now.
The scheduler adapts to call cost: RAYON_NUM_THREADS by default; one process-wide override via parallel::set_threads (0 default, 1 serial), which •prefs ["threads":n] writes directly. The timing harness sets one thread after each settings reset. Nested Rayon work doesn't split again; wasm is serial.
A block stage splits in pieces of whole blocks: elementwise via parallel::extended, folded via reduce::fold_pieces. On one thread an elementwise stage appends straight to its result (copying each block into a piece's slots cost up to 1.2 copies there).
Small converted chunks limit the scheduler's work: don't infer multicore execution from a large array alone.

Products: a pair with a rustymath product kernel (+.×, ⌈.+, ∨.∧, +.=) runs via linalg::Product::run, which Python's inner also takes (pervasive::known_product). Other native pairs: its bounded inner_with runner and reduce::right_fold, carrying the right tail across blocks; basedpl supplies primitive kernels, tolerance and the exact-widening fallback. An integer product not fitting 64 bits gives way to the exact path.

pervasive::inner takes each operand's Orientation; float and complex products read a transposed operand in place, as reverse mode's product_transpose needs.
A float +.× whose operand is a pending copy, or a view of one, hands rustymath's Inner::inner_mapped the copy's source and map (fusion::source, pervasive::mapped_product). A +.× whose operands are both pending conversions of compact u8, i16 or f16 arrays to a wider type reads those arrays; rustymath's kernel converts each item as it packs it (fusion::widened, pervasive::widened_product). A dfn program leaves such a conversion pending for its product (Unit::Pending).
Rustymath gives gemm an operand its map reads at strides (a transpose) where it lies (Mapped::matrix), and reads any other copy a block of rows at a time, except in a one-column product whose copy's rows are adjacent in storage (a vector, a single-channel image), where it adds each column of the copy where it lies (linalg::by_columns). Neither builds the copy. A copy a name holds is read again for each product, not cached.
Reverse mode reads a windows copy's cotangent back through rustymath's Transpose (linear::read_back): one read for each position in a window, a correlation, with fills between frames that move by more than 1 (structure::correlated), as a transposed strided convolution reads its cotangent. No scatter-add or zero-padded copy of the cotangent is made.
Structural gather and compress and interval index also reuse rustymath. Measure changes to generic or backend code in both projects: the callers' codegen and layout decide costs a kernel measured alone misses.

Evaluation and frontend boundaries

eval_with/call_with install EvalOptions. Echo (default true) off suppresses implicit display, not explicit output or display commands. Output: ordered Output { kind, data } MIME bundles, to an optional sink as produced, else retained by the evaluation. Values, functions, operators, outputs and errors stay separate results.

Execution::begin puts each evaluation's interrupt handle, deadline and poll in execution::STOPS, a static every thread reads; its Running guard restores those of an enclosing evaluation (e.g. the one calling ⍎). execution::stopped reads them; the first evaluation installs it as rustymath's interrupt check (rustymath::interrupt) → rustymath's long computations (exact matrix functions, primes, stacks of matrices) stop for basedpl's interrupts and deadlines, on its worker threads too. A thread without a turn (e.g. Python's in) also stops at the running evaluation's interrupt.

Evaluation code reads the process's one session via session() and takes a &Span where it reports errors. Boundary checks and long loops call execution::check: counts down a static before reading clocks and flags or calling the poll hook; takes no span or session; turnless code (Python's in) calls it too. Single BigInt operations and bounded native kernels aren't preempted. Session state sits in Cells and RefCells whose methods return owned values → no caller holds a borrow while a dfn evaluates (meta/session.md). An optional Input supplies ⎕ and •nget "-"; without one they raise IO. Python cancels cooperatively. The process worker can kill an unresponsive process after a grace period, losing its state; requests are never replayed.

Session::call resolves and calls a function expression on existing arrays, with no temporary argument bindings; retained functions use call_function_with. Spans: Function::call gives each spanless error from a function's code the call's span; evaluator errors take the spans of the nodes they report; a statement gives any still-spanless error its own; errors outside an evaluation (e.g. Python indexing) have none. Diagnostics display with Unicode width. Python translates spans to character offsets; process JSON keeps bytes. Native function and operator handles cross the in-process Python boundary, not process JSON.

Python

python/basedpl/__init__.py owns the shared bpl workspace and the Array, result and error wrappers. An Array with a function item is a FunctionArray, chosen by Array._classify in __init__ and after __setitem__ → callable() is false for every other array. fn aliases bpl.fn; builtins and magics share the workspace; registering the magics gives it the REPL's display defaults (Session::make_interactive). Plain calls print explicit output and return Array, Function or Operator values; capturing calls return Result; capture mode is positional-only (keywords bind BPL names). _Session.eval, .call: the PyO3 entry points; request: their internal Rust helper.

functions.py builds native nodes and lazy exports from Rust metadata. Builtins win name collisions with workspace names; operation names choose valence and currying. __all__ omits Python-builtin collisions; explicit imports still work. Keyword calls use a keyed left argument. One positional argument stays the right argument unchanged; zero or several form a vector. Operator properties and _Pending assemble native nodes, never generated BPL source. printing.py reads node parts without evaluating or resolving names.

.fn() keeps a late-bound expression: calls resolve affected nodes against the calling session, sharing unaffected graph nodes → primitive fast paths, inverses and reduction identities survive. The binder still resolves ordinary names during execution. Cyclic resolution: LIMIT error.

Arrays keep native values; .py and .np copy; NumPy is optional and lazy.

The native _Array buffer export lends compact items read-only. An unbuilt structural copy with an affine read (Value::affine) lends its source's items with strides → windows reach NumPy unbuilt. own() moves compact items into a writable NumPy array where nothing else holds them, Booleans as u8, characters as i32 (NumPy can write any bytes).
Numeric buffers import with one copy via rustymath_buffer::read_array; Value::imported_buffer keeps supported item widths; the shared reader normalizes signed bytes, wider unsigned integers and foreign byte order. Sibling snapshots carry keys and names. A NumPy uint64 array with a value above int64's range imports item by item as exact integers. Exact integers and fractions cross as native integers, not decimal strings. String and object arrays convert per item.
Python data converts in Rust: the session's array (import, py/src/lib.rs) walks nested lists via rustymath_buffer::walk, rustymath's Python package's walker, with a collector that nests a level whose items differ in shape. Lists follow the shared import rule; dicts read as literals do. rustymath_buffer::labels: the one rule for dict keys and frame column labels, in both Python packages. A frame reads via rustymath_buffer::frame as a keyed vector of columns; .df turns it back. None as an item reads as NaN; a bare None doesn't convert → an operator given one returns NotImplemented.
syntax.py wraps the native syntax tree with character spans and span-editing helpers.

Array(existing) makes an independent native handle. Assignment and .keys/.names setters replace the owner's handle, leaving prior views and captured functions unchanged. Both Python packages read written selectors as structure::IndexPart tokens via rustymath/buffer/src/selection.rs and plan them with structure::plan_parts, ⌷'s structural planner; structure::addressed_parts routes source groups and expands Rest against the applicable frame. On a BPL array, Python's own kinds (selection::native: slices, keys, integers, lists of keys) decode in the buffer crate; Boolean masks select where true; other selectors become BPL values decoding as ⌷ decodes them (Value::planned). Pick paths select alone, as ⌷ reads them, keeping the written parts until the ending cell's geometry is known. _Array.select, _Array.assign: carrier-aware structural selection and writeback. Python assignment uses leading-axis broadcasting; language writeback keeps its own agreement.

Each _Session request waits for its evaluator mutex without the GIL, then evaluates on the caller's thread with the GIL released. A separate interrupt handle allows interruption from another thread; signal polling briefly takes the GIL. Python input and optional streaming-output callbacks cross at explicit boundary adapters. No Python numeric objects enter core values; arrays can drop on any thread without an unsafe Send implementation.

Process, CLI and notebooks

protocol.rs owns JSON request, value, error and output encoding. Requests: objects with fields such as code, call, args, bindings. Exact integers: arbitrary-precision JSON numbers; fractions and complex values: tagged; output bytes: base64. The worker omits eligible keyed entries holding functions or operators and returns value: null for other non-exportable values, keeping display output. JSON data serialization is a separate contract and can reject such values.

cli/src/worker.rs processes JSON lines in order; a separate reader handles interrupt controls, never evaluating BPL. IDs optional; bad requests don't end the session. python/basedpl/worker.py owns process lifetime, deadlines and hard-kill fallback. Protocol serialization walks native values directly, with no intermediate JSON tree per reply.

CLI -e, files and REPL stream outputs to stdout; a failed write interrupts execution; a closed pipe exits successfully. -e, files and interactive input provide StandardInput; program text read from stdin doesn't. The REPL parses for completeness without evaluating twice. Rustyline owns terminal editing, history and modes; only typed backtick sequences auto-expand; paste, history and navigation cancel expansion. Shared symbol metadata supplies completion and highlighting. Keep pseudo-terminal tests for Ctrl-C, Ctrl-D and paste.

Jupyter adapter (cli/src/kernel.rs): kernmini and a ThreadWorker; keeps transport and interrupts responsive; explicit output → streams, implicit → results; notebook input via input_request. Wheel and local install share the kernelspec. inspection.rs embeds the glyph and system pages for help, under the REPL's and Python's names. Change names and shortcuts in symbols.rs and shared layout data, not per frontend.

Browser

cargo wasm: an incremental release-profile npm package with wasm-bindgen glue, language and input assets, grammar and canvas support. --profile dist selects O3/cross-crate ThinLTO for authoritative x86 budgets; --out DIR leaves the docs' local playground assets alone; --strip removes function-name metadata for distribution. Keep wasm-bindgen glue and crate versions aligned through the workspace lock. Avoid fat LTO here: its rebuild cost and larger stack frames conflict with browser development and recursion limits.

Stack frames: Liftoff, V8's first compiler, gives each wasm local its own stack slot, used or not → a function that recursion passes through keeps its general path out of line, as Each, Rank and the outer product do (each_items), and ErrorKind, held by every frame's Result, stays 16 bytes in wasm32 (transform::Stop). In wasm, Binder::reduce and Bindings::enter stay out of line, keeping their locals out of the frames held at every recursion level. To find a frame that grew: read the function names in the stack of the overflow's RangeError, then compare those functions' wasm locals between builds; TurboFan frame size per function, in "stack slots": node --no-liftoff --print-wasm-code-function-index=N. wasm-objdump -h DIR/basedpl_wasm_bg.wasm: section sizes, name included. twiggy diff matches functions by full name, including crate hashes that change between builds → compare two builds' twiggy top -n 1000000 -f json listings with hashes removed.

Recursion limits: CI's Test wasm job takes Node's version from .node-version: TurboFan frame sizes vary between V8 versions enough to decide whether a recursion test overflows (one build's each shape: 0.68 MB for 171 levels under Node 24, 0.62 MB under Node 26). tests/wasm.mjs checks depth with Node's 192 KB reserve plus 80% of a Chrome worker's roughly 470 KB, 0.5575 MB in all → frames growing into the remaining 20% fail it. CI's x86-64 V8 takes more stack per level than Apple silicon's: MAX_CALL_DEPTH (src/eval.rs), the browser's depth limit, allows levels up to 1.48 times their Apple-silicon size. basedpl.bpltests.wasm_stack(DIR) measures each recursion shape's stack per level and below its first level, the figures the limit is set from; each trial runs in a fresh node process whose worker compiles the module, as the test's does (code compiled once and shared across workers gets different frames).

Browser Session.run streams MIME events and returns a protocol error or null. eval transfers values via js.rs, discards output, can't export BPL functions or operators; bind imports page values and functions. Typed arrays transfer without another page-worker copy. JavaScript functions live in a thread-local registry and core function nodes hold slots → values stay Send/Sync. host_web.rs: the browser's clock, synchronous I/O and fetch, RegExp behaviour. File and URL reads resolve against the configured base. Unsupported native filesystem and locale operations stay unavailable; browser •nput writes only stdout; •js is browser-only.

The playground and gallery load released assets from jsDelivr, or local nbs/playground/ assets on localhost. Shared bpl.js owns editors, worker startup and output. The authored ES modules and font in python/basedpl/browser/ need no evaluator: browser/bpl-editor.js exports grammar, configuration, editor options, font installation and explicit install(). Importing data never mounts the bar. Solveit, the playground and the gallery supply a normal-flow #bpl-toolbar slot; elsewhere the same component overlays the page without padding or height adjustment. One persisted collapse button keeps keyboard input active. install() reuses a connected controller; its dispose() closes it. Python magics call Solveit's window.loadBplInput() when present, else import the same public module from jsDelivr. Restarting a trapped or stopped worker loses its state. worker.js, page.js and gallery.js own their protocols and UI; nbs/driver.js documents canvas drawing data. Gallery demos: .bpl sources with shared preferences, checked natively by language_examples. Browser recursion tests cover both V8 compilers and fail with LIMIT before stack exhaustion.

System functions and rendering

system.rs owns the system table: one SystemFunction per system function: case-insensitive name, call, valence, extension, result prototype, inverse. Its extension (primitive::Extension) is the field each primitive form holds too. A system function takes its right argument whole, or takes items (primitive::Item), mapping over a right argument holding several, with the left argument whole for each; one taking text also maps over each row of a character matrix, as ⍎ does.

A function that acts asks the running levels where it acts (Context::acts), as output and input do: reading or writing files, network, clock or terminal, or making something new each call (•index, •rand). Others ask nothing; a batched run calls them once per cell.
The parser resolves a known system name to its table entry (system::Entry) via Builtin::find, which also finds glyph primitives; primitive::Builtin holds either → a system name is a NodeKind::Function node, compiles to a constant and resolves to the function a process-wide table holds per builtin (Builtin::function), as a glyph does. An unknown system name stays a NodeKind::System node: VALUE error when evaluated.
Code identifies a table function by its SystemFunction's Entry (Entry::PREFS), never by name. A factory-made function (•element "div") has no entry, though it shares its factory's name.
Native records use ordinary function nodes and calls. Empty native item mapping returns its declared prototype without calling the function. data::Options: the shared keyed-option decoder; formats invent no option parsers of their own.

optimize.rs: ⇄ on rustymath::optimize; the left argument holds options. A method that can use a derivative (none named, L-BFGS, Levenberg–Marquardt, CG, MINRES) → one linearisation at the start (derivative::linearized) finds whether ∂ differentiates the function and gives its value there. Goal evaluates the function at points in the start's layout for minimize; under L-BFGS each value comes from a linearisation that gradient transposes at the same point (linearized_at) → one function run per value and gradient. With "solve":$t, Residual is the Mapping solve takes, or solve_affine with "linear":$t too. Where ∂ differentiates the function, Residual gives JVPs and vector–Jacobian products from one record per point, or the start's record for an affine function. A later point where ∂ gives UNSUPPORTED → Residual::jvp declines; rustymath takes forward differences for that Jacobian and later ones. A square linear system takes no transpose products → GMRES on A, not the normal equations. Option precondition's function gives the preconditioner.

CSV columns, JSON arrays and Python lists share data::imported: a non-fill finite float, or any nonfinite float, promotes to float when every number converts losslessly; finite fills alone don't; otherwise items keep their kinds. JSON objects and Python dicts read as literals do (keyed::vector, Value::from_parts). Binary data: exact bytes. Writes validate before opening and create exclusively unless overwrite is explicit. Exports: •csv⁻¹, •json⁻¹, •xml⁻¹.

regex.rs shares host regex state between match and replace native functions; hosts keep their own regex syntax. xml.rs: keyed element trees; XML escaping and validation kept apart from HTML raw script/style and void-element rules. SVG has its own renderer. image.rs owns pictures, their options and animation timing, as plain pixel buffers; its codec module decodes and encodes via ImageIO on Apple targets (image/imageio.rs), the image and png crates elsewhere (image/codecs.rs). Native and browser renderers use the MIME and byte formats their frontends support. plot.rs stores specs and draws SVG on demand; the renderer owns defaults and validation. Rendering options are documented in their user pages and implementation comments.

display.rs owns plain, boxed and function display, elision and MIME rendering. Display settings affect implicit presentation, not returned values or ⍕; explicit output prints all items. Display and source text read items, not storage: Value::shown_dtype (array/text.rs) gives the dtype compact storage of the items would take (storage::items_dtype's items rule), read by the shared ₓ or width marker, plain display's choice between brackets and a strand, the bracketed grid plain display and ⍕ share (display::table, display::bracketed), a keyed table's corner and the boxed marker. •mime calls a renderer with echo off and cancellation active. Failed implicit rendering falls back to text; interrupts and timeouts propagate. canvas.rs serializes drawing data and typed bytes; canvas.js and nbs/driver.js own browser playback. Keep formatting, serialization and rich output separate contracts.

Tests and reference data

tests/core.rs: public storage, layout and API behaviour, ownership, parser diagnostics and recovery. equiv_in! for session workflows, fails_in for errors; each expected expression runs in a fresh session after its code, under the settings the code leaves. tests/core.rs never changes settings: they're one process-wide store → checks that change them live in tests/settings.rs, run in turn in one test, each from the defaults. Both binaries share tests/common/mod.rs. Keep constructors for representation, prototype and buffer checks. Self-contained language cases: tests/reference/core.bpl. cli/tests/cli.rs and Python's pseudo-terminal tests exercise the installed process and editor boundary.

tests/jit.rs::compiled_stages_match_interpreted compares every case at four original sizes. Set BASEDPL_JIT_SIZE_INDEX to 0, 1, 2 or 3 to check one size before allocating its fixtures: 16×DEFAULT.block plus 0, 1, 7 or 37 items. Invalid indices fail. With the variable unset, all four sizes run. Each completed size reports its item count and dynamic/static case counts. Use the selector to partition the existing checks within a runner's execution budget; it does not resize inputs or skip expressions at the selected size.

reference.rs serves Rust tests and worker requests.

Every reference case gets a fresh session; documentation examples share one per BPL code block, starting from default settings. Each page starts with no kept plans or compiled kernels, as a new process does (program::forget_plans, jit::forget_kernels) → an example showing block counts or compiling gives the same output alone (check_page) or after other pages (language_examples).
A reference file's leading •prefs line sets all its cases' settings. Optional ⍝⍝ setup: shared definitions, initialized separately for each run and its expectation. A section's inline •prefs suffix runs before setup.
Each case runs under the normal compile policy, with every available block stage compiled, with compilation disabled, and as the body of a dfn called twice. The second dfn call checks replay's value, storage kind, error and output. Cases that cannot be dfn bodies omit that run. compiled skips the compilation-disabled run. Shared setups that reference test_compiled receive that fixture, true in the forced-compiled run. Other reference sessions do not gain the name. Reporting tests use it to require actual "compiled" execution without changing the scheduler's mode. Capture each workload separately, including a warmed read that reuses code. compiles reports code generation, not execution.
Every run checks each program before it runs (program::verify): each step reads values before its own; each read lays out as many items as its node holds; each node's description is what the shared planners give its step on its operands. A breach panics with the program's listing.
Run the complete reference cohort with: from basedpl.bpltests import check_file; report = check_file("tests/reference"). The report contains every parsed case's applicable modes, actual failures, incomplete checks and coverage totals. Healthy single-native-thread workers are reused; fixture thread preferences override that default. A watchdog replaces its worker and continues unattempted independent modes and queued cases, never retries the stopped mode. Fresh continuations are labelled; replay needs a successful normal warmup, and cross-mode storage comparison stays blocked without all standard outcomes. report["complete"] means no expected check is missing, not that every result passed.
Each reference evaluation and its final value read have a two-second deadline in ordinary correctness runs. The managed runner's outer watchdog remains separate. Keep known over-deadline cases at full size in section-specific tests/skipped.txt entries linked to their open work; never disable deadline checks globally.
Each expectation evaluates after its code, under the settings the code leaves. A float matches within 1e-13, or 8 ulps of its own width where greater, unless the header sets a tolerance (reference::same_atom). .bpl files define active coverage. Captured foreign expectations have no exactness; local expectations compare exactness and structural values independently, and fix where storage is mixed: the result must be mixed exactly where the expectation is, at each nesting level (kind_difference), unless the header carries mixed.
Pending gaps go in inventory JSONL, not passing error tests. Format and review/activation workflow: tests/reference/README.md. python/basedpl/reference.py, bpltests.py and scripts/reference.py own import, capture, inventory and worker checks. lib/*.bpl: readable dfn examples; modules import other library files with private relative names.

tests/scaling.rs: time and allocation growth, result-size allocation budgets. tests/speed.rs reads native and browser budgets from speed/*.bpl via timing.rs; the worker and Python measurement tools reuse it. Both, and tests/reference.rs's children, register rustymath's memory guard and turn on its checks (memory::set_checks) in one-case-at-a-time processes: a reference case leaving more bytes held in large allocations than it found, on each of two runs, fails as a leak, as does a speed case whose timed runs leave more than its first run. A free or reallocation passing more bytes than its block holds aborts with a backtrace. Python tests cover conversion, cancellation and frontend behaviour, not a second Rust semantic suite. Combine related checks into readable tests.

Documentation and generated assets

nbs/index.ipynb generates the homepage and README. Reference pages (nbs/glyphs/, nbs/system/) describe forms, options and errors; concept pages explain them. sidebar.yml owns navigation. Edit source notebooks and pages, not generated outputs. magics.ipynb and dyalog.ipynb export Python modules; other Python modules are hand-written. Cargo owns versions. nbs/llms.txt: the docs' index for LLMs: per page, how BPL differs from Dyalog, an example, the defining page. src/inspection.rs builds it in (]help llms after cargo develop); the project's CLAUDE.md includes it in every session's instructions.

bpl-nb
bpl-nb nbs/index.ipynb --save
nbdev-readme
nbdev-test --save
nbdev-test nbs/dyalog.ipynb --flags dyalog
nbdev-preview

Quarto renders saved outputs. bpl-nb --save updates notebook outputs without changing execution counts: use it for BPL examples and figures, nbdev-test for Python notebook code. bpl-nb --compare prints a diff for each cell whose shown text differs from its stored outputs, and exits with status 1 if any do. Rust language_examples checks BPL page and gallery examples; pytest checks Python page examples; BASEDPL_PAGE focuses page checks. Don't run a full documentation build for routine output updates. Before a release, bpl-tests prerelease runs the normal development build and nbdev's preparation ("Release").

cargo develop and ws-sync run basedpl.editors in a fresh Python process after installing the extension, before copying wheel data: it generates grammars, highlighters, editor templates, keyboard bundles and glyph metadata, Rust-derived browser/language.js included. JavaScript UI changes are authored modules Vite reads directly: no generator or npm refresh. cargo wasm copies the whole browser directory into the npm package and local playground. browser/layout.json owns composed input; captured keyboards.json allows generation without macOS; basedpl.keyboards.capture() refreshes native data when needed; keyboards.write() generates regional macOS bundles and maps. Regenerate keyboard sources through the normal build, not one-off transformations. basedpl-install copies packaged fonts and macOS bundles from python/basedpl/assets to the user's directories. Target-specific work: editors/jetbrains/README.md, editors/windows/README.md. Rebuild the common icon only when its design changes.

nbs/bpl.xml, the APL and BQN grammars and vim rules share generated glyph classes. hpl.xml includes BPL and adds Hanzi aliases from the same symbol table. Fences: hpl for Hanzi highlighting, bpl for Hanzi names. Hanzi notebooks: kernelspec.language hpl, kernelspec.name bpl; these labels don't enable runtime Hanzi aliases. TextMate exports feed Monaco and Shiki. fonts.css owns BPL and APL font choices; rerun affected magics after changing them. tests/highlighting.mjs, input.mjs, keylayout and Windows tests and glyph-page checks cover their generated interfaces. All artifact generation lives in repository functions and scripts, not ad hoc kernel transformations.

Release

Before approval: verify successful tests for the release commit and the relevant installed artifacts. Isolated wheel installs only for packaging-sensitive work or release preparation, with a small relevant subset of existing tests. ship-rs-build --profile dist builds the distributed configuration; bare ship-rs-build uses release, for local checks.

Before each release, run bpl-tests prerelease from basedpl's root, in a shell, not a kernel, while no one else edits either repository. Development builds, native timings and target builds use the release profile unless --profile NAME selects another. The checked test profile stays separate. It runs both repositories' release checks and saves:

Builds, tests and saves run in parallel once their dependencies pass: development installation, managed native correctness tests, pytest -q, nbdev-prepare, and cargo test --profile checked in its own target directory. basedpl's development installation waits for rustymath's. It includes editor generation; basedpl's Node tests, notebooks and target_builds wait for it, and browser tests follow target_builds.
Timings, one at a time, nothing else running: native speed tests in each repository with <NAME>_COSTS set, rustymath's Python speed test, thread_scaling, and browser speed tests in selector batches. Each browser batch has the existing 30-second execution budget and verifies every selected (file, section, code) identity, including retries. With --keep-going, a failed batch does not hide later batches.
nbdev-clean in both repositories, in a finally block after failures, launch errors or cancellation.

The first failure ends the run unless --keep-going. Summary: each step's status, time and log. --dry-run lists the steps in order without running them. Cargo builds have no outer deadline; managed tests retain their execution watchdogs. The preflight stops before starting if the pinned Node version is unavailable or Node cannot resolve shiki. It notes when basedpl needs an unreleased rustymath version (rustymath then ships first). It never commits, pushes or tags.

cargo test --profile checked: the test profile with debug assertions on, adding session()'s turn check and the standard library's checks in unsafe code. They cost 12-55% per call, too much for the test profile the budgets measure; the speed and scaling tests mark themselves ignored under debug assertions → that run checks correctness only.

Confirm the clean tree and [workspace.package].version, then run ship-release without flags. Python gets that version via dynamic = ["version"]. Fastship pushes and tags the current version, leaves publication to the Release workflow, and bumps Cargo without rebuilding. No changelog step. Never commit, push, tag or publish without approval.

Navigation Menu

Repository navigation

DEV.md

DEV.md

Development

Required first reading

Development reference (on demand)

Machine code

Workspace, builds and CI

Codegen and rebuilds

Measure a representative rebuild

Locate the critical path

Check both compiler caches

Find generic expansion before optimizing code

Escalate telemetry and reduce a repro

Code map

Parser, evaluator and scopes

Values, layouts and assignment

Numbers and backend ownership

Kernels, fusion and threading

Evaluation and frontend boundaries

Python

Process, CLI and notebooks

Browser

System functions and rendering

Tests and reference data

Documentation and generated assets

Release

FilesExpand file tree

DEV.md

Latest commit

History

DEV.md

File metadata and controls

Development

Required first reading

Development reference (on demand)

Machine code

Workspace, builds and CI

Codegen and rebuilds

Measure a representative rebuild

Locate the critical path

Check both compiler caches

Find generic expansion before optimizing code

Escalate telemetry and reduce a repro

Code map

Parser, evaluator and scopes

Values, layouts and assignment

Numbers and backend ownership

Kernels, fusion and threading

Evaluation and frontend boundaries

Python

Process, CLI and notebooks

Browser

System functions and rendering

Tests and reference data

Documentation and generated assets

Release