Mergeblock

profile.sh

@@ -1,2 +1,2 @@
 #!/bin/bash
-PERFFLAGS="-F 200 -g --call-graph dwarf" cargo flamegraph --profile profiling --features rex-jit,lightning --bin iris
+PERFFLAGS="-F 200 -g --call-graph dwarf" cargo flamegraph --profile profiling --features rex-jit,lightning,j2wp,tcache --bin iris

rules/jitv2/block-fragmentation-blocks-cse.md

@@ -0,0 +1,117 @@
+# Block fragmentation, not callouts, is what starves `opt_level=speed`
+
+Measured 2026-09-01 on 300 real IRIX corpus pages (`jitv2_corpus/`) via
+`zz_corpus_sizes` with `IRIS_JIT_DISASM=1` and `IRIS_OPT_SPEED=1`.
+
+## First: measure without `developer`
+
+`CODEGEN_OPT_LEVEL_SPEED` (codegen.rs) defaults to
+`!cfg!(feature = "developer")` — production and `lightning` get
+`opt_level=speed`; `developer` gets `none`. Two consequences for anyone
+measuring emitted code:
+
+1. A `developer` build measures **unoptimized** codegen unless you set
+   `IRIS_OPT_SPEED=1`.
+2. Even with that set, `emit_dev_trace_bp` adds a `call_indirect` **per
+   instruction**. In the first run of this investigation, **11,214 of 15,723
+   callouts (71%) were the dev-trace hook** — and each one is an opaque
+   clobber of the whole core struct plus a `brif`, so it wrecks both the
+   callout statistics and the block-size distribution.
+
+`zz_corpus_sizes` used to require `developer` (it calls `last_code_size()`,
+which is developer-gated) and so silently measured the wrong thing. It now
+builds without it, reporting size 0 in that case.
+
+**Rule: any claim about emitted-code shape must come from a non-`developer`
+build.** The first pass of this investigation produced a completely wrong
+field-traffic ranking (it made `pc`/`in_delay_slot` look like a 5:1 majority
+of stores; the real figure is 1.59:1) purely from this contamination.
+
+## The actual numbers (clean build, `speed`, 300 pages)
+
+```
+regions=300  total_asm_instrs=495353  bytes=2146233
+  mean region 7154 B, median 3200 B
+  loads 65908 (13%)  stores 22639 (4%)  calls 4509 (0.9%)
+  machine blocks=89906  mean 5.5 instrs  median 3.0
+```
+
+Field-level traffic (offsets: `hot.interrupts`=0x0, `hot.cycles`=0x8,
+`pc`=0x50, `in_delay_slot`=0x58, `gpr[]`=0x68..0x164, `fpr[]`=0x178):
+
+| field | loads | stores |
+|---|---|---|
+| `gpr[*]` | 12,027 | 6,790 |
+| `hot.interrupts` | 8,720 | — |
+| `pc` | 3,959 | 5,884 |
+| `in_delay_slot` | — | 4,900 |
+| `hot.cycles` | — | 2,509 |
+
+## Finding 1: Cranelift's CSE works *within* a block, and only there
+
+Zero redundant same-address loads inside any machine block. That zero is real,
+not broken instrumentation — ignoring block boundaries finds **27,668**
+duplicate loads (11,522 in the clean build). Of the clean build's duplicates:
+
+- **58% separated by a block boundary only** — no call in between. Pure
+  structural loss: Cranelift would have eliminated these had the instructions
+  shared a block.
+- 24% separated by call + block.
+- 17% within the scan window with nothing between (mostly cross-region
+  artifacts of a flat-file scan).
+
+## Finding 2: callouts are *not* the main barrier
+
+Only 4,509 calls across 495,353 instructions (0.9%). The inline L1-D fast path
+(`emit_inline_mem_guard`, gated on `dc_geometry.supported`) is doing its job —
+most loads/stores never reach `emit_mem_read_callout`. Callout clobbering of
+the core struct is real but rare enough not to dominate.
+
+**Corollary for benchmarking**: `Codegen::dc_geometry` defaults to
+`unsupported()`, which skips the inline path entirely and makes every access
+call out. Any harness measuring emitted code must stamp real geometry (as
+`zz_corpus_sizes` does) or it measures callout-only code and is blind to the
+whole inline path.
+
+## Finding 3: the interrupt preamble is smaller than it looks, but fragments everything
+
+`emit_pending_interrupt_preamble` emits, per head instruction, an
+`atomic_load` of `core.hot.interrupts` + test + `brif` to a cold bail block.
+Exactly 8,720 sequences, matching the 8,720 `hot.interrupts` loads (the two
+counts cross-validate the detector). That is 26,160 instructions = **5.3%** of
+emitted code — not the main cost by volume.
+
+Its real cost is structural: it is a **seqcst** load (deliberately — see the
+comment at its definition, `speed` mode would otherwise be free to hoist it),
+which is a full barrier for alias analysis, and its `brif` splits the block at
+**every instruction boundary**. That is what holds machine blocks at a median
+of 3 instructions.
+
+## Ranking (by evidence, not intuition)
+
+1. **Block fragmentation** — 6,745 provably-recoverable redundant loads,
+   median block of 3 instructions. Merging straight-line runs into single
+   blocks is the real lever.
+2. **Hoisting the interrupt check to block granularity** — small by volume,
+   but it is the *enabler* for (1): merging pass-1 blocks without hoisting the
+   preamble buys little, since the preamble re-splits every instruction.
+   Must stay per-instruction under `jitv2_lockstep`.
+3. **`pc`/`in_delay_slot` traffic** — **done** (2026-09-02): the exception
+   ABI now passes EPC/BD as arguments, so the per-slot bracket is
+   lockstep/developer-only. Total emitted stores across this same 300-page
+   corpus went 24,222 -> 12,431. See [[inlined-slot-pc-bd-bracket]].
+4. **GPR load/store traffic** — smaller than expected, and partly fixed for
+   free by (1), since Cranelift already CSEs these within a block.
+
+## What was *wrong* about the initial hypothesis
+
+The intuition going in was "callouts clobber the core struct, so nothing can
+stay in host registers." That is true in principle and near-irrelevant in
+practice at 0.9% call density. The measurement inverted the ranking: the
+barrier is the block structure the JIT itself emits, not the callouts.
+
+Note also that `emit_read_gpr`/`emit_write_gpr` are plain `load`/`store`
+against `core_ptr` with `MemFlagsData::trusted()` (= `notrap + aligned`, **no
+alias region**). There is no register cache; promoting GPRs to host registers
+is entirely Cranelift's redundant-load-elimination, which is why block scope
+determines how much of it happens.

rules/jitv2/deferred-delay-slots-unified.md

@@ -0,0 +1,91 @@
+# A branch whose delay slot can't be inlined: one path, not two
+
+**2026-09-02.** Until this change, jitv2 had two unrelated answers to the same
+question, and one of them was a leftover from before the answer was known.
+
+## The question
+
+A branch/jump's delay slot is architecturally indivisible from it (§6.1.4):
+the slot always executes exactly once, so codegen inlines it into the
+branch's own compiled unit. Sometimes it can't:
+
+- the slot is on the **next physical page** (branch at offset 0xFFC), or
+- the slot is an **`Excluded`** instruction (COP0/MTC0, `cache`, `eret`,
+  `syscall`, `break`, LL/SC, BC1, CP2, any unimplemented opcode), which by
+  definition has no native emitter and must run through the interpreter in
+  *head* position, or
+- the slot is otherwise unvisited.
+
+## The two old answers
+
+| slot | branch | mechanism |
+|---|---|---|
+| off-page (0xFFC) | **compiled**, slot deferred | `is_0xffc_branch` skips `visit_slot`; `taken_exit: ForeignPageSlot`; codegen's `emit_foreign_page_slot_exit` arms the pending transfer |
+| excluded | **declined outright** | `visit_slot` returns `false`, branch never marked visited, falls out of the region |
+
+The second is what the first used to do, before the foreign-page case was
+worked out. The analyzer's own comment gave it away — *"a slot that can't
+complete disqualifies the outermost branch exactly like an excluded slot
+always did"* — describing inertia, not a reason.
+
+`is_inlinable`'s doc comment had *already* asserted the unification: all
+three ways a slot can fail to inline "collapse to the same analyzer-side fact
+and the same codegen-side consequence: deferred to the next dispatch." The
+analyzer just didn't act on it.
+
+## The unified rule
+
+**If the slot can't be inlined, compile the branch and hand the interpreter a
+pending transfer** — arm `core.delay_slot_target`, set `core.in_delay_slot`,
+land `core.pc` on the slot word, return `EXEC_COMPLETE`. The interpreter runs
+the slot and retires the transfer. It does not care *why* the slot was
+deferred.
+
+The slot's address is derivable identically in both cases:
+`emit_word_addr(ctx, word + 1)`. At word 1023 that is `vbase + 1024*4` =
+`vbase + 0x1000` — the next page's word 0 — because it is an `iadd`, so the
+carry into bit 12 just works. (An on-page excluded slot is the easier case:
+no carry at all.) There is no address asymmetry between the two; that was
+the one thing that made them *look* like different problems.
+
+## What changed
+
+- **analyzer `visit`**: a failed `visit_slot` now sets `deferred_slot`
+  instead of `return false`. The branch is visited with
+  `has_inline_slot = false` and both edges forced to
+  `StopReason::ForeignPageSlot` (whose doc comment now says it covers both
+  causes — the name is historical).
+- **`is_inlinable` (codegen)**: now also rejects `is_fallback` heads.
+  **Necessary, not cosmetic**: an `Excluded` word *can* be `visited` — as a
+  fallback head admitted by some other path — which would otherwise make it
+  look inlinable purely because the `visited` bit was set.
+- **codegen otherwise unchanged.** It was already keyed on `is_inlinable`
+  rather than on `word == 1023`, so the deferred case flows through the
+  existing foreign-slot emitters untouched. That is the payoff: one predicate,
+  one path.
+
+## Payoff: simplification, not speed
+
+Measured honestly, on the 300-page IRIX corpus: **22 branch-with-excluded-slot
+occurrences across 300 pages**, and emitted instruction count identical
+(480,780 before and after). The shape is rare, and where it occurs the region
+often ended nearby anyway.
+
+Do not expect a benchmark to move. The reason to have done it is that jitv2
+is hairy enough already, and this removes a special case that existed only
+because of the order things were figured out in. A delay slot at a page break
+is genuinely plausible in real code; a weird excluded instruction in one
+mostly is not.
+
+## Tests
+
+- `analyzer::tests::walk_excluded_delay_slot_defers_the_slot_and_still_compiles_the_branch`
+  (rewritten — it previously asserted the old decline-the-branch contract).
+- `equiv_test::tests::{branch_taken,branch_not_taken,jump}_with_excluded_delay_slot_defers_like_the_interpreter`
+  — execution-level, checking the JIT reproduces the interpreter's
+  `pc`/`in_delay_slot`/`delay_slot_target` exactly. Analyzer-level tests alone
+  would only prove the region compiles, not that it *runs* right.
+
+Related: [[inlined-slot-pc-bd-bracket]] — same area, same session; that change
+is what made `emit_foreign_page_annulled_not_taken_exit`'s inherited
+`in_delay_slot` an explicit parameter.

rules/jitv2/emit_absolute_pc_exit-in_delay_slot-followup.md

@@ -11,6 +11,21 @@ because something upstream already cleared the flag first:
   `emit_branch_taken_edge`/`emit_nested_branch_slot`) runs after
   `emit_slot_semantics`'s non-terminating tail, which unconditionally clears
   the flag and restores `saved_pc` before returning.
+  **(2026-09-02: this guarantee is GONE.)** The bracket is now
+  `#[cfg(any(feature = "jitv2_lockstep", feature = "developer"))]` — the
+  exception ABI passes `Cause.BD` and EPC as arguments, so nothing reads
+  those fields back for an inlined slot. See [[inlined-slot-pc-bd-bracket]].
+  Compiled code no longer *sets* `in_delay_slot` for an inlined slot either,
+  so these call sites remain correct — the flag is simply never true there to
+  begin with — but they are now correct **by luck of what runs before them**,
+  not by an upstream guarantee. Which is exactly what this note warned about:
+  the removal immediately broke
+  `emit_foreign_page_annulled_not_taken_exit`, which had been silently
+  inheriting `in_delay_slot = 1` from the bracket. That one now takes an
+  explicit `pending_outer_transfer` parameter (its two callers need opposite
+  values). The general fix below — move the clear *inside*
+  `emit_absolute_pc_exit` — is still not done, and is now more clearly worth
+  doing rather than less.
 - The annulling-Likely not-taken arm never sets the flag in the first place
   (the slot is skipped entirely, mirroring `handle_branch_likely_skip`).
 

rules/jitv2/inlined-slot-pc-bd-bracket.md

@@ -0,0 +1,122 @@
+# The inlined delay slot's `core.pc` / `in_delay_slot` bracket
+
+**History, in order — read all three parts before touching this.**
+
+1. It looked dead. It was not.
+2. It was made dead, deliberately, by changing the exception ABI.
+3. Two other things silently depended on it. Both are now explicit.
+
+## What it is
+
+`emit_slot_semantics` (src/jitv2/codegen.rs) wraps every **inlined** delay
+slot:
+
+```
+in_delay_slot = 1
+saved_pc = load core.pc
+core.pc  = <slot's own address>
+  ... the slot instruction's real semantics ...
+in_delay_slot = 0
+core.pc  = saved_pc
+```
+
+As of 2026-09-02 all six of those memory operations are
+`#[cfg(any(feature = "jitv2_lockstep", feature = "developer"))]`.
+
+## Part 1 — why it was NOT removable (2026-09-01, reverted)
+
+A first attempt cfg-gated the bracket on the argument that nothing in a
+compiled region reads either field back: an in-region branch edge is a plain
+`jump` (`emit_target_edge`), a region-leaving exit writes its own `core.pc`,
+and only bits 12..63 of `core.pc` matter for in-region addressing.
+
+All true, and all beside the point. **`deliver_exception` (mips_core.rs) read
+both fields straight out of memory:**
+
+```rust
+if core.in_delay_slot {
+    cause |= CAUSE_BD;
+    core.cp0_epc = core.pc.wrapping_sub(4);
+}
+```
+
+and `emit_exception_call_block_body` called `handle_exception` with only
+`(core_ptr, status)`. `ctx.bd` selected *which stage block ran*, not what the
+callee saw. EPC also needs the **exact word**, not the page, so the
+"low bits are dead" argument did not apply.
+
+Caught by six `equiv_test` delay-slot exception tests, `cp0_cause` differing
+by exactly bit 31. **`cpu-tests` passed the broken build (2101/61, identical
+to baseline) and IRIX booted fine** — the common case self-heals, because
+delivering with the stale pc resumes at the branch and simply re-executes it.
+It breaks only where a handler *inspects* rather than retries (reading
+`Cause.BD` to find the faulting instruction, or a non-restartable
+trap/overflow/breakpoint in a slot, which would loop).
+
+## Part 2 — how it was actually removed (2026-09-02)
+
+Not by deleting the stores, but by removing the reason they existed: **pass
+EPC and BD as arguments instead of through memory.**
+
+- `mips_core::deliver_exception_at(core, status, fault_pc, bd)` holds the
+  logic; `deliver_exception(core, status)` is now a two-line wrapper reading
+  the fields, so interpreter and `jitv2_verify` callers are untouched.
+- `MipsExecutor::handle_exception_at`, and a new
+  `MipsCore::handle_exception_at_fn` FFI hook `(ctx, status, fault_pc, bd)`.
+- Codegen splits the exit into two wrappers over **one** shared call block
+  (the two outer stage blocks are deleted):
+  - `emit_exception_exit_const` — `emit_word_addr(ctx.word)` + `iconst(ctx.bd)`.
+    Every ordinary in-region instruction, **including an inlined slot**.
+  - `emit_exception_exit_live` — two loads. Only for the entry word and
+    branch-fallback successor, which inherit state from outside the region.
+
+Measured over 300 real IRIX corpus pages (`zz_corpus_sizes`,
+`IRIS_JIT_DISASM=1`, `opt_level=speed`, no `developer`):
+
+| | before | after |
+|---|---|---|
+| `pc` stores | 5,884 | 1,243 |
+| `in_delay_slot` stores | 4,900 | 259 |
+| `pc` loads | 3,959 | 1,967 |
+| `gpr` stores | 6,790 | 6,790 |
+| **total stores** | **24,222** | **12,431** |
+
+Half of all emitted store traffic. (`gpr` unchanged is the correctness
+check — that traffic is architectural and must not move.)
+
+## Part 3 — the two hidden dependencies it was masking
+
+Both were silent inheritances of the bracket's unconditional writes, and both
+now set what they need explicitly:
+
+**`trust_live_pc_bd_on_exc` leaked into slots.** When a branch is itself an
+entry word (or branch-fallback successor) that flag is set on its `ctx`, and
+`emit_slot_semantics` inherited it — routing the *slot's* fault down
+`emit_exception_exit_live`, which loads a flag the bracket no longer writes.
+Fixed by clearing it alongside `ctx.bd = true`: a slot's fault state is
+always compile-time known.
+
+**`emit_foreign_page_annulled_not_taken_exit` inherited `in_delay_slot`** —
+and its two callers need **opposite** values. Head-level branch-likely at
+0xFFC: nothing pending, `false`. Nested: the *outer* branch's transfer is
+still live, `true`. Now a `pending_outer_transfer` parameter. This is exactly
+the fragility [[emit_absolute_pc_exit-in_delay_slot-followup]] flagged for the
+mirror-image case, hit from the other direction.
+
+## What still needs the bracket
+
+- **`jitv2_lockstep`** — the compare reads `core.pc`/`in_delay_slot` as the
+  slot's post-state (and `delay_slot_target` as the expected pc, since a slot
+  retires *from* `in_delay_slot = true`).
+- **`developer`** — `emit_dev_trace_bp` reports the slot's own pc for `dt`.
+
+## Testing rule
+
+**`equiv_test` is the only suite that covers delay-slot exception BD/EPC.**
+cpu-tests and a full IRIX boot both passed the broken build. Run
+`cargo test --release --features jitv2 --lib jitv2::` before believing any
+change to `emit_slot_semantics`, the exception path, or anything touching
+`core.pc`/`core.in_delay_slot`.
+
+Related: [[block-fragmentation-blocks-cse]] (where the measurement came from),
+[[deferred-delay-slots-unified]] (the sibling cleanup in the same area).