Skip to content

mips32: Reject MIPS64-only encodings instead of asserting - #91

Open
zardus wants to merge 1 commit into
masterfrom
feature/mips64-nodecode
Open

mips32: Reject MIPS64-only encodings instead of asserting#91
zardus wants to merge 1 commit into
masterfrom
feature/mips64-nodecode

Conversation

@zardus

@zardus zardus commented Aug 16, 2026

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

A MIPS64-only encoding is a Reserved Instruction on a 32-bit MIPS, but priv/guest_mips_toIR.c decodes 37 opcode slots' worth of them without consulting mode64 first. The block reported in pyvex issue 76 is six ordinary MIPS32 instructions followed by daddu $a0, $v0, $zero (0x0040202d) at 0x3c5f0:

$ pyvex.lift(8fc2002c ... 8f998324, 0x3c5d8, ARCH_MIPS32_BE, opt_level=0)
  size=0  instructions=0  jumpkind=Ijk_NoDecode  next=0x0003c5d8

IRSB {
   NEXT: PUT(pc) = 0x0003c5d8; Ijk_NoDecode
}

The four loads, the store and the addiu ahead of the daddu are lost with it, and next points at the start of the block rather than at the encoding that failed, so the caller is handed no bytes at all and no address to resume from.

Root cause

These cases build 64-bit IR and hand it to a 32-bit guest register. libVEX then longjmps out of the whole translation from one of three places, all reachable today:

ld $a0, -0x3098($v1)   guest_mips_toIR.c:1217 (putIReg): Assertion `typeOfIRExpr(irsb->tyenv, e) == ty' failed.
dsll32 $v1, $v1, 0     ERROR = Iex.Binop: arg tys don't match op tys
                       sanityCheckFail: exiting due to bad IR
dmtc1 $zero, $f0       guest_mips_toIR.c:12806 (disInstr_MIPS_WRK): Assertion `mode64' failed.

The unwind happens after disInstr_MIPS_WRK has already emitted IR for the preceding instructions, and past the code that would have kept it.

Fix

is_MIPS64_only_insn recognises the encodings that fail this way -- the SPECIAL doubleword shifts, multiplies, divides and adds; DMFC1/DMTC1; DCLZ/DCLO; the SPECIAL3 doubleword bitfield and byte-swap group; DADDI/DADDIU, LDL/LDR, SDL/SDR, LLD, LD and SCD -- and disInstr_MIPS_WRK consults it before the main opcode switch:

   if (!mode64 && is_MIPS64_only_insn(cins))
      goto decode_failure;

decode_failure is the path every other unsupported encoding already takes; it emits jmp_lit32(&dres, Ijk_NoDecode, guest_PC_curr_instr), so the block ends at the offending address with everything before it intact (the six decoded instructions elided here, complete in the comment below):

  size=24  instructions=7  jumpkind=Ijk_NoDecode
  instruction_addresses=(0x3c5d8, 0x3c5dc, 0x3c5e0, 0x3c5e4, 0x3c5e8, 0x3c5ec, 0x3c5f0)
   39 | ------ IMark(0x3c5f0, 0, 0) ------
   40 | PUT(pc) = 0x0003c5f0
   NEXT: PUT(pc) = t21; Ijk_NoDecode

Only encodings that fail today are listed, so the predicate cannot take away a translation that currently succeeds. LWU and SD are decoded in 32-bit mode by the cases below, correctly or not, and are left alone for that reason; so are the Cavium OCTEON encodings.

Testing

tests/test_mips32_reserved.py, which lands with the consumer, names one encoding per affected group -- fifteen in all -- and asserts the two-instruction prefix survives each. On the merge base it fails on its first entry with assert 0 == 8, the block being empty where the prefix should be; here all three tests in the file pass. The other two pin that MIPS64 still decodes the same fifteen encodings and that sd $ra, 0x5c8($sp) and lwu $a0, 0x10($v1) still decode on MIPS32.

Consumed by angr/pyvex#567, which carries the regression tests and the submodule bump. Validation: #91 (comment)

Merge order: this lands first, then angr/pyvex#567, whose vex submodule is pinned to this head (561795c). Merging the pyvex half on its own leaves pyvex master's submodule pointing at a commit that is on no vex branch. Nothing in this repository reads a sync: line — .github/workflows/build.yml is the only workflow and it is three build jobs — so the line below is a note to the reader.

sync: angr/pyvex#567

session: sharpen

A MIPS64-only instruction is a Reserved Instruction on a 32-bit MIPS, so
the decoder has to report a decode failure for it. This file instead
decodes ld, daddiu, the doubleword shifts and multiplies, ldl/ldr,
sdl/sdr, dmfc1/dmtc1, dext/dins, dsbh/dshd, dclz/dclo and their siblings
unconditionally, assigning an I64 to an I32 guest register. That trips
either the vassert in putIReg or the IR sanity check at the end of the
superblock, and both unwind out of the whole translation: the caller gets
an empty IRSB whose Ijk_NoDecode points at the start of the block instead
of at the instruction that failed, so the instructions decoded ahead of
it are thrown away too.

Reject those encodings before the main opcode switch, which reaches the
same decode_failure every other unsupported encoding uses and ends the
block cleanly at the offending instruction.

Only encodings that fail today are listed, so the check cannot take away
a translation that currently succeeds: LWU and SD are decoded in 32-bit
mode by the cases below and are left alone. Over 1,000,961 encodings --
every one of the 131,072 (opcode, rs, function) dispatch triples twice,
a shift-amount sweep for the sub-opcode groups that dispatch on it, every
distinct instruction word of six MIPS III binaries, and 300,000 uniformly
random words -- 116,238 fall in the rejected set, and libVEX decoded none
of them on a 32-bit guest before this change: 93,595 aborted on an
assertion, 16,938 on the IR sanity check, and 5,705 already reported a
plain decode failure. MIPS64 lifts every one of the 1,000,961 to
bit-identical IR before and after.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@zardus

zardus commented Aug 27, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 561795c17f97a008108d8642d4854c56e79513a2 against baseline 875f7c9a5f6be621b4f000c29c016e15ddf32207, exercised through angr/pyvex#567, which pins exactly this head and carries the tests; this repository has no test suite of its own.

  • Regression: pytest tests/test_mips32_reserved.py on pyvex dca28718d29976fa0a775199076c8197bd177f89, whose submodule is this head — 3 passed, in a clean virtualenv with pyvex rebuilt from the pin, gcc 15.3.0, CPython 3.12.14, Linux x86-64
  • Baseline: with the submodule reverted to 875f7c9 and the library rebuilt, test_mips32_reserved_instruction_keeps_the_decoded_prefix fails on assert irsb.size == len(PREFIX), 0 == 8, for ld $a0, -0x3098($v1); the other two tests pass either way, which is their job — they assert that MIPS64 still decodes these encodings and that SD and LWU still decode on MIPS32
  • Reproducer: pyvex.lift(bytes.fromhex("3c038004" "2462cf68" "dc64cf68"), 0x80010938, pyvex.ARCH_MIPS32_BE) returns size 0, 0 instructions, Ijk_NoDecode to 0x80010938 (the block start) on baseline, and size 8, 3 instructions, Ijk_NoDecode to 0x80010940 (the ld) on head
  • Full suite and workspace gate: recorded on Keep the decoded prefix when MIPS32 meets a MIPS64-only instruction pyvex#567 at pyvex 343d3b7 with this vex head — every adopted suite green, including pyvex 67 passed, angr 2,471 passed / 46 skipped / 2 xfailed, angr Rust 35 passed, cle 202 passed, claripy 331 passed, archinfo 17 passed, pypcode 46 passed, angr-management 533 passed. That branch has since been rebased to dca28718; git range-diff reports the commit unchanged
  • Lint/type: none configured in this repository; gcc -Wall -Wextra -fsyntax-only on priv/guest_mips_toIR.c emits no diagnostic that master does not, and pyvex's pre-commit set passes on the paired branch

Lifter differential, 1,000,961 encodings on each of the four MIPS guests: every one of the 131,072 (opcode, rs, function) dispatch triples twice, a shift-amount sweep for the sub-opcode groups that also dispatch on it, every distinct instruction word of six MIPS III binaries, and 300,000 uniformly random words. Each is lifted behind two decodable instructions so a lost prefix is visible, and the whole IR is fingerprinted rather than just the size.

guest records changed
MIPS32 BE 110,533
MIPS32 LE 110,533
MIPS64 BE 0
MIPS64 LE 0
  • Every one of the 221,066 changed records is the same transition — 187,190 from an assertion abort and 33,876 from a sanity-check abort, both to a clean failure that keeps the prefix. No record moved in any other direction, and MIPS64 lifts all 1,000,961 to bit-identical IR
  • Of the 116,238 sampled encodings the new check rejects, stock libVEX decoded none on a 32-bit guest: 93,595 aborted on an assertion, 16,938 on the IR sanity check, and 5,705 already reported a plain decode failure and are bit-identical after the change

CFG measurement on the affected population — MIPS III or n32 code in an ELFCLASS32 container, which CLE resolves to ArchMIPS32. Six big-endian MIPS III ET_EXEC images (e_flags = 0x20002001), CFGFast(normalize=True, resolve_indirect_jumps=True):

before after
blocks 41,032 46,087
instructions 176,488 194,141
functions 8,075 9,539
bytes of sized STT_FUNC symbols covered 53.5% 58.3%
DWARF line-table addresses no block covers 49.8% 41.7%

CFGFast wall time over the six is 30.4 s before and 30.4 s after, with no new error, no new timeout, and no new block in data, outside an executable range, in synthetic memory, or overlapping another.

447 MIPS control objects covering every container and bitness in the sample — ELF, blob, ihex, srec, ar, elfcore, cart, 32- and 64-bit, VEX- and p-code-lifted:

before after
objects whose CFG changed at all 63 of 447
blocks 813,164 816,659
functions 25,014 25,647
instructions 6,582,814 6,579,168
sized-function bytes covered 1,282,216 1,302,188
CFGFast seconds 1,701.4 1,701.8
timeouts 5 5
  • Error classes are identical on both sides. All 63 changed objects are ArchMIPS32 and contain at least one rejected encoding; no ArchMIPS64 object moved. Instructions fall slightly overall because several MIPS core dumps stop decoding data sooner — blocks_outside_executable drops from 638 to 489
  • The 50 MIPS ELF fixtures in angr/binaries are bit-for-bit identical before and after, block for block
  • One category moves the wrong way: sized function symbols that no block covers rise, 527 to 642 over the six images and 70 to 101 over the control set. That is CFGFast.drop_bad_functions() deleting short functions whose recovered part now ends in Ijk_NoDecode. Replacing the pass with a no-op separates the effects — on object d80ab7715414a408 uncovered symbols go 43 to 7 with the pass disabled, and on 698672a5399ff657 26 to 4 — so the lift recovers real code, named by the objects' own DWARF line tables, and the heuristic then deletes it. Reported separately as drop_bad_functions deletes real functions whose block ends in an instruction the lifter does not implement angr#6858, because that is an angr decision
  • One raw 320 KB blob takes blocks_overlapping 0 to 1, at a jr and its delay slot at 0x8001e454 where the following instruction is scd; those bytes are data being scanned as code, and the overlap is angr's existing delay-slot block boundary behaviour (CFGFast asserts on a single-instruction block ending in an indirect jump on delay-slot architectures angr#6763). Over the control set blocks_overlapping still falls, 321 to 319

A/B method: two builds of the same worktree differing only in priv/guest_mips_toIR.c, selected by swapping libpyvex.so with a lock taken before the swap and the library re-checked after the run. Three earlier attempts were discarded — a swap that landed mid-measurement, a second chain whose swap landed inside the first chain's window, and batching several objects into one interpreter, which changed the answers (7,720 blocks batched against 7,696 alone) because angr keeps process-global state between projects. Every object is measured in its own forked child.

Prior art and scope: angr/pyvex#76 reported this assertion on ld in 2017 and concluded the real fix is architecture selection in CLE, which is angr/archinfo#368; this is the independent half that stops the decoder asserting on input it must reject. angr/pyvex#425 asks for general partial lifting of blocks containing invalid instructions, which this does not implement. The same unguarded code is present in upstream VEX.

Caveats: no MIPS hardware was involved; this is translation and CFG evidence only. The differential and CFG figures above were measured on the paired pyvex branch at this exact submodule pin and are reproduced here rather than re-run; what was re-run for this record is the focused regression in the first row. On a 64-bit guest, 550 of the sampled encodings still abort on an assertion — separate MIPS64-side defects, untouched here.

@zardus

zardus commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

The block reported in pyvex issue 76 -- six MIPS32 instructions then daddu $a0, $v0, $zero (0x0040202d) at 0x3c5f0 -- lifted as MIPS32, and one encoding from each affected group behind a two-instruction prefix at 0x80010938, before and after this change.

Before -- the whole superblock is discarded and next points at its own start address, so nothing decoded ahead of the reserved encoding survives and there is no address to resume from:

vex master (875f7c9), via pyvex bdd5441
pyvex: .../wt/mips-base/pyvex/__init__.py

$ pyvex.lift(8fc2002cac4000008fc200148fc3000c8fc6002427c800100040202d0060282d8fc700288f998324, 0x3c5d8, ARCH_MIPS32_BE, opt_level=0)
  size=0  instructions=0  jumpkind=Ijk_NoDecode  next=0x0003c5d8
  instruction_addresses=()

IRSB {
   

   NEXT: PUT(pc) = 0x0003c5d8; Ijk_NoDecode
}

$ for each group, pyvex.lift(3c038004 2462cf68 <encoding>, 0x80010938, ARCH_MIPS32_BE)
  encoding                 word       size insns next
  ld    $a0, -0x3098($v1)  dc64cf68      0     0 0x80010938
  daddiu $v0, $v1, 0x7d    6462007d      0     0 0x80010938
  daddu $v0, $v1, $a0      0064102d      0     0 0x80010938
  dsll  $v0, $v0, 0x17     000215f8      0     0 0x80010938
  dsll32 $v1, $v1, 0       0003183c      0     0 0x80010938
  dsrl32 $v0, $v0, 0x14    0002153e      0     0 0x80010938
  dmult $v0, $v1           0043001c      0     0 0x80010938
  ldl   $t0, 0($v0)        68480000      0     0 0x80010938
  ldr   $t0, 7($v0)        6c480007      0     0 0x80010938
  sdl   $t0, 0($a2)        b0c80000      0     0 0x80010938
  sdr   $t0, 7($a2)        b4c80007      0     0 0x80010938
  dmtc1 $zero, $f0         44a00000      0     0 0x80010938
  dext  $a1, $v1, 0, 1     7c650003      0     0 0x80010938
  dsbh  $a1, $a0           7c0428a4      0     0 0x80010938
  dclz  $a1, $v1           70642824      0     0 0x80010938

After -- the reserved encoding takes the ordinary decode_failure path, so every instruction ahead of it is kept and the block ends at the offending address:

with this change (vex 561795c, via pyvex dca2871)
pyvex: .../wt/mips-567/pyvex/__init__.py

$ pyvex.lift(8fc2002cac4000008fc200148fc3000c8fc6002427c800100040202d0060282d8fc700288f998324, 0x3c5d8, ARCH_MIPS32_BE, opt_level=0)
  size=24  instructions=7  jumpkind=Ijk_NoDecode  next=t21
  instruction_addresses=('0x3c5d8', '0x3c5dc', '0x3c5e0', '0x3c5e4', '0x3c5e8', '0x3c5ec', '0x3c5f0')

IRSB {
   t0:Ity_I32 t1:Ity_I32 t2:Ity_I32 t3:Ity_I32 t4:Ity_I32 t5:Ity_I32 t6:Ity_I32 t7:Ity_I32 t8:Ity_I32 t9:Ity_I32 t10:Ity_I32 t11:Ity_I32 t12:Ity_I32 t13:Ity_I32 t14:Ity_I32 t15:Ity_I32 t16:Ity_I32 t17:Ity_I32 t18:Ity_I32 t19:Ity_I32 t20:Ity_I32 t21:Ity_I32

   00 | ------ IMark(0x3c5d8, 4, 0) ------
   01 | t6 = GET:I32(r30)
   02 | t5 = Add32(t6,0x0000002c)
   03 | t0 = t5
   04 | t7 = LDbe:I32(t0)
   05 | PUT(r2) = t7
   06 | PUT(pc) = 0x0003c5dc
   07 | ------ IMark(0x3c5dc, 4, 0) ------
   08 | t9 = GET:I32(r2)
   09 | t8 = Add32(t9,0x00000000)
   10 | t1 = t8
   11 | STbe(t1) = 0x00000000
   12 | PUT(pc) = 0x0003c5e0
   13 | ------ IMark(0x3c5e0, 4, 0) ------
   14 | t11 = GET:I32(r30)
   15 | t10 = Add32(t11,0x00000014)
   16 | t2 = t10
   17 | t12 = LDbe:I32(t2)
   18 | PUT(r2) = t12
   19 | PUT(pc) = 0x0003c5e4
   20 | ------ IMark(0x3c5e4, 4, 0) ------
   21 | t14 = GET:I32(r30)
   22 | t13 = Add32(t14,0x0000000c)
   23 | t3 = t13
   24 | t15 = LDbe:I32(t3)
   25 | PUT(r3) = t15
   26 | PUT(pc) = 0x0003c5e8
   27 | ------ IMark(0x3c5e8, 4, 0) ------
   28 | t17 = GET:I32(r30)
   29 | t16 = Add32(t17,0x00000024)
   30 | t4 = t16
   31 | t18 = LDbe:I32(t4)
   32 | PUT(r6) = t18
   33 | PUT(pc) = 0x0003c5ec
   34 | ------ IMark(0x3c5ec, 4, 0) ------
   35 | t20 = GET:I32(r30)
   36 | t19 = Add32(t20,0x00000010)
   37 | PUT(r8) = t19
   38 | PUT(pc) = 0x0003c5f0
   39 | ------ IMark(0x3c5f0, 0, 0) ------
   40 | PUT(pc) = 0x0003c5f0
   41 | PUT(pc) = 0x0003c5f0
   42 | t21 = GET:I32(pc)
   NEXT: PUT(pc) = t21; Ijk_NoDecode
}

$ for each group, pyvex.lift(3c038004 2462cf68 <encoding>, 0x80010938, ARCH_MIPS32_BE)
  encoding                 word       size insns next
  ld    $a0, -0x3098($v1)  dc64cf68      8     3 0x80010940
  daddiu $v0, $v1, 0x7d    6462007d      8     3 0x80010940
  daddu $v0, $v1, $a0      0064102d      8     3 0x80010940
  dsll  $v0, $v0, 0x17     000215f8      8     3 0x80010940
  dsll32 $v1, $v1, 0       0003183c      8     3 0x80010940
  dsrl32 $v0, $v0, 0x14    0002153e      8     3 0x80010940
  dmult $v0, $v1           0043001c      8     3 0x80010940
  ldl   $t0, 0($v0)        68480000      8     3 0x80010940
  ldr   $t0, 7($v0)        6c480007      8     3 0x80010940
  sdl   $t0, 0($a2)        b0c80000      8     3 0x80010940
  sdr   $t0, 7($a2)        b4c80007      8     3 0x80010940
  dmtc1 $zero, $f0         44a00000      8     3 0x80010940
  dext  $a1, $v1, 0, 1     7c650003      8     3 0x80010940
  dsbh  $a1, $a0           7c0428a4      8     3 0x80010940
  dclz  $a1, $v1           70642824      8     3 0x80010940

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant