Skip to content

s390x: keep the EXRL prefetch inside the code buffer - #89

Open
zardus wants to merge 1 commit into
masterfrom
feature/fix-pyvex-s390x
Open

s390x: keep the EXRL prefetch inside the code buffer#89
zardus wants to merge 1 commit into
masterfrom
feature/fix-pyvex-s390x

Conversation

@zardus

@zardus zardus commented Aug 11, 2026

Copy link
Copy Markdown
Member

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

s390_irgen_EXRL reads its execute target straight out of the buffer being
translated, and nothing bounds where it reads. Six well-formed bytes take the
process down — c660ffffffff is exrl %r6,.-2, the backward form compilers
emit for literal-pool-relative MVC and CLC:

$ pyvex.lift(bytes.fromhex("c660ffffffff"), 0x1000, archinfo.ArchS390X())
  [killed by SIGSEGV]
$ pyvex.lift(bytes.fromhex("c66000000003"), 0x1000, archinfo.ArchS390X())
  NEXT: PUT(ia) = 0x0000000000001000; Ijk_InvalICache

The sign of the displacement is the whole difference. A truncated encoding does
the same: a75bffffc650ff is aghi %r5,-1 followed by the first three bytes of
an exrl, which is what a scanning CFG hands the lifter routinely. There is no
exception to catch — the client process dies.

Root cause

The displacement is read unsigned, and the resulting pointer is never checked
against the buffer:

Addr64 bytes_addr = guest_IA_curr_instr + offset * 2UL;
UChar *bytes = exrl_bytes + offset * 2UL;
if (!last_execute_target)
   last_execute_target = ((ULong)bytes[0] << 56) | ...

offset is UInt, so a negative RIL-b displacement becomes a forward offset of
up to 8 GB. Of the 26 pc-relative RIL handlers in priv/guest_s390_toIR.c this
is the only one that reads the field unsigned; 24 spell it
(ULong)(Long)(Int)i2 << 1. Where the read happens to land on mapped memory it
does not fault at all: it fabricates an execute target out of whatever bytes are
there and bakes that into the IR.

Fix

Record the buffer base in disInstr_S390, sign-extend the displacement to
Long, and prefetch only when all six target bytes lie inside the buffer.
Otherwise last_execute_target stays zero and s390_irgen_EX emits the
run-time lookup it already emits for an unknown target, so the same six bytes
now lift and the lookup reads the right address:

   00 | ------ IMark(0x1000, 6, 0) ------
   01 | t1 = LDbe:I64(0x0000000000000ffe)
   02 | t4294967295 = DIRTY 1 TODO(effects) ::: s390x_dirtyhelper_EX(t1)
   NEXT: PUT(ia) = 0x0000000000001000; Ijk_InvalICache

The buffer's extent is the client's to state, so VexControl gains
guest_bytes_size; zero means unstated, and the bound falls back to
guest_max_bytes, which is what every existing caller gets. A target that lies
in the caller's buffer but before the address it stated is deliberately not
prefetched; lookback_amount could allow that later.

Testing

Regressions live in angr/pyvex#564, which carries the submodule bump and
tests/test_s390x_exrl.py. All six pass on this branch, five runs of five. At
the merge base two are killed by SIGSEGV, one fails its assertion in all five
runs, and one is unstable — it reads uninitialised heap 8 KiB past the buffer
and passed four of five.

Validation: #89 (comment)

Merge order: this lands first, then angr/pyvex#564, whose vex submodule is pinned to this head (88aa12d). Merging the pyvex half on its own leaves pyvex master's submodule pointing at a commit that is on no vex branch. Nothing in this repository reads a sync: line — .github/workflows/build.yml is the only workflow and it is three build jobs — so the line below is a note to the reader.

sync: angr/pyvex#564

session: sharpen

s390_irgen_EXRL reads its target straight out of the buffer being
translated, at exrl_bytes + offset * 2, with nothing bounding the
result. Nothing constrains the displacement field either, so a bogus or
truncated EXRL points anywhere within 4GB of the buffer: lifting the
seven bytes a75bffff c650ff as s390x is enough to kill the process. The
field is also read as unsigned, unlike every other RIL-b handler, which
turns a negative displacement into a read about 8GB ahead of the buffer
and puts that address in the emitted lookup as well.

Record the buffer base in disInstr_S390, sign-extend the displacement,
and prefetch only when the whole six-byte target lies inside the buffer.
Leaving last_execute_target at zero makes s390_irgen_EX emit the generic
run-time lookup, which is what it already does when the target is
unknown.

The buffer is normally larger than the block being translated, so
VexControl gains guest_bytes_size for the client to say how much of it
may be read; without it only guest_max_bytes counts.
@zardus

zardus commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 88aa12d54fb50e91635e81c0f3c0bffa1b712691 against baseline 875f7c9a5f6be621b4f000c29c016e15ddf32207, built and exercised through pyvex bc8b2245ff1ef9ffa5250b06972e14bd134bcf43 (angr/pyvex#564), which is where s390x lifting has tests. CPython 3.12.13, Linux x86-64, GCC 15.2.0.

  • Regression: python -c 'import archinfo, pyvex; pyvex.lift(bytes.fromhex("a75bffffc650ff"), 0x1000, archinfo.arch_from_id("s390x"))' — baseline dies with SIGSEGV, exit 139; head returns a 4-byte Ijk_Boring block
  • Focused: pytest tests/test_s390x_exrl.py — 6 passed
  • Full suite: pytest tests — 69 passed, 0 skipped

EXRL survey over the 67 s390x ELFs in angr/binaries 12d015e510e2a5fe1217ef061166b7ed7fda7a04: every six-byte c6?0 site in every executable segment, 1309 in total, translated with the whole segment as the code buffer and guest_max_bytes at 400, one fork per site, IRSB text digests compared.

Outcome Baseline Head
Target prefetched, block continues past the EXRL 157 157
Ijk_InvalICache, target left to the run-time lookup 21 90
Ijk_NoDecode 1066 1062
Killed by SIGSEGV 65 0
  • The 157 that prefetch are byte-identical on both revisions, so no resolution is lost. The exrl at 0xcf204 in tests/s390x/libc.so.6 executes the xc at 0xcf22a on both, for instance.
  • All 69 sites that differ had the baseline reading outside the buffer, and all 69 now stop at Ijk_InvalICache.

Other front ends are untouched by construction, and measured so: 27,918 lifts over X86, AMD64, ARMEL, AARCH64, MIPS32, MIPS64, PPC32, PPC64 and S390X from the same angr/binaries revision are byte-identical on baseline and head.

Caveats: no s390x hardware was involved; this is translation evidence only. A target inside the code buffer but before guest_code is not prefetched, which lookback_amount could allow later; no such site exists in the corpus above.

@zardus

zardus commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Three s390x EXRL encodings lifted at 0x1000, before and after this change. Each lift runs in its own child process so a fatal signal is visible, and each is repeated six times. libVEX is exercised through pyvex: the before side is vex 875f7c9, this branch's merge base and current angr/vex master, and the after side is this branch at 88aa12d. Both are built under pyvex, at bdd5441 and at fee0c3a (angr/pyvex#564) respectively, which is where s390x lifting has tests.

Before — the backward exrl and the truncated one each take the process down; only the forward one lifts:

angr master
pyvex: .../wt/s390-b564/pyvex/__init__.py

$ pyvex.lift(bytes.fromhex("a75bffffc650ff"), 0x1000, archinfo.ArchS390X()).pp()   # aghi %r5,-1 ; truncated exrl
  [killed by SIGSEGV]
  [5 further runs: killed by SIGSEGV x5]

$ pyvex.lift(bytes.fromhex("c660ffffffff"), 0x1000, archinfo.ArchS390X()).pp()   # exrl %r6,.-2   (backward, well-formed)
  [killed by SIGSEGV]
  [5 further runs: killed by SIGSEGV x5]

$ pyvex.lift(bytes.fromhex("c66000000003"), 0x1000, archinfo.ArchS390X()).pp()   # exrl %r6,.+6   (forward, control)
  IRSB {
     t0:Ity_I64 t1:Ity_I64 t2:Ity_I64
  
     00 | ------ IMark(0x1000, 6, 0) ------
     01 | t1 = LDbe:I64(0x0000000000001006)
     02 | t4294967295 = DIRTY 1 TODO(effects) ::: s390x_dirtyhelper_EX(t1)
     03 | PUT(cmstart) = 0x0000000000001000
     04 | PUT(cmlen) = 0x0000000000000004
     NEXT: PUT(ia) = 0x0000000000001000; Ijk_InvalICache
  }
    size = 6  jumpkind = Ijk_InvalICache
  [ok]
  [5 further runs: ok x5]

After — all three lift; the out-of-buffer targets fall back to the run-time lookup s390_irgen_EX already emits:

with this change
pyvex: .../wt/s390-h89/pyvex/__init__.py

$ pyvex.lift(bytes.fromhex("a75bffffc650ff"), 0x1000, archinfo.ArchS390X()).pp()   # aghi %r5,-1 ; truncated exrl
  IRSB {
     t0:Ity_I64 t1:Ity_I64 t2:Ity_I64 t3:Ity_I64 t4:Ity_I64
  
     00 | ------ IMark(0x1000, 4, 0) ------
     01 | t0 = GET:I64(r5)
     02 | t1 = Add64(t0,0xffffffffffffffff)
     03 | PUT(736) = 0x0000000000000004
     04 | PUT(744) = t0
     05 | PUT(752) = 0xffffffffffffffff
     06 | PUT(760) = 0x0000000000000000
     07 | PUT(r5) = t1
     NEXT: PUT(ia) = 0x0000000000001004; Ijk_Boring
  }
    size = 4  jumpkind = Ijk_Boring
  [ok]
  [5 further runs: ok x5]

$ pyvex.lift(bytes.fromhex("c660ffffffff"), 0x1000, archinfo.ArchS390X()).pp()   # exrl %r6,.-2   (backward, well-formed)
  IRSB {
     t0:Ity_I64 t1:Ity_I64 t2:Ity_I64
  
     00 | ------ IMark(0x1000, 6, 0) ------
     01 | t1 = LDbe:I64(0x0000000000000ffe)
     02 | t4294967295 = DIRTY 1 TODO(effects) ::: s390x_dirtyhelper_EX(t1)
     03 | PUT(cmstart) = 0x0000000000001000
     04 | PUT(cmlen) = 0x0000000000000004
     NEXT: PUT(ia) = 0x0000000000001000; Ijk_InvalICache
  }
    size = 6  jumpkind = Ijk_InvalICache
  [ok]
  [5 further runs: ok x5]

$ pyvex.lift(bytes.fromhex("c66000000003"), 0x1000, archinfo.ArchS390X()).pp()   # exrl %r6,.+6   (forward, control)
  IRSB {
     t0:Ity_I64 t1:Ity_I64 t2:Ity_I64
  
     00 | ------ IMark(0x1000, 6, 0) ------
     01 | t1 = LDbe:I64(0x0000000000001006)
     02 | t4294967295 = DIRTY 1 TODO(effects) ::: s390x_dirtyhelper_EX(t1)
     03 | PUT(cmstart) = 0x0000000000001000
     04 | PUT(cmlen) = 0x0000000000000004
     NEXT: PUT(ia) = 0x0000000000001000; Ijk_InvalICache
  }
    size = 6  jumpkind = Ijk_InvalICache
  [ok]
  [5 further runs: ok x5]

@zardus

zardus commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

A corpus measurement, plus three pieces of provenance that make this change
easier to accept than the diff alone suggests.

It is a rule this file already follows everywhere else. EXRL takes a
RIL-b operand — a signed 32-bit halfword displacement — and is the only one of
the three RIL-b handlers that reads it unsigned:

s390_irgen_EXRL   guest_IA_curr_instr + offset * 2UL              /* UInt, unsigned */
s390_irgen_LARL   guest_IA_curr_instr + ((ULong)(Long)(Int)i2 << 1)
s390_irgen_BRASL  guest_IA_curr_instr + ((ULong)(Long)(Int)i2 << 1)

LARL and BRASL sign-extend; EXRL does not, and it is the one that
dereferences the result. So this is a guard the file has at its sibling sites
and never got at this one, rather than a new check.

Upstream fixed exactly this five years ago. Valgrind's
b77dbefe72e4a5c7bcf1576a02c909010bd56991 (Andreas Arnez, 2021-10-22, KDE
444242, "s390x: Sign-extend 'relative long' offset in EXRL") replaced the same
expression with addr_rel_long(offset), which sign-extends, and its message
says the bug typically crashed Valgrind on a negative offset. This tree's s390
front end was last synced at upstream 379c62017 in 2019, so the gap is a
local one and there is nothing to report upstream. The bounds half has no
upstream counterpart at all: exrl_bytes does not exist there, because
upstream reads the execute target out of the guest process's own mapped memory
rather than out of a heap buffer.

It has been known here since 2018. angr/vex issue 21, "s390 EXLR will
probably segfault if target is outside the current segment", was filed two days
after the code that causes it landed and is still open.

How often it fires. 300 objects drawn uniformly at random from the 7,354
s390x objects in a private corpus of vendor- and distribution-produced
binaries, probed with CFGFast: 11 die with SIGSEGV, 267 complete —
3.67%, Wilson 95% [2.06, 6.45], about 270 of the 7,354. Every crash site is
a compiler-emitted EXRL in .text with a small backward displacement (-129
to -4306 halfwords) whose correctly sign-extended target lands inside the same
section and decodes to the MVC or XC template s390_irgen_EX already
special-cases — ordinary variable-length memmove and memset sequences, not
exotica.

The sign fix alone would not be enough, which is why the bounds check earns its
place: one crasher in that sample is not a real EXRL at all but mis-decoded
data, c600c010000a, whose unsigned target is 6 GiB forward. And on the
current code a positive displacement reads up to half a megabyte past a
six-byte buffer without any error, changing the emitted IR when the garbage
happens to decode — a silent wrong answer that no crash count shows.

Measured against a build of this branch with the client half applied: at the
baseline, displacements -1 and -129 and a truncated encoding all take SIGSEGV;
with this branch all of them lift cleanly, as do a positive displacement past
the end of a 64 KiB buffer and INT32_MIN.

The corpus is not redistributable, so the objects are described by architecture
and format rather than named.

session: sharpen

@zardus

zardus commented Aug 29, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

A second corpus draw, seeded independently of the one above, and the half that
measurement leaves out: the same objects run against this branch, not only
against the baseline.

Arms. Two trees materialised with git archive and built into separate
virtualenvs, so neither can shadow the other. Every child process records
pyvex.__file__ and whether the native symbol vex_lift_with_buffer_size
exists, and a run refuses to start if either disagrees with the arm it was
asked for. All 600 rows carry the expected arm.

  • basevex 875f7c9a5f6be621b4f000c29c016e15ddf32207, this branch's
    merge base and current angr/vex master, through pyvex bdd5441. Symbol
    absent.
  • headvex 88aa12d54fb50e91635e81c0f3c0bffa1b712691, this branch's
    head, through pyvex fee0c3a (Lifting: Tell libVEX how much of the lift buffer it may read. pyvex#564, whose submodule is pinned to
    it). Symbol present.

Six bytes. Each lift in its own child so a fatal signal is visible, five
runs of each case. At the base, c660ffffffff (exrl %r6,.-2) and
a75bffffc650ff (an aghi followed by a truncated exrl) are killed by
SIGSEGV in 5 of 5 runs, while the forward-displacement c66000000003 lifts in
5 of 5. At this head all three lift in 5 of 5 — the first to Ijk_InvalICache,
6 bytes, 5 statements; the truncated one to Ijk_Boring, 4 bytes, 8 statements,
having stopped before the incomplete instruction.

Corpus. 300 s390x objects drawn uniformly at random from the catalogue,
seed s390x-exrl-2026-08-29, out of a 7,354-object s390x population inside a
757,469-object internal corpus of compiler- and vendor-produced binaries; all
300 staged and sha256-checked, none missing, all 300 probed in both arms.
CFGFast under a 300 s cap:

  • merge base — 13 killed by SIGSEGV, 4.33% (Wilson 95% 2.55–7.27), which
    scales to roughly 319 of the 7,354 (interval 188–535), or 0.042% corpus-wide
  • this head — 0

What the 13 do here. Three finish CFGFast outright, recovering 6,368,
8,048 and 17,282 functions. The other ten reach the 300 s cap instead — the
crash had been ending the analysis early, so removing it costs time it never
used to spend. To separate the fix from the analysis budget, the same 13 were
swept linearly through their executable sections, lifting every block, capped
at 3 MB per object: 13 of 13 SIGSEGV at the base; 13 of 13 complete here,
1,639,763 blocks lifted over 32,366,812 bytes, zero lift errors.

Control. 258 of the 300 completed CFGFast at the merge base. All 258
still complete here, 256 with identical function, block and edge counts. The
two that differ move by one function and a handful of blocks — but two runs of
the base arm alone on one of them give 5,506 then 5,510 functions, so
CFGFast is not reproducible at that size and neither difference is
attributable to this change. The 22 objects that never reach the CFG at all — 18
to an AssertionError in pyelftools' dynamic.py:183:_get_stringtable,
reached through cle's ELF backend, and 4 to a NotImplementedError at angr's
sim_type.py:4455, both inside Project.__init__ — fail identically on both
arms.

Four of the 300 are public. They are byte-identical to files tracked in
angr/binaries at 9d9d9e0tests/s390x/allcmps, tests/s390x/ctype_b_loc,
tests/s390x/scanf_test, tests/s390x/vfg_0 — and give the same CFG on both
arms, so they are control rather than reproducers.

One footnote on the line number. guest_s390_toIR.c:13063 computes the
guest address and is harmless by itself; the read that leaves the buffer is the
next line, UChar *bytes = exrl_bytes + offset * 2UL at 13064, dereferenced at
13067-13069. exrl_bytes has no upstream counterpart — it was added here in
2018 by 87028799, whose own message says "we assume the target is in the same
buffer as EXRL ... In other cases the code would still crash or emit garbage
IR."

The rest of the corpus is not redistributable, so those objects are described
by architecture, format and sha256 rather than named.

session: sharpen

@zardus

zardus commented Aug 31, 2026

Copy link
Copy Markdown
Member Author

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Two notes from the pyvex work. Neither has changed this branch.

Measured on this branch (vex 88aa12d under pyvex ef34018): an EXRL whose target is +1000, lifted through a cffi buffer that starts 16 bytes before a PROT_NONE page. With the extent unstated, so the prefetch falls back to guest_max_bytes, it gives a 6-byte Ijk_InvalICache block at guest_max_bytes=16 and SIGSEGV at 5000. With the extent stated as 16 it gives the same 6-byte block at both.

guest_bytes_size sits in VexControl, which is process-wide, while its own comment describes VexTranslateArgs::guest_bytes, which is per call; VexTranslateArgs carries guest_bytes and guest_bytes_addr but no length for them. Say if you would rather the size sat there.

session: sharpen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant