Skip to content

Commit fa8dcf2

Browse files
committed
fix(hermes-base): handle empty functions and debug dump failures
Accept declared zero-byte functions without weakening non-empty body or function-count completeness checks. End every debug sink on spawn errors and cancellation before waiting for stream completion. Resolve restricted-global string operands and normalize classic SwitchImm physical table offsets while preserving control-flow targets. Document the specific v98 function-header snapshot rather than assuming all HBC 98 compilers share its layout. Add isolated debug-output timeout/abort/pre-abort/ENOENT regressions, real classic lexical-declaration coverage, and pinned real Metro self and base/plain comparisons. Extend CI to hermes-compiler 250829098.0.17 and run debug failure checks on Node 18. Validated full suites with real Metro fixtures: HBC96 548 passed; HBC98 .16/.17 each 547 passed, one classic-only case skipped, zero failures. Lint, typecheck, build and Node 22 debug/smoke checks passed.
1 parent 5e1943d commit fa8dcf2

8 files changed

Lines changed: 347 additions & 12 deletions

File tree

.github/workflows/test.yml

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,11 @@ jobs:
107107
node tests/fixtures/hermes-async-check.cjs
108108
'{"operation":"abort","modulePath":"./lib/utils/hermes-base.js"}'
109109
110+
- name: Check debug-output failure cleanup on Node 18
111+
env:
112+
HERMES_TEST_NODE: node
113+
run: bun test tests/hermes-blockers.test.ts
114+
110115
publish-dry-run:
111116
runs-on: blacksmith-4vcpu-ubuntu-2404
112117
timeout-minutes: 10
@@ -141,21 +146,28 @@ jobs:
141146
run: npm publish --dry-run --access public --tag dry-run
142147

143148
hermes-integration:
144-
name: hermes-hbc-${{ matrix.hbc }}
149+
name: hermes-hbc-${{ matrix.hbc }}${{ matrix.suffix }}
145150
runs-on: blacksmith-4vcpu-ubuntu-2404
146151
timeout-minutes: 15
147152
strategy:
148153
fail-fast: false
149154
matrix:
150155
include:
151156
- hbc: 96
157+
suffix: ""
152158
package: react-native@0.77.3
153159
directory: react-native
154160
executable: sdks/hermesc/linux64-bin/hermesc
155161
- hbc: 98
162+
suffix: ""
156163
package: hermes-compiler@250829098.0.16
157164
directory: hermes-compiler
158165
executable: hermesc/linux64-bin/hermesc
166+
- hbc: 98
167+
suffix: -patch17
168+
package: hermes-compiler@250829098.0.17
169+
directory: hermes-compiler
170+
executable: hermesc/linux64-bin/hermesc
159171
steps:
160172
- uses: actions/checkout@v7
161173
with:
@@ -184,6 +196,20 @@ jobs:
184196
"$HERMESC" -version
185197
bun -e 'import {probeHbcVersion} from "./src/utils/hermes-base"; if (probeHbcVersion(process.env.HERMESC) !== Number(process.env.EXPECTED_HBC)) throw new Error("unexpected HBC version");'
186198
echo "HERMESC=$HERMESC" >> "$GITHUB_ENV"
199+
- name: Fetch pinned real Metro fixtures
200+
shell: bash
201+
run: |
202+
set -euo pipefail
203+
root="$RUNNER_TEMP/hermes-metro-fixtures"
204+
mkdir -p "$root"
205+
base="https://raw.githubusercontent.com/sunnylqm/hbc-diff-benchmark/e6a870a1c26c4b64c7860d7e1aa575707d22ad88"
206+
for file in base.jsbundle s3-medium-feature.jsbundle; do
207+
curl --fail --location --retry 2 --max-time 60 "$base/fixtures/$file" -o "$root/$file"
208+
done
209+
curl --fail --location --retry 2 --max-time 60 "$base/LICENSE" -o "$root/LICENSE"
210+
echo "11c8ad8f7e8c7c59ee45582c77d896a35fa646617f3ba0f5b338a425a7c93b7d $root/base.jsbundle" | sha256sum --check
211+
echo "a693e68254b6c13fae8f839d20c14f9d11c5ab98d4be1b8d13ba1e929a12d752 $root/s3-medium-feature.jsbundle" | sha256sum --check
212+
echo "HERMES_METRO_FIXTURES=$root" >> "$GITHUB_ENV"
187213
- name: Run real compiler and fallback regressions
188214
run: bun test tests/hermes-*.test.ts
189215
- name: Run seeded differential fuzzing
@@ -192,6 +218,6 @@ jobs:
192218
if: failure()
193219
uses: actions/upload-artifact@v7
194220
with:
195-
name: hermes-fuzz-hbc-${{ matrix.hbc }}
221+
name: hermes-fuzz-hbc-${{ matrix.hbc }}${{ matrix.suffix }}
196222
path: ${{ runner.temp }}/hermes-fuzz
197223
if-no-files-found: ignore

docs/hermes-base-verification.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,14 +37,18 @@ v98 的 shape 索引和 offset 一样只用于定位(delta 可能重排 shape
3737

3838
为什么不能按 dump 的整段文本比:Hermes 的缓冲区构建器会**重叠/去重**序列化后的字面量——一个字面量的最后一个值字节可以同时是下一个字面量的 tag 字节(模糊测试实测:`61 52 | cd 09 b3 05 11`,前一段以 `[String 82]` 结尾,后一条指令的 offset 正指向 `52`)。顺序解析整段缓冲区(hermesc 的 dump 就是这么打印的)从这里开始失步,之后的条目全是噪声;delta 构建的 id 宽度不同,重叠位置也不同,于是两段"噪声"在某处不一致就被判为差异。2026-09-10 的 20 轮冒烟模糊测试里 3 次误杀全部源于此,改按指令比较后全部等价。无法读二进制缓冲区(文件结构不识别)时两侧一起比较整段文本,仅辅助诊断;即使文本相等也返回 `dump-failed` 并回退 plain,不能以丢失 offset/count 的文本确认等价。结果里 `literals: 'buffer'` 标明这一点。
3939

40-
`normalizeDisassemblyLine` 只折叠表示层差异:按指令解析后的字面量地址、已知宽度后缀、引号外的列对齐空白、switch 表的物理偏移、debug 偏移。字符串内部的连续空格、跳转目标标签、寄存器均保留。未知 string ID、无法解码的字面量、未知 buffer 操作数形态直接失败;两侧都无法解析也不等价。
40+
`normalizeDisassemblyLine` 只折叠表示层差异:按指令解析后的字面量地址、已知宽度后缀、引号外的列对齐空白、switch 表的物理偏移(含经典 `SwitchImm`、debug 偏移。字符串内部的连续空格、跳转目标标签、寄存器均保留。未知 string ID、无法解码的字面量、未知 buffer 操作数形态直接失败;两侧都无法解析也不等价。
4141

4242
**原始操作数核对**`hermes-raw.ts` 从 raw dump 读取指令起点和操作数类型,并检查操作数与 HBC 字节一致、指令覆盖完整函数体。字符串从 small/overflow string table 与 string storage 按完整 ASCII/UTF-16 code unit 解码;BigInt、正则和 double 读取真实字节(保留 `-0` 和尾部精度);函数引用保留索引,与顺序对齐的函数表共同检查,同名函数不能互换。地址映射为目标指令序号;整数和字符串 switch 从二进制恢复 case 值与目的地。函数运行时 flags 和参数/寄存器等字段也参与比较,剔除的仅是物理地址、debug presence 与 compact/overflow 表示。
4343

44-
这不是一个可以忽略所有新指令的通用语义证明器。支持范围限定为已实现的 HBC v87–96v98;升级布局或引入新的字符串/shape 引用指令时,需要复核 `hermes-raw.ts` 的解码规则及测试,不可仅扩展 diff-transform 的布局表。
44+
这不是一个可以忽略所有新指令的通用语义证明器。已验证的函数头范围为 HBC v87–96,以及 `hermes-compiler@250829098.0.16/.17`v98 快照;升级布局或引入新的字符串/shape 引用指令时,需要复核 `hermes-raw.ts` 的解码规则及测试,不可仅扩展 diff-transform 的布局表。
4545

4646
结果三态:`equivalent` / `different`(带第一处差异:函数、行号、两侧内容,或缓冲区条目)/ `dump-failed`(无法解析完整语义数据、dump 进程失败/超时/提前结束;带原因或 stderr 末行)。后两种都放弃 base,但日志分开。
4747

48+
**零长度函数**:Static Hermes 的真实 Metro 产物可能保留 `bytecodeSizeInBytes == 0` 的死函数。raw 校验要求实际指令字节总长等于函数头声明长度,而不是要求每个函数至少有一条指令;空函数仍比较运行时元数据,遗漏整个函数头仍由总函数数检查拒绝。非空函数的指令被截断仍为 `dump-failed`
49+
50+
**v98 函数头快照边界**:当前大头按 37 字节、flags 位于 `[36]` 读取,小头 cache 位域为 6/1/1,绑定 `250829098` 稳定快照。上游 [7193d4485b](https://github.com/facebook/hermes/commit/7193d4485beeb87cd7a3b6ca8b6b5d97a1a433c4) 删除 `NumCacheNewObject` 后,仍报 v98 的构建曾使用 36 字节 / flags `[35]`、cache 位域 7/1。`hbcTransform` 的两套 v98 文件头布局不能识别这次**函数头**变化;本轮不宣称支持该后续快照,也不能仅凭文件头或版本号选择它。更换编译器时必须补对应的大头、小头和真实 Metro 测试;未审核的 v98 构建应使用 `--hermesBase none`
51+
4852
pretty 输出本身会截断长字符串与 BigInt、用函数名替代函数索引,并可能把 `-0` 显示为 `0`,因此不再以 pretty 相等作为最终结论。新增归一化规则时必须同时添加真实 HBC 负例,确保没有把语义差异折叠掉。
4953

5054
## 3. 已完成与待办
@@ -82,7 +86,9 @@ raw 核对保留函数引用索引,与出现顺序对齐的函数体及二进
8286
- **资源开销**:base/plain 编译并发完成后,执行 pretty 和 raw 两遍 dump;每遍两个进程,raw 不增加编译。二进制元数据读取目前持有两份 HBC、完整字符串映射及字面量缓冲区,反汇编仅保留当前函数;这是完整数据核对的额外内存和时间开销。不要以删除验证数据来优化内存,可后续改为按段读取或降低并行度。
8387
- **进程期限**:版本探测默认 30 秒(`PUSHY_HERMES_PROBE_TIMEOUT_MS`),完整校验两遍合计默认 120 秒(`PUSHY_HERMES_VERIFY_TIMEOUT_MS`),单个编译/源码映射子进程默认 300 秒(`PUSHY_HERMES_COMPILE_TIMEOUT_MS`)。环境变量单位均为毫秒,必须是 1–2147483647 的整数,否则用默认值。超时终止子进程,优化失败回退 plain;真正的 plain 编译或最终 sourcemap 失败仍使构建失败。校验函数还接受 `AbortSignal`;base 下载任务的取消传播尚未统一。
8488
- **源码映射竞态**:推测执行的 base sourcemap 合成任务启动时立即观察拒绝,之后再根据最终采用哪份字节码决定抛出错误还是为 plain 重做合成。
85-
- **CI**`hermes-hbc-96` / `hermes-hbc-98` job 分别安装固定的 `react-native@0.77.3` / `hermes-compiler@250829098.0.16`,校验可执行文件与真实 HBC 版本后运行 Hermes 回归和 50 轮固定种子 fuzz;缺少编译器会失败,不静默跳过。
89+
- **CI**`hermes-hbc-96` / `hermes-hbc-98` / `hermes-hbc-98-patch17` 分别安装固定的 `react-native@0.77.3` / `hermes-compiler@250829098.0.16` / `.17`。除小程序回归与 50 轮固定种子 fuzz 外,三组都执行真实 Metro bundle 的自比和 base/plain 比较;v98 测试断言产物确实包含零长度函数。缺编译器或指定的 fixture 缺失/哈希不符会失败,不静默跳过。
90+
- **真实 Metro 数据**[hbc-diff-benchmark](https://github.com/sunnylqm/hbc-diff-benchmark/tree/e6a870a1c26c4b64c7860d7e1aa575707d22ad88)`base.jsbundle``s3-medium-feature.jsbundle`(MIT,保留其 LICENSE;来源与生成步骤见该仓库 `fixtures/GENERATION.md`)。CI 固定提交和 SHA256,不运行下载的 JS,仅交给固定编译器。离线运行:`HERMESC=<hermesc> HERMES_METRO_FIXTURES=<fixtures目录> bun test tests/hermes-metro.test.ts`;普通单元测试不联网下载。
91+
- **调试输出异常回归**`tests/hermes-blockers.test.ts` 在独立进程覆盖带 `dumpTo` 的超时、运行中取消、预取消、ENOENT,同时检查返回标记、退出状态和晚到的异步错误。错误处理立即 unpipe 全部目的地并结束 debug 文件,避免 `finish()` 等待一个只会在后续 `kill()` 中结束的流;Node 18 CI 也执行相同场景。
8692
- **`hbcdump`/`hbc-diff`**:Hermes 仓库自带的工具,RN 的 hermesc 不随附;如果将来 hermes-compiler 包里带上,可替代文本 dump 解析。
8793

8894
## 4. 明确接受的剩余风险

src/utils/hermes-base.ts

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1134,14 +1134,16 @@ export function normalizeDisassemblyLine(
11341134
// the table header hermesc prints for them) moves with instruction widths.
11351135
// The two switch instructions carry that offset in different operands:
11361136
// StringSwitchImm rX, <id>, <jtOffset>, <defaultLabel>, <count>
1137-
// UIntSwitchImm rX, <jtOffset>, <defaultLabel>, <min>, <max>
1137+
// UIntSwitchImm (classic: SwitchImm) rX, <jtOffset>, <defaultLabel>, <min>, <max>
11381138
// Folding only the first shape let a shifted UIntSwitchImm offset read as a
11391139
// real difference and drop an otherwise good delta build.
11401140
if (folded === 'StringSwitchImm') {
11411141
m = /^(\s*StringSwitchImm r\d+, \d+, )\d+(, L\d+, \d+)$/.exec(line);
11421142
if (m) line = `${m[1]}<jt>${m[2]}`;
1143-
} else if (folded === 'UIntSwitchImm') {
1144-
m = /^(\s*UIntSwitchImm r\d+, )\d+(, L\d+, \d+, \d+)$/.exec(line);
1143+
} else if (folded === 'UIntSwitchImm' || folded === 'SwitchImm') {
1144+
m = /^(\s*(?:UIntSwitchImm|SwitchImm) r\d+, )\d+(, L\d+, \d+, \d+)$/.exec(
1145+
line,
1146+
);
11451147
if (m) line = `${m[1]}<jt>${m[2]}`;
11461148
} else if (folded === 'offset' && /^\s*offset \d+$/.test(line)) {
11471149
line = line.replace(/\d+$/, '<jt>');
@@ -1341,9 +1343,12 @@ class DumpReader {
13411343
let processError: Error | undefined;
13421344
// a spawn failure (ENOENT) may leave stdout open and never 'close'
13431345
proc.on('error', (error) => {
1344-
proc.stdout?.unpipe(pass);
1346+
// destroy() need not emit end, so pipe() will not finish its sinks.
1347+
// End the debug file here: finish() awaits it before finally/kill().
1348+
proc.stdout?.unpipe();
13451349
proc.stdout?.destroy();
13461350
pass.end();
1351+
this.debugOutput?.end();
13471352
processError = error;
13481353
if (!proc.pid) resolve({ code: null, signal: null, error, stderr });
13491354
});

src/utils/hermes-raw.ts

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,14 @@ export async function readHermesSemanticData(
9797
}
9898
const string = (id: number) =>
9999
requireValue(strings.get(id), `unresolved string id ${id}`);
100+
// Audited v98 function schema: hermes-compiler 250829098.0.16/.17
101+
// (250829098 stable snapshot), NOT every static_h build reporting HBC 98.
102+
// Large headers: 37 bytes, flags[36]; small cache bits: 6/1/1. Upstream
103+
// 7193d4485beeb87cd7a3b6ca8b6b5d97a1a433c4 removed NumCacheNewObject
104+
// without immediately bumping HBC: 36 bytes/flags[35], cache bits 7/1.
105+
// hbcTransform's file-header variants do not distinguish that function
106+
// schema change. A compiler upgrade needs independent large/small fixtures;
107+
// do not infer either schema from numStringSwitchImms or HBC version alone.
100108
const shaped = resolved.version === 98;
101109
const entrySize = shaped ? 12 : 16;
102110
const headers = section('functionHeaders');
@@ -225,6 +233,7 @@ const OPERAND_BYTES: Record<string, number> = {
225233
// DefineOwnById annotation supplied explicitly. Keep classic and v98 variants.
226234
const STRING_OPERANDS: Record<string, number[]> = {
227235
DeclareGlobalVar: [0],
236+
ThrowIfHasRestrictedGlobalProperty: [0],
228237
GetById: [3],
229238
GetByIdWithReceiver: [4],
230239
TryGetById: [3],
@@ -340,7 +349,10 @@ export function normalizeRawHermesFunction(
340349
targets.set(inst.offset, index);
341350
end += inst.size;
342351
}
343-
if (end !== fn.size || instructions.length === 0) {
352+
// Static Hermes retains legal zero-byte (dead) functions in the table.
353+
// Their metadata is still audited, and the raw reader still requires every
354+
// function header. Only a body shorter/longer than its declared size fails.
355+
if (end !== fn.size) {
344356
throw new UnverifiableHermesBytecode(
345357
'raw dump ended before the function body',
346358
);

tests/fixtures/hermes-async-check.cjs

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,25 @@ async function main() {
3737
);
3838
assert.equal(result.status, 'dump-failed');
3939
assert.match(result.detail, /abort/i);
40+
} else if (config.operation === 'verify') {
41+
const { compareHermesBytecode } = require(path.resolve(config.modulePath));
42+
const controller = new AbortController();
43+
let timer;
44+
if (config.abortAfterMs === 0) controller.abort();
45+
else if (config.abortAfterMs !== undefined) {
46+
timer = setTimeout(() => controller.abort(), config.abortAfterMs);
47+
}
48+
try {
49+
result = await compareHermesBytecode(
50+
config.command,
51+
'missing-a',
52+
'missing-b',
53+
{ ...config.options, signal: controller.signal },
54+
);
55+
assert.equal(result.status, 'dump-failed');
56+
} finally {
57+
clearTimeout(timer);
58+
}
4059
} else if (config.operation === 'compile') {
4160
const { compileHermesByteCode } = require(path.resolve(config.modulePath));
4261
result = await compileHermesByteCode(config.options);

0 commit comments

Comments
 (0)