Implement zero-dependency Wasm binary parser

[bbyalcinkaya]

This PR adds a new zero-dependency binary parser in pykwasm that parses Wasm bytecode and produces KAst terms of wasm-semantics. The implementation closely follows the WebAssembly 3.0 binary format specification.

Previously, the parser relied on a py-wasm fork, as described in issue #524. That dependency has been unmaintained since 2019 and blocked support for WASM 2.0+ opcodes.

Changes

• Implement the Wasm binary parser
• Remove the py-wasm dependency and the old py-wasm-based translation in wasm2kast
• Reject modules that use features the KWasm semantics AST cannot represent (multi-memory, memory64/table64, non-funcref table imports) instead of silently discarding information — see the Spec Divergences section below
• Enable all binary parsing integration tests for the new parser
• Add unit tests for individual parsing components
• Removed the unused BlockMetaData field, which was previously used for coverage tracking
• Implement differential testing to compare the old py-wasm-based parser with the new parser

Spec Divergences in the Binary Parser

The parser follows the current WebAssembly spec (WASM 3.0) for decoding, but the K semantics predates several additions. Where the K AST cannot represent a feature, the parser rejects the module with a clear error rather than silently discarding information; where information is discarded, it is semantically inert (noted per entry below).

Blocktype type-index form (instructions.py) — A blocktype can be a type index encoded as i33. Only the empty and single-valtype cases are handled; the i33 case raises a parse error. The K semantics block types cannot reference the type section.

Typed select (instructions.py) — The typed select variant (0x1C) carries a value-type vector. It is parsed and discarded; the untyped SELECT node is emitted. The K semantics has a single untyped select.

Memory load/store memarg (instructions.py) — memarg encodes an alignment hint and an optional memory index alongside the offset. Only the offset is forwarded to the AST; alignment is discarded (it is a hint with no runtime semantics). Affects all 23 load/store opcodes. The K semantics models a single memory with offset-only instructions.

Multi-memory (indices.py, memidx) — The K semantics models a single memory; multi-memory support is future work. A non-zero memory index anywhere (load/store memarg, memory.size/memory.grow/memory.copy/memory.fill, active data segments, memory exports) raises a parse error rather than being silently treated as memory 0. Memory index 0 is parsed and discarded where the K semantics has no slot for it.

Recursive types (module.py) — The spec wraps all type definitions in rectype groups that can be mutually recursive. We only accept singleton groups; actual recursion raises a parse error. The K semantics has no recursive type construct.

Table inline initializer (module.py) — The spec allows an optional inline initializer for tables (prefix 0x40 0x00). Encountering it raises a parse error. No counterpart exists in the K semantics.

Address type in limits (types.py, limits) — Limit encodings 0x04/0x05 signal 64-bit (memory64/table64) addressing, which the K semantics cannot represent: it assumes 32-bit addressing throughout. They raise a parse error rather than being silently treated as 32-bit. The shared-memory encodings 0x02/0x03 (threads proposal) likewise raise a parse error, since the K semantics has no notion of shared memory.

Only two reference types are supported: funcref and externref (types.py) — The GC proposal added further reference types (any, eq, i31, struct, array, none, noextern, nofunc, exn, noexn) and a general 0x63/0x64 encoding for (ref null? <heap type>). None of these are recognized; only funcref (0x70) and externref (0x6F) parse. v128 (SIMD) is likewise unrecognized wherever a value type is expected. The K semantics predates the GC and SIMD proposals and has no construct for any of these types.

NaN payloads (floats.py) — f32.const/f64.const are decoded via Python's struct.unpack, which collapses distinct NaN bit patterns (sign, payload, canonical vs. arithmetic) into a single Python float. This is not introduced by the binary parser: the K semantics' Float sort has no representation for NaN payloads either (see the TODO in wasm.md on the Float production).

Import section externtype (types.py) — The K ImportDefn sort still has the WASM 1.0 structure, so import descriptors are decoded from the WASM 3.0 grammar but returned as WASM 1.0 nodes. Two kinds of import cannot be represented in that structure and raise a parse error: tag imports (0x04), and table imports with a reference type other than funcref.

[ehildenb]

Wasm 3.0 compliance review — binary parser (PR #754)

Review of the in-repo binary parser in pykwasm/src/pykwasm/binary/ against the WebAssembly 3.0 binary-format specification.

The PR body declares a set of intentional divergences (typed select, blocktype i33, memarg alignment/memidx discard, recursive types, table inline initializer, address-type discard, tag imports).
Those all check out and are excluded here.
The findings below are bugs or gaps beyond what the PR body declares.

Summary

#	Severity	Location	Issue
1	High	binary/types.py:118	limits address-type flag bytes are wrong (0x02/0x03 vs spec 0x04/0x05)
2	High	binary/module.py:44	Custom sections are not skipped — stream is left misaligned
3	Medium	binary/instructions.py:146	0xFC sub-opcodes 0–11 missing (trunc_sat, memory.init/copy/fill, data.drop)
4	Low	binary/types.py:90,197	reftype/heaptype only decode the 2.0 subset (undeclared)
5	Low	binary/floats.py:12	NaN bit patterns may not round-trip

Verified correct while reviewing: signed/unsigned LEB128 range logic, tabletype field order (reftype-then-limits), call_indirect operand order, elem/data segment variant tables, section ordering (datacount before code), and sized exact-length enforcement.

Finding 1 — limits address-type flag bytes (High)

Location: binary/types.py:118-135

The Wasm 3.0 binary grammar (memory64) encodes the address type in the limits flag byte as:

limits ::= 0x00 n:u64        ⇒ (i32, [n .. ε])
         | 0x01 n:u64 m:u64  ⇒ (i32, [n .. m])
         | 0x04 n:u64        ⇒ (i64, [n .. ε])
         | 0x05 n:u64 m:u64  ⇒ (i64, [n .. m])

The parser uses 0x02/0x03 for the i64 (64-bit) cases and never handles 0x04/0x05.
Consequences:

• A valid memory64/table64 module (flag 0x04/0x05) raises Invalid limit descriptor — a false rejection of spec-compliant input.
• 0x02/0x03 are not i64 in the core spec (they're the shared-memory/threads encodings); accepting them as i64 mis-decodes.

Since the address type is discarded downstream anyway, the fix is just to relabel the flag values: 0x00/0x01 → i32, 0x04/0x05 → i64 (and reject 0x02/0x03 unless shared memory is intended to be accepted-and-discarded).

Finding 2 — custom sections not consumed (High)

Location: binary/module.py:44-63

def parse_custom_section(s):
    try:
        n = peek_byte(s)
    except WasmEOFError:
        return b''
    if n != 0:
        return b''
    skip(1, s)
    return read_bytes(n, s)   # n is 0 here!

When a custom section is present, n == 0 (the section id).
The code skips the 1 id byte, then calls read_bytes(0, s), reading nothing — it never reads the section's size:u32 or its payload.
It returns b'' (falsy), so parse_custom_sections immediately breaks, leaving the stream positioned right after the id byte, before the size.

Every subsequent section(...) call then peeks the wrong byte, doesn't match its expected id, and silently returns its default — so any module containing a custom section anywhere (name section, producers section, DWARF/debug info, etc.) drops everything after that section.
Test modules produced by wat2wasm without --debug-names have no custom sections, which is likely why differential testing passes.

A custom section is 0x00 size:u32 name:name bytes*.
The fix is to read the size after skipping the id and consume that many bytes:

skip(1, s)
size = u32(s)
return read_bytes(size, s)

Finding 3 — missing 0xFC opcodes (Medium)

Location: binary/instructions.py:146-165

The 0xFC handler only covers sub-opcodes 12–17 (table.*).
Missing sub-opcodes:

• 0–7: i32/i64.trunc_sat_{s,u}_{f32,f64}
• 8: memory.init, 9: data.drop, 10: memory.copy, 11: memory.fill

These fall through to case _ → Unsupported opcode.
Notably, memory.copy and memory.fill are already implemented in the semantics (aCopy/aFill, see wasm.md) and reachable via the text frontend, so the binary parser can't reach a capability the semantics already has.
Given the PR's stated motivation (unblocking Wasm 2.0+ opcodes), these core 2.0 instructions seem in scope.

When adding them, remember the trailing immediates must be consumed: memory.fill is 0xFC 11 0x00, memory.copy is 0xFC 10 0x00 0x00, memory.init is 0xFC 8 x:dataidx 0x00, data.drop is 0xFC 9 x:dataidx.

If trunc_sat / memory.init / data.drop are intentionally out of scope, they belong in the "Spec Divergences" list in the PR body alongside the others.

Finding 4 — reftype/heaptype subset (Low)

Location: binary/types.py:90-97, 197-205

reftype accepts only 0x70 (funcref) / 0x6F (externref) and heaptype only 0x70/0x6F.
This is a reasonable scope limit for a pre-GC semantics, but the 3.0 grammar also has the 0x63 ht/0x64 ht prefixed (ref null? ht) forms and the other abstract heap types (any, eq, i31, struct, array, none, noextern, nofunc, exn, noexn).
Anything using those (e.g. ref.null any) raises.
Same story for vectype/v128 in valtype (line 73, # TODO implement vectypes).
Suggest adding these to the PR's "Spec Divergences" list so the scope is explicit.

Finding 5 — NaN payloads (Low)

Location: binary/floats.py:12-21

struct.unpack('<f'/'<d', ...) yields a Python float, which collapses distinct NaN encodings (sign bit, canonical vs. arithmetic, payload bits).
If the downstream K f32.const/f64.const representation doesn't preserve the raw bits, const instructions with non-canonical NaNs won't match the spec's NaN semantics.
Worth a quick check that a f64.const NaN payload survives parse → K → execution; may be pre-existing behavior inherited from the old parser.

Minor note — memarg flag bit

Location: binary/instructions.py:51-61

memarg detects the memidx flag with a range check (n < 2**6 / elif n < 2**7) rather than testing bit 6 (n & 0x40) and masking the alignment.
For all valid modules the alignment exponent is tiny, so this never triggers a spurious error — noted only for completeness.

[bbyalcinkaya]

Wasm 3.0 compliance review — binary parser (PR #754)

Review of the in-repo binary parser in pykwasm/src/pykwasm/binary/ against the WebAssembly 3.0 binary-format specification.

...

Went through each finding:

• 1 (limits address-type flags) — fixed in 70ae762, with regression tests. Relabeled to 0x04/0x05 for i64 per spec; 0x02/0x03 (shared memory) now correctly rejected instead of misdecoded.
• 2 (custom sections not consumed) — fixed in cab1047, with a regression test. Was reading 0 bytes instead of the section's actual size, misaligning everything after it.
• 3 (missing 0xFC opcodes) — memory.copy/memory.fill now implemented in 072a674, with tests. trunc_sat/memory.init/data.drop remain unimplemented since they have no backing K construct at all.
• 4 (reftype/heaptype subset) — no change. Only funcref/externref supported in the semantics; GC ref types and SIMD v128 unsupported since the K semantics predates both proposals.
• 5 (NaN payloads) — no change. Pre-existing K semantics limitation (not introduced by this parser), treating as an accepted corner case.
• memarg flag-bit note — no change. Checked the actual spec grammar and the existing range-check code the definition in the spec, which doesn't use bitwise comparison.

[bbyalcinkaya]

Fixed after self review:

• No end-of-stream check in parse_module: trailing/garbage bytes, or any unrecognized section, were silently ignored rather than rejected. parse_module now raises WasmParseError if any bytes remain after the last section (binary/module.py), with regression tests in TestTrailingData (test_binary_parser.py).
• Dead code in binary/combinators.py: iterate was defined but never called anywhere in the parser.
• parse_module raised a bare Python ValueError (from zip(..., strict=True)) on func/code section length mismatch instead of the parser's own WasmParseError. Now checked explicitly before the zip, with regression tests in TestFuncCodeLengthMismatch (test_binary_parser.py).
• if without an else branch failed to parse
• peek_bytes did not restore the stream position on a partial-EOF read
  -Non-zero memory indices are rejected instead of silently treated as memory 0. Applies everywhere a memidx appears. Previously a valid multi-memory module parsed cleanly and then executed against the wrong memory.
• 64-bit addressing (memory64/table64) and non-funcref table imports are rejected. Both were decoded and silently discarded; neither is representable in the K AST. The spec-correct handling is kept in place as commented-out skeleton for future support.
• Error reporting and exception hygiene:
  • magic/version checks no longer rely on assert (stripped under python -O) and report expected vs. actual bytes
  • invalid UTF-8 in a name raises WasmParseError instead of leaking UnicodeDecodeError
  • the blocktype error names the unsupported type-index (i33) form
  • the 0xFC handler catches WasmParseError instead of Exception.
  • No accept/reject behavior changes.
• Minor:
  • clearer bytecode comments in the parser unit tests
  • the test-only recursion limit is scoped into run_module with a measured value instead of a module-level sys.setrecursionlimit(1500000000).