fix(parquet): size a page from what its data says, not what its header claims (FB-4054)

[moshap-firebolt]

Found by PackDB's parquet_input fuzzer.

A page header declares its uncompressed size, and nothing checked it against the data present. SerializedPageReader::DecompressIfNeeded sized its output buffer straight from the file, so a 1358-byte input requested 2 GiB:

requested allocation size 0x80000000 exceeds maximum supported size of 0x40000000
  #4 arrow::PoolBuffer::Resize
  #5 parquet::SerializedPageReader::DecompressIfNeeded   column_reader.cc

Not memory-unsafe — the allocation succeeds or fails cleanly — but a small file that costs gigabytes is a denial of service on a reader that takes files from users, and this path is reachable from an external table.

Ask the data, don't guess

The first version of this PR bounded the declared size by an invented expansion ratio. That was rightly pushed back on in review: nobody can say what ratio is safe, and getting it wrong rejects valid files.

So instead: Codec gains DecompressedLength(), which snappy answers from its varint prefix and zstd from its frame header. Every other codec keeps the base implementation and is unaffected.

Three outcomes, and the third is what closes this:

result	meaning	action
a value	the data states its length	the header must agree
nullopt	the format records no length (lz4 raw, gzip)	nothing learned, carry on as before
an error	the format does record one, and this input is too damaged to read it	it cannot decompress at all, so don't allocate for it

The reproducer is the third case exactly — five bytes of noise declaring 2 GiB. zstd can't parse a frame header out of it, so the page is refused before the buffer is sized.

That distinction was found by measurement, not design. An earlier revision only compared lengths when the codec could report one, and the reproducer walked straight through:

DIAG compressed_len=20 uncompressed_len=11          codec_says=11    <- valid page, agrees
DIAG compressed_len=5  uncompressed_len=2147483600  codec_says=-1    <- attack, codec silent

Treating "codec cannot read this" as a rejection rather than a shrug is the whole fix.

Verification

reproducer (1358 bytes)	2 GiB abort → rejected before allocating
parquet fuzz corpus, strict allocator	11,773 executions, 0 artifacts, coverage 6249 → 6954
Parquet / codec / compression tests	1432 pass

Coverage still climbing is the part that matters: the check isn't quietly rejecting real files. Note the valid pages of the reproducer's own file still read — one states 11 bytes against a declared 11.

Same shape as the FB-3438 work (#44, #45).

---

Note

Medium Risk
Touches untrusted Parquet decompression and allocation sizing; behavior changes for malicious or mismatched headers, with intentional strictness for Snappy/Zstd.

Overview
Adds Codec::DecompressedLength so callers can read the decompressed size embedded in Snappy/Zstd payloads (default nullopt for codecs that do not record it).

Parquet SerializedPageReader::DecompressIfNeeded now queries that length on the compressed page bytes before resizing the decompression buffer: corrupt/unreadable compressed data fails fast instead of allocating from the header alone, and when the codec reports a size it must match the page header’s claimed uncompressed size (after level bytes). Corded-buffer and no-embedded-length codecs are unchanged.

^{Reviewed by Cursor Bugbot for commit d81fab7. Bugbot is set up for automated code reviews on this repo. Configure here.}

[moshap-firebolt]

Closing — the check does not do what it needs to.

Reviewing this properly: every quantity it relied on is attacker-controlled. ZSTD_getFrameContentSize reads the frame header, which is just bytes in the file, so ~14 bytes of crafted header makes the frame agree with the Parquet header and the check passes. It catches inconsistency between two numbers the attacker owns — which is why the fuzzer's random garbage tripped it and a deliberate attacker would not.

The earlier ratio version had the right anchor (compressed_len is verified against bytes actually read, so it is the one number the file cannot inflate) but no defensible constant, and even a generous ratio leaves real amplification: at 100,000x a 1 MB upload still commands 100 GB.

What actually bounds this is already in place. In production PackDB's MemoryTracker fails the query cleanly on budget; under the fuzzing build ASAN's allocator returns null and the reader reports a clean error. The exposure is a query wasting its own budget on a file that never contained the data — not a crash, and not unbounded.

So this is a wrong-severity finding rather than a missing check, and the right outcome is no change to Arrow. FB-4054 closed as won't-fix with the reasoning recorded there.