fix(parquet): bound schema nesting depth; prevent negative array offsets (FB-3438) - #44
Merged
Merged
Conversation
A Parquet schema is a flat list of SchemaElements whose num_children fields describe the tree, so a file can nest as deep as it has elements. schema::Unflatten rebuilds that tree with a recursive lambda and no depth guard, running one native frame per level. A schema a few tens of thousands deep overflows the thread stack while the metadata is being parsed, before any consumer sees a single row: an unauthenticated remote crash on any read of an attacker-supplied Parquet file. Reject schemas nesting deeper than kMaxSchemaNestingDepth (1000) by threading a depth counter through NextNode. 1000 matches the JSON reader's kMaxNestingDepth so every untrusted-nesting path in the library fails at the same number, and it is far above any schema a real writer produces (Parquet's own 3-level LIST encoding spends 2 elements per array level). The file that motivated this is a metadata-only Parquet file (no row groups) whose schema nests 100000 groups: pre-patch it SIGSEGVs inside Unflatten; post-patch it raises "Parquet schema nesting depth exceeds the maximum of 1000". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012wER7pGuTzevy6j4DS7hcL
ValidateLayout checks a negative length but not a negative offset, and the two are not equivalent. Every buffer-size check in the function sizes buffers by length_plus_offset; a negative offset shrinks that, so a buffer far too small for the array satisfies all of them. ArrayData::GetValues() then hands out buffers[i]->data() + offset, a pointer before the start of the buffer, and every consumer that trusts Validate() reads out of bounds. Nothing in the columnar format permits a negative offset, but an ArrowArray arriving over the C data interface carries whatever its producer wrote: ArrayImporter::AllocateArrayData copies c_struct_->offset into ArrayData unchecked, and the importer's own buffer-size arithmetic is wrong in the same direction. Validate() is the boundary where an imported array is supposed to become trustworthy, so the check belongs here. Two lines next to the existing length check; recursion into child_data means nested arrays are covered too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012wER7pGuTzevy6j4DS7hcL
tobias-fire
approved these changes
Aug 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent stack/bounds gaps on untrusted-input paths, both found while auditing Firebolt's Arrow column converter.
schema::Unflatten: unbounded recursionA Parquet schema is a flat list of
SchemaElements whosenum_childrenfields describe the tree, so a file can nest as deep as it has elements.Unflattenrebuilds that tree with a recursive lambda and no depth guard, one native frame per level. A schema a few tens of thousands deep overflows the thread stack while the metadata is parsed, before any consumer sees a row — an unauthenticated remote crash on any read of an attacker-supplied file.Threads a depth counter through
NextNodeand rejects pastkMaxSchemaNestingDepth(1000), matching the JSON reader'skMaxNestingDepth. Far above anything a real writer emits; Parquet's 3-level LIST encoding spends 2 elements per array level.Repro: a metadata-only file (no row groups) nesting 100000 groups. Pre-patch, SIGSEGV inside
Unflatten. Post-patch,Parquet schema nesting depth exceeds the maximum of 1000.ValidateLayout: negative array offsetValidateLayoutrejects a negativelengthbut not a negativeoffset, and the two are not equivalent. Every buffer-size check in the function sizes buffers bylength_plus_offset; a negative offset shrinks that, so a buffer far too small for the array satisfies all of them.ArrayData::GetValues()then returnsbuffers[i]->data() + offset— a pointer before the buffer — to every consumer that trustedValidate().Nothing in the format permits it, but an
ArrowArraycrossing the C data interface carries whatever its producer wrote:ArrayImporter::AllocateArrayDatacopiesc_struct_->offsetin unchecked, and the importer's own buffer sizing is wrong in the same direction.Validate()is where an imported array is meant to become trustworthy, so the check belongs there. Two lines beside the existing length check; the recursion intochild_datacovers nested arrays.