fix(orc): reject out-of-range DECIMAL precision/scale instead of aborting (FB-4175) - #48
Merged
lorenzhs merged 1 commit intoSep 16, 2026
Conversation
…ting (FB-4175) The ORC adapter's GetArrowType() reads a DECIMAL type's precision and scale verbatim from the (untrusted) ORC footer and forwards them to decimal128(). decimal128() constructs a Decimal128Type, whose constructor enforces 1 <= precision <= 38 with ARROW_CHECK_OK(ValidateDecimalPrecision<...>) -- which logs FATAL and aborts the process rather than returning a Status. A malformed footer declaring e.g. precision 48 therefore crashes any caller of ORCFileReader::ReadSchema() on otherwise well-formed input, a denial of service on untrusted ORC. Scale is not validated at all. Validate precision and scale in the DECIMAL case and return Status::TypeError on violation, keeping the precision == 0 legacy HIVE sentinel. The raw uint64 footer values are compared before the narrowing static_cast<int>, so a precision above INT_MAX cannot wrap into the valid range. GetArrowType() already returns Result<>, so the error propagates to ReadSchema() instead of aborting. Add a self-contained adapter_test.cc unit test driving GetArrowType() over createDecimalType() for precision > 38, precision > INT_MAX, scale > precision, the HIVE precision == 0 sentinel, and valid decimals including scale == precision. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
moshap-firebolt
force-pushed
the
moshap/fb-4175-orc-decimal-precision-scale
branch
from
September 16, 2026 15:33
dcc0755 to
f4c2f2b
Compare
Collaborator
|
CI is broken due to minio mirror retirement, should fix but not caused by this PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
arrow::adapters::orc::GetArrowType()'sDECIMALcase reads a type's precision and scale verbatim from the (untrusted) ORC footer and forwards them todecimal128():decimal128()constructs aDecimal128Type, whose constructor enforces1 <= precision <= 38viaARROW_CHECK_OK(ValidateDecimalPrecision<Decimal128Type>(precision))— which logsFATALand callsstd::abort()rather than returning aStatus. A malformed footer declaring e.g. precision 48 therefore crashes the process in any caller ofORCFileReader::ReadSchema()on otherwise well-formed input — a denial of service on untrusted ORC. Scale is not validated at all.Found by a downstream (Firebolt) ORC schema fuzzer; reproduces against vanilla
arrowon a 251-byte file.Fix
Validate precision and scale in the
DECIMALcase and returnStatus::TypeErroron violation, keeping theprecision == 0legacy HIVE sentinel. The rawuint64footer values are compared before the narrowingstatic_cast<int>, so a precision aboveINT_MAXcannot wrap into the valid range.GetArrowType()already returnsResult<>, so the error propagates toReadSchema()instead of aborting.Test
adapter_test.ccgainsGetArrowTypeRejectsOutOfRangeDecimal, drivingGetArrowType()overcreateDecimalType()for: precision > 38, precision >INT_MAX, scale > precision, theprecision == 0sentinel, and valid decimals includingscale == precision.Note
Medium Risk
Changes untrusted-input handling on ORC schema read; behavior shifts from process abort to TypeError for bad decimals, with low risk to valid files.
Overview
Fixes a denial-of-service when reading untrusted ORC files: malformed DECIMAL metadata in the footer no longer crashes the process.
GetArrowType()’s DECIMAL branch used to narrow footer precision/scale tointand calldecimal128()directly, which aborts on invalid precision viaARROW_CHECK. The change validates rawuint64precision and scale first—rejecting precision above 38 (including values aboveINT_MAXthat could wrap after cast), rejecting scale greater than precision, and preserving the legacy Hiveprecision == 0→decimal128(38, 6)mapping—then returnsStatus::TypeErrorso schema read fails cleanly.Adds
GetArrowTypeRejectsOutOfRangeDecimalinadapter_test.ccfor invalid and valid decimal type conversions.Reviewed by Cursor Bugbot for commit f4c2f2b. Bugbot is set up for automated code reviews on this repo. Configure here.