Conversation
Merging this PR will not alter performance
|
mhk197
added this pull request to stack #10070
September 25, 2026 20:37
mhk197
requested review from
connortsui20,
gatesn,
joseph-isaacs and
robert3005
and removed request for
connortsui20,
gatesn and
joseph-isaacs
September 25, 2026 20:38
connortsui20
force-pushed
the
mk/decimal-refine
branch
from
September 25, 2026 20:52
81e23a9 to
13cb081
Compare
mhk197
force-pushed
the
mk/decimal-refine
branch
from
September 25, 2026 21:06
13cb081 to
2683380
Compare
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
mhk197
force-pushed
the
mk/decimal-refine
branch
from
September 25, 2026 21:20
2683380 to
c8a8c3c
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #10069.
Summary
Wide
i128andi256decimals compress into multi-partDecimalBytePartsarrays when the compressor's allowed serialized IDs includevortex.decimal_byte_parts.v2. The session registers the v1 decimal scheme, andbuild()refines it to v2 wherever both decimal IDs are allowed.This PR sets the stage for versioned schemes.
Scheme::refineNew defaulted hook in
vortex-compressor:Nonekeeps the scheme.SchemeId, and every ID it declares inproduced_encodingsmust still be allowed for it to be used.empty(),unrestricted()) are represented exactly.Builder
build()now:The allowed IDs are the session's
AllowedIdsminus the mode's excluded serialized IDs. TheCudamode excludesvortex.decimal_byte_parts.v2, since multi-part arrays have no CUDA decode kernel, so the CUDA preset keeps v1.DecimalSchemeDecimalSchemegains v1 and v2 modes withv1()andv2()constructors;Defaultis v1. The session registers v1.i64canonical after narrowing. v2 splits them withsplit_decimaland compresses the most significant part and each lower part as children. Single-part arrays serialize as v1 in either mode.refinereturns v2 when both decimal IDs are allowed and v1 otherwise, from either mode. Unlike Upgrade DecimalScheme to the v2 wire format when the writer permits it #10035'stry_upgrade, a v2 scheme is refined down to v1 rather than dropped.Behaviour
from_sessionwith the core editionfrom_session(..).disable_editions(), including the writer withdisable_editions()unrestricted()/empty()with the v2 schemeonly_cuda_compatible()Default writes are unchanged: the core edition allows v1 only. Strategies built from
WriteStrategyBuilder::from_sessionkeep the enabled editions' restrictions even when a write disables editions.Compared with #10035,
vortex-file's writer andvortex-cudaneed no changes: the writer'sdisable_editionspath already lifts the edition restriction, and the CUDA restriction lives in theCudamode.Tests
vortex-btrblocks/src/schemes/decimal.rs:refinefor every combination of allowed decimal IDs, from both modes.vortex-btrblocks/tests/decimal_config.rs: decimal output for no editions, the core edition, editions disabled, unrestricted, and the CUDA preset; excluding the decimal scheme is not undone by refinement; signed/nullablei128/i256round trips with and without child compression.vortex-file: the default writer splits wide decimals only with editions disabled; strategies built from the session keep the enabled editions' restrictions.Checks
cargo clippy -D warningsonvortex-btrblocks,vortex-compressor,vortex-filewith--all-targets --all-features,--all-targets, and--no-default-features: passedcargo nextest run --all-features -p vortex-btrblocks -p vortex-compressor -p vortex-file -p vortex -p vortex-layout -p string-bench -p vortex-compat -p vortex-decimal-byte-parts: 988 passed, 1 skippedcargo test --doc -p vortex-btrblocks -p vortex-compressor: passedRUSTDOCFLAGS="-D warnings" cargo doc -p vortex-btrblocks -p vortex-compressor --no-deps --all-features: passedrustfmt +nightly-2026-09-10 --checkon touched files: passedvortex-cuda(needsnvcc), benchmarks, workspace-wide clippy and tests.