Register compression schemes on the session by mhk197 · Pull Request #10034 · vortex-data/vortex

mhk197 · 2026-09-24T19:18:47Z

Summary

Compression schemes become a session registry, like array encodings and layouts, and every BtrBlocks compressor is built from a session. The compressor keeps the registered schemes whose serialized IDs the session's enabled editions permit, so a default compressor can no longer emit an encoding the edition-enforcing writer rejects. BtrBlocksCompressorBuilder and the session-less BtrBlocksCompressor::default() are removed; nothing takes a scheme list any more.

Model

Registry = availability. CompressionSession holds the schemes a session can compress with, in registration order. It is add-only.
Plugins and editions = permission. Which registered schemes a compressor may use is decided at construction from each scheme's produced_encodings(): every encoding must have a registered array plugin and, when editions are enforced, be permitted by the enabled editions. This is the same rule the file writer applies to the arrays it serializes.
Policy = a session. Wanting a different scheme set means configuring a session, not filtering a list at the call site: install a standard registry (default, compact, cuda, empty), register more schemes on it, or build a compressor with BtrBlocksOptions.

Scheme registry (`vortex-btrblocks`)

session::CompressionSession is a session variable, like LayoutSession. Its Default carries the built-in scheme set, which was ALL_SCHEMES.

Constructor	Schemes
`CompressionSession::default()`	the built-in set, in tie-break order
`CompressionSession::compact()`	the defaults plus Zstd for strings and binary and Pco for numerics, each behind its feature
`CompressionSession::cuda()`	the defaults CUDA kernels decode, keeping FSST, plus array-level and buffer-level binary Zstd
`CompressionSession::empty()`	none

register(&mut self, &'static dyn Scheme) deduplicates by SchemeId; schemes() reads the list. The scheme lists themselves are private. DELTA_SCHEME stays public because Delta is opt-in and not part of any edition.

CompressionSessionExt gives session access: compression() for the registry and register_scheme.

VortexSession::default() registers CompressionSession; sessions built from array_session() get the default registry lazily on first use. A session that already has one swaps it with session.register(CompressionSession::cuda()), as vortex_arrow::initialize replaces ArrowSession.

Compressor API (`vortex-btrblocks`)

BtrBlocksCompressor stays the thin wrapper over CascadingCompressor; vortex-compressor is unchanged.

pub struct BtrBlocksOptions {
    pub enforce_editions: bool,        // default true
    pub exclude_schemes: Vec<SchemeId>, // default empty
}

impl BtrBlocksCompressor {
    pub fn new(schemes: Vec<&'static dyn Scheme>) -> Self;
    pub fn empty() -> Self;
    pub fn from_session(session: &VortexSession) -> Self;
    pub fn from_session_with_options(session: &VortexSession, options: &BtrBlocksOptions) -> Self;
}

from_session keeps the registered schemes whose encodings the session can serialize and the enabled editions permit. enforce_editions: false drops only the edition check, for in-memory compression; exclude_schemes leaves schemes out. new takes a scheme list as given, for tests and callers that own the list.

Removed: BtrBlocksCompressorBuilder (empty, with_new_scheme, with_compact, only_cuda_compatible, exclude_schemes, retain_allowed_encodings, build), BtrBlocksCompressor::default(), ALL_SCHEMES.

File writer (`vortex-file`)

WriteStrategyBuilder::from_session(&session) replaces default(), and from_session_with_options(&session, options) replaces with_btrblocks_builder. The builder holds the session and the options, and at build creates the data compressor with IntDictScheme added to exclude_schemes, because the dict layout already dictionary-encodes columns, and the stats compressor with the options as given. with_compressor still installs an opaque CompressorPlugin used as-is for both.

The writer's default strategy is from_session_with_options with enforce_editions set to whether editions are enforced, so disable_editions() still writes every registered encoding.

CUDA (`vortex-cuda`)

CompressionSession::cuda() replaces the only_cuda_compatible preset. Sessions that write CUDA files install it next to register_cuda_layout: the CLI, vx_cuda_session_new, the CUDA sink, the write-strategy tests, and compress-bench's GPU backend, which gets its own GPU_SESSION so the host backends keep the default schemes. Read-only CUDA sessions are untouched. cuda_write_strategy is WriteStrategyBuilder::from_session, with BtrBlocksCompressor::from_session as the opaque compressor and BtrBlocksCompressor::empty() as the probe in the block-rows path.

Callers

Compact writes come from a session holding CompressionSession::compact(): Python's fork-safe compact_session() behind VortexWriteOptions.compact(), vortex-bench's COMPACT_SESSION via CompactionStrategy::session(), a compact session built with the caller's runtime handle in the TUI, the fuzz COMPACT_SESSION, compat-gen, and the tracing example.
Subsets use exclude_schemes: the golden snapshots without OnPair, the varbin and StringDict tests, and string-bench's forced encoder. Tests that want a single scheme start from CompressionSession::empty().
Scheme-selection tests build compressors from explicit scheme lists with BtrBlocksCompressor::new; the golden snapshots, which pin the edition-filtered set, register the encoding plugins on their session.
The facade's vortex::compressor module exports BtrBlocksCompressor, BtrBlocksOptions, CompressionSession, CompressionSessionExt, Scheme, SchemeId.

Behavior

An explicit WriteStrategyBuilder::from_session(&session).build() only emits permitted encodings. The edition test that previously asserted such a strategy emits vortex.sequence and is rejected now asserts the write succeeds.
Zstd and Pco are in the core edition, so they are deliberately not registered by default; compact remains an explicit session choice.
Python's vx.compress is restricted to the session's enabled editions instead of every scheme.
Opening a CUDA sink through the FFI on a plain session replaces that session's schemes with the CUDA set, as vx_cuda_session_new does at creation.
The strategy builder reads the session's registry at build, not at from_session.
A session whose editions enable an encoding it never registered, such as array_session() with the core edition but without register_default_encodings, writes successfully: the compressor skips schemes whose output the writer could not serialize. write_uses_only_registered_encodings pins this.

Validation

Run locally on macOS (arm64): cargo +nightly-2026-09-10 fmt on the touched crates; cargo clippy --all-targets --all-features -- -D warnings on vortex-compressor, vortex-btrblocks, vortex-file, vortex-layout, vortex, vortex-cuda, gpu-scan-cli, compress-bench, vortex-python, vortex-tui, vortex-bench, vortex-compat, string-bench, duckdb-bench; vortex-btrblocks also without features and with pco and zstd alone; vortex with zstd but not files; vortex-fuzz with all features; all clean. cargo nextest run --all-features on vortex-btrblocks, vortex-file, vortex-layout, vortex: 543 passed, 1 skipped; vortex-btrblocks without features: 61 passed; string-bench: 14 passed; the two cuda_write_strategy tests pass. Doctests on vortex-btrblocks, vortex-file, vortex: passed. cargo check -p vortex-cuda-ffi (lib): clean.

Not run: GPU tests and wasm-test, which need a device or a wasm target, and the Python test suite, which needs a Maturin rebuild.

codspeed · 2026-09-24T19:26:25Z

Merging this PR will not alter performance

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️

3 benchmarks measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. These results are not comparable, so they count as unchanged.

Preventing compiler optimizations

✅ 2222 untouched benchmarks
⏩ 343 skipped benchmarks¹
🗄️ 1 archived benchmark run²

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚠️	Simulation	`fixed_16_advancing_ptr_safe[100]`	< 1 ns	< 1 ns	N/A
⚠️	Simulation	`preverify_advancing_ptr_unchecked[1000]`	< 1 ns	< 1 ns	N/A
⚠️	Simulation	`preverify_advancing_ptr_unchecked[10000]`	< 1 ns	< 1 ns	N/A

_{Comparing mk/scheme-registry (c6fc22e) with develop (38fa7e3)}

343 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
1 benchmark was run, but is now archived. If it was deleted in another branch, consider rebasing to remove it from the report. Instead if it was added back, click here to restore it. ↩

connortsui20

I feel like we are mixing up too many things in this PR / there is some other work that needs to be done prior to this.

The most obvious thing is that we really should have a cuda family edition instead of forcing the compressor to hardcode this decision.

Additionally, my understanding of the issue with the current system is that we didn't want this BtrBlocksCompressorBuilder instantiation with all of the extra builder methods (like only_cuda_compatible and with_compact) all over the place, and I don't think that this PR solves this (in some cases I think it makes this more unwieldy).

And the additional code smell is that we do not remove that big static list of default schemes, which I thought we wanted to get rid of. Even if we don't do that, the registration, selection, and compatibility checks for all of this seem to still have overlapping owners, which is not ideal.

I am going to write up an issue detailing how I think we should do it, and we should discuss implementation details there.

robert3005 · 2026-09-24T20:40:46Z

+/// Delta, kept out of [`DEFAULT_SCHEMES`](crate::DEFAULT_SCHEMES) because it is slower to
+/// decompress than the schemes that would otherwise win. Callers that want it opt in with
+/// [`with_new_scheme`](BtrBlocksCompressorBuilder::with_new_scheme) and permit `fastlanes.delta`.
 ///
-/// This list is order-sensitive: the builder preserves this order when constructing
-/// the final scheme list, so that tie-breaking is deterministic.
-pub const ALL_SCHEMES: &[&dyn Scheme] = &[
-    ////////////////////////////////////////////////////////////////////////////////////////////////
-    // Integer schemes.
-    ////////////////////////////////////////////////////////////////////////////////////////////////
-    // NOTE: FoR must precede BitPacking to avoid unnecessary patches.
-    &integer::FoRScheme,
-    // NOTE: ZigZag should precede BitPacking because we don't want negative numbers.
-    &integer::ZigZagScheme,
-    &integer::BitPackingScheme,
-    &integer::SparseScheme,
-    &integer::IntDictScheme,
-    &integer::RunEndScheme,
-    &integer::SequenceScheme,
-    &integer::IntRLEScheme,
-    // Delta is omitted here: see [`DELTA_SCHEME`].
-    ////////////////////////////////////////////////////////////////////////////////////////////////
-    // Float schemes.
-    ////////////////////////////////////////////////////////////////////////////////////////////////
-    &float::ALPScheme,
-    &float::ALPRDScheme,
-    &float::FloatDictScheme,
-    &float::NullDominatedSparseScheme,
-    &float::FloatRLEScheme,
-    ////////////////////////////////////////////////////////////////////////////////////////////////
-    // String schemes.
-    ////////////////////////////////////////////////////////////////////////////////////////////////
-    &string::StringDictScheme,
-    // Both string-fragmentation schemes are registered; the sample-based
-    // selector keeps whichever is smaller per column.
-    &string::FSSTScheme,
-    &string::OnPairScheme,
-    &string::NullDominatedSparseScheme,
-    ////////////////////////////////////////////////////////////////////////////////////////////////
-    // Binary schemes.
-    ////////////////////////////////////////////////////////////////////////////////////////////////
-    &binary::BinaryDictScheme,
-    &binary::VarBinScheme,
-    // Decimal schemes.
-    &decimal::DecimalScheme,
-    // Temporal schemes.
-    &temporal::TemporalScheme,
-];
-
-/// Delta, kept out of [`ALL_SCHEMES`] because it is slower to decompress than the schemes that
-/// would otherwise win. Callers that want it opt in with
-/// [`with_new_scheme`](BtrBlocksCompressorBuilder::with_new_scheme).
-///
-/// TODO(robert): Return it to [`ALL_SCHEMES`] once we have scheme filtering.
+/// TODO(robert): Return it to [`DEFAULT_SCHEMES`](crate::DEFAULT_SCHEMES) once we have scheme
+/// filtering.


with this change DELTA should come back to default schemes since it will be filtered out

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

…sion Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

…build time Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

connortsui20

This is much, much better, and at least to me this feels like a more elegant solution (or at the very least it is better than what we had before...)

I think there are a few implementation things that can be improved, as well as some followups that we probably want to get started on immediately

connortsui20 · 2026-09-25T02:32:51Z

+    /// Schemes to leave out.
+    pub exclude_schemes: Vec<SchemeId>,


Do we really need this? I'm pretty sure we don't want 2 places that decide what schemes we "want" (to use the terminology from #10038), and at least in this PR I don't think this is strictly necessary either?

Basically this is caller convenience, but it creates a leaky abstraction.

See

vortex/vortex-file/src/strategy.rs

Lines 180 to 185 in 133aacd

CompressorConfig::BtrBlocks(builder) => Arc::new(

builder

.clone()

.exclude_schemes([IntDictScheme.id()])

.build(),

),

and #10038 (comment), there is a single place that we need to exclude internally, but I still don't think that it should be a public feature

We can just iterate through session.permitted_schemes() and filter out this scheme explicitly instead if preferred

connortsui20 · 2026-09-25T02:35:04Z

+    /// Creates a compressor over the schemes registered on `session` whose serialized IDs the
+    /// session's enabled editions permit.
+    pub fn from_session(session: &VortexSession) -> Self {
+        Self::from_session_with_options(session, &BtrBlocksOptions::default())
+    }


~~I personally would prefer from_session to take options no matter what (and then the caller must provide the options) but you can decide.~~

Edit: nvm I proposed a different API entirely without the options

connortsui20 · 2026-09-25T02:37:17Z

+    /// Keep only the registered schemes whose serialized IDs the session's enabled editions
+    /// permit, which a file writer requires. Off, every registered scheme is used, for in-memory
+    /// compression where no edition applies.
+    pub enforce_editions: bool,


And even this doesn't seem super necessary? (see below)

connortsui20 · 2026-09-25T02:38:40Z

+    /// Creates a compressor over the schemes registered on `session`, per `options`.
+    pub fn from_session_with_options(session: &VortexSession, options: &BtrBlocksOptions) -> Self {
+        let mut schemes = if options.enforce_editions {
+            session.permitted_schemes()
+        } else {
+            session.registered_schemes()
+        };
+        schemes.retain(|scheme| !options.exclude_schemes.contains(&scheme.id()));
+        Self(CascadingCompressor::new(schemes))
+    }


A potentially more consise API:

impl BtrBlocksCompressor { /// Registered schemes the enabled editions permit (for writes). This is the /// "correct" path pub fn from_session(session: &VortexSession) -> Self; /// Exactly these schemes like `session.registered_schemes()`. pub fn new(schemes: Vec<&'static dyn Scheme>) -> Self; }

So that the caller is forced to either use editions (as it is intended) or must provide it themselves (which should be easy via session.registered_schemes() like above), which should work fine for in-memory tests

Options might be necessary for now so we can support choosing compressor mode after initializing session

(mode is not in options now but adding it)

Modes here will essentially be filters, e.g. default will filter out compact.

It would be nice to remove modes in general (and just to rely on what users register in session) but there are some places where we alter compressor state after initializing session:

Python write uses compact based on a flag

Conversion for tui uses compact based on a flag

I'm assuming we can break the first but not sure about the second?

connortsui20 · 2026-09-25T02:40:31Z

+/// Delta, kept out of the default schemes because it is slower to decompress than the schemes
+/// that would otherwise win. Callers that want it register it and permit `fastlanes.delta`.
+///
+/// TODO(robert): Return it to the defaults once we have scheme filtering.
+pub static DELTA_SCHEME: integer::DeltaScheme = integer::DeltaScheme::new(1.25);


We should be able to move this back now that it will get filtered out by the editions filter?

yes that can be a followup

connortsui20 · 2026-09-25T02:50:42Z

+    /// The default schemes that CUDA kernels decode, keeping FSST for string compression, plus
+    /// Zstd for binary compression when the `zstd` feature is on.
+    ///
+    /// Both the array-level and the buffer-level Zstd schemes are added. Buffer-level compression
+    /// preserves binary arrays' buffer layout for zero-conversion GPU decompression, but belongs
+    /// to the opt-in `zstd` edition, so a session's enabled editions decide which of the two a
+    /// writer uses. Files written with these schemes may be larger than with the defaults: the
+    /// set picks encodings the GPU decodes, not the smallest ones.
+    pub fn cuda() -> Self {
+        // Keep FSST, which has a CUDA decoder and direct Arrow offset-based export. Other string
+        // fragmentation and dictionary schemes still require unsupported decode paths. Delta is
+        // not a default either: it has a CUDA decode kernel, but GPU delta decode has not been
+        // benchmarked against the schemes it would displace.


This really should become a cuda edition, though that can be a separate PR. CC @robert3005

connortsui20 · 2026-09-25T02:52:07Z

+    pub fn compact() -> Self {
+        let mut this = Self::default();
+        for scheme in COMPACT_SCHEMES {
+            this.register(*scheme);
+        }
+        this
+    }


I feel like instead of doing this, we should split the vortex_zstd::initialize(&SESSION) into that PLUS something like vortex_zstd::register_schemes(&SESSION) so that it is easy to split read (initialize) and write (register schemes) concerns. What do you think?

Oh this would mean that the schemes would need to move to their owning crates, but I think that is fine? And actually I dont think it is actually incorrect for schemes to live next to their encodings UNLESS they have exclusion rules relating each other... Something is weird here but that can be figured out after this PR

We can do that but would have to figure out ordering as well, could be followup

I don't think we would need to figure out ordering here, at least for the compact schemes they don't have any ordering constraints iirc, and this would solve the problem you mention at #10034 (comment)

I mean for all schemes, there are commented ordering constraints

connortsui20 · 2026-09-25T02:52:38Z

+            string::StringDictScheme.id(),
+            binary::BinaryDictScheme.id(),


Can GPU handle this now? (issue with previous code probably)

mhk197 · 2026-09-24T19:38:47Z

+///
+/// Compressors built with [`BtrBlocksCompressorBuilder::from_session`] start from the schemes
+/// registered on their session. Registration is idempotent, so this may run more than once.
+pub fn initialize(session: &VortexSession) {


Move this and ALL_SCHEMES to a session.rs file.

ALL_SCHEMES should be renamed DEFAULT_SCHEMES

mhk197 · 2026-09-25T03:11:59Z

+    /// Schemes to leave out.
+    pub exclude_schemes: Vec<SchemeId>,


We can just iterate through session.permitted_schemes() and filter out this scheme explicitly instead if preferred

mhk197 · 2026-09-25T03:38:56Z

+    /// Creates a compressor over the schemes registered on `session`, per `options`.
+    pub fn from_session_with_options(session: &VortexSession, options: &BtrBlocksOptions) -> Self {
+        let mut schemes = if options.enforce_editions {
+            session.permitted_schemes()
+        } else {
+            session.registered_schemes()
+        };
+        schemes.retain(|scheme| !options.exclude_schemes.contains(&scheme.id()));
+        Self(CascadingCompressor::new(schemes))
+    }


Options might be necessary for now so we can support choosing compressor mode after initializing session

mhk197 · 2026-09-25T03:39:13Z

+    /// Creates a compressor over the schemes registered on `session`, per `options`.
+    pub fn from_session_with_options(session: &VortexSession, options: &BtrBlocksOptions) -> Self {
+        let mut schemes = if options.enforce_editions {
+            session.permitted_schemes()
+        } else {
+            session.registered_schemes()
+        };
+        schemes.retain(|scheme| !options.exclude_schemes.contains(&scheme.id()));
+        Self(CascadingCompressor::new(schemes))
+    }


(mode is not in options now but adding it)

mhk197 · 2026-09-25T03:39:18Z

+/// Delta, kept out of the default schemes because it is slower to decompress than the schemes
+/// that would otherwise win. Callers that want it register it and permit `fastlanes.delta`.
+///
+/// TODO(robert): Return it to the defaults once we have scheme filtering.
+pub static DELTA_SCHEME: integer::DeltaScheme = integer::DeltaScheme::new(1.25);


yes that can be a followup

…ressor::new Signed-off-by: Matt Katz <mhkatz97@gmail.com>

joseph-isaacs · 2026-09-25T13:06:01Z

+/// the compression schemes the GPU decodes. Separate from vortex-bench's `SESSION`, so the host
+/// backends keep the default schemes.
+static GPU_SESSION: LazyLock<VortexSession> = LazyLock::new(|| {
+    let session = VortexSession::default().with_tokio();


This is not correct!

This needs to be injected no?

what do you mean injected?

joseph-isaacs · 2026-09-25T13:06:25Z

+/// the compression schemes the GPU decodes. Separate from vortex-bench's `SESSION`, so the host
+/// backends keep the default schemes.
+static GPU_SESSION: LazyLock<VortexSession> = LazyLock::new(|| {
+    let session = VortexSession::default().with_tokio();


This needs to be injected no?

joseph-isaacs · 2026-09-25T13:07:35Z

+pub static COMPACT_SESSION: LazyLock<VortexSession> = LazyLock::new(|| {
+    let session = new_session();
+    session.register(CompressionSession::compact());
+    session
+});


should this be static?

mhk197 · 2026-09-25T14:14:44Z

        store: Option<AnyVortexStore>,
    ) -> PyVortexResult<()> {
-        let session = session();
+        let session = if self.use_compact_encodings {


@connortsui20 this is how we have to switch right now

joseph-isaacs · 2026-09-25T14:25:12Z

+fn default_compressor() -> BtrBlocksCompressor {
+    BtrBlocksCompressor::new(CompressionSession::default().schemes().to_vec())
+}


how did you get this session? this should be passed a session?

mhk197 force-pushed the mk/scheme-registry branch from d068a7a to 2fb17f9 Compare September 24, 2026 19:25

mhk197 added the changelog/break A breaking API change label Sep 24, 2026

mhk197 mentioned this pull request Sep 24, 2026

Upgrade DecimalScheme to the v2 wire format when the writer permits it #10035

Draft

mhk197 force-pushed the mk/scheme-registry branch from 2fb17f9 to 70b0ab1 Compare September 24, 2026 19:51

connortsui20 mentioned this pull request Sep 24, 2026

Tracking Issue: Scheme Design with Editions #10038

Open

mhk197 force-pushed the mk/scheme-registry branch from 70b0ab1 to 830c260 Compare September 24, 2026 20:02

connortsui20 requested changes Sep 24, 2026

View reviewed changes

mhk197 force-pushed the mk/scheme-registry branch 2 times, most recently from 80af2c5 to 70b0bfe Compare September 24, 2026 20:34

robert3005 reviewed Sep 24, 2026

View reviewed changes

connortsui20 mentioned this pull request Sep 24, 2026

Experiment: compose scheme registration with output editions #10040

Closed

mhk197 force-pushed the mk/scheme-registry branch from 70b0bfe to b9329d9 Compare September 24, 2026 22:05

Register compression schemes on the session

bca8e15

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

mhk197 force-pushed the mk/scheme-registry branch from b9329d9 to bca8e15 Compare September 24, 2026 22:59

mhk197 added 9 commits September 24, 2026 19:44

Default the scheme registry, drop initialize, configure CUDA sessions

d18f543

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Remove with_schemes: compact and no-editions writes configure the ses…

17805da

…sion Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Add CompressionSession::compact and ::cuda, hide the scheme lists

2f5d6cc

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Add BtrBlocksOptions, build strategy compressors from the session at …

54f3d9b

…build time Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Restore the facade zstd feature to develop's definition

ae0e6b5

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Fold the writer's default strategy arms into one

795d305

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Use exclude_schemes instead of rebuilding registries minus one scheme

09a9844

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Fix the CompressionSession::cuda doc link in vortex-cuda

3f28596

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

don't replace the caller's schemes in the cuda sink

033636e

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

connortsui20 requested changes Sep 25, 2026

View reviewed changes

mhk197 commented Sep 25, 2026

View reviewed changes

Only compress with encodings the session can write; add BtrBlocksComp…

c6fc22e

…ressor::new Signed-off-by: Matt Katz <mhkatz97@gmail.com>

mhk197 marked this pull request as ready for review September 25, 2026 04:38

joseph-isaacs reviewed Sep 25, 2026

View reviewed changes

mhk197 commented Sep 25, 2026

View reviewed changes

joseph-isaacs reviewed Sep 25, 2026

View reviewed changes

mhk197 closed this Sep 25, 2026

		/// Schemes to leave out.
		pub exclude_schemes: Vec<SchemeId>,

	CompressorConfig::BtrBlocks(builder) => Arc::new(
	builder
	.clone()
	.exclude_schemes([IntDictScheme.id()])
	.build(),
	),

		string::StringDictScheme.id(),
		binary::BinaryDictScheme.id(),

Conversation

mhk197 commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Model

Scheme registry (vortex-btrblocks)

Compressor API (vortex-btrblocks)

File writer (vortex-file)

CUDA (vortex-cuda)

Callers

Behavior

Validation

Uh oh!

codspeed Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Performance Changes

Footnotes

Uh oh!

connortsui20 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

connortsui20 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

connortsui20 Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

connortsui20 Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhk197 commented Sep 24, 2026 •

edited

Loading

Scheme registry (`vortex-btrblocks`)

Compressor API (`vortex-btrblocks`)

File writer (`vortex-file`)

CUDA (`vortex-cuda`)

codspeed Bot commented Sep 24, 2026 •

edited

Loading

connortsui20 Sep 25, 2026 •

edited

Loading

connortsui20 Sep 25, 2026 •

edited

Loading