Experiment: compose scheme registration with output editions - #10040
connortsui20 wants to merge 3 commits into
Conversation
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
|
this is bad |
Merging this PR will regress 2 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | decode_avx512[8192, (Inline, OneNullInEight)] |
81.7 µs | 99.2 µs | -17.65% |
| ❌ | WallTime | decode_avx512[8192, (Inline, AllValid)] |
96.6 µs | 112.5 µs | -14.12% |
| ⚡ | Simulation | from_vec_drop_arrow[16384] |
40.4 µs | 31.4 µs | +28.7% |
| ⚡ | WallTime | filtered_sink_i64_avx512[OneNullInEight] |
26.3 µs | 22.3 µs | +18.26% |
| Simulation | fixed_16_advancing_ptr_safe[100] |
< 1 ns | < 1 ns | N/A | |
| Simulation | preverify_advancing_ptr_unchecked[1000] |
< 1 ns | < 1 ns | N/A | |
| Simulation | preverify_advancing_ptr_unchecked[10000] |
< 1 ns | < 1 ns | N/A |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing ct/composable-schemes (ffe45b2) with develop (035aeff)
Footnotes
-
343 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
-
1 benchmark was run, but is now archived. If it was deleted in another branch, consider rebasing to remove it from the report. Instead if it was added back, click here to restore it. ↩
Summary
Makes scheme registration the writer's opt-in to compression. Encoding packages own their schemes, and compressor construction resolves those registrations against the permitted serialized IDs before starting the cascading engine. Registering an encoding alone still supports reading without enabling its compression scheme.
Changes
vortex-btrblocksas a thin initialization and construction facade, with compatibility re-exports.Scoped Clippy passed with warnings denied for the affected core and encoding crates. Compiler checks passed for the affected core crates, including tests, benches, examples, all-feature compression builds, and the facade without file support. Added regression coverage for registration, permissions, concurrency, and edition selection. Rust and TOML formatting passed. Tests were not executed. Snapshot review, CUDA execution, Wasm execution, and benchmarks remain unrun. The whole-workspace pre-push Clippy hook was skipped after scoped checks. Stateful training in #8434 remains separate.
API Changes
Removes
BtrBlocksCompressorBuilder,with_compact, andonly_cuda_compatible. Call package initialization functions, select editions, then constructBtrBlocksCompressor::from_session. The layout builder remains and accepts an already constructed compressor throughwith_btrblocks_compressor.VortexSession::clonecontinues to share registrations.fork_compressioncopies the compression registry for local configuration while sharing other services.register_cuda_layoutonly registers support, whilecuda_write_strategyselects the CUDA edition on the supplied session. The cascading engine retains sampling, estimates, exclusions, and recursive compression.