feat(btree): add btree global index reader/writer/factory#89
Open
lszskye wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add BTree Global Index Reader with Lazy Filtering Support
Summary
Implement BTree-based global index reader with lazy file filtering, complementing the existing BTree global index writer. This enables efficient predicate evaluation over BTree global index files with metadata-based file pruning to minimize I/O.
New Classes
BTreeGlobalIndexer— Top-level indexer implementation for BTree global index. Instead of building a B-Tree directly in memory, it forms a logical B-Tree via multi-level metadata over SST files, significantly reducing memory pressure during index reads. Responsible for creating corresponding Writer and Reader instances.BTreeGlobalIndexReader— Single-file reader for BTree global index. Evaluates filter predicates (Equal, Range, In, StartsWith, etc.) against an SST file structure where each key maps to a list of row IDs. Supports null bitmap handling and min/max key pruning for fast short-circuiting.BTreeGlobalIndexWriter— Writer for BTree global index files. Writes keys in monotonically increasing order into SST files, merging entries with identical keys into compact row ID lists. Null keys are tracked via a separate RoaringBitmap, serialized and appended to the file end on close.BTreeGlobalIndexerFactory— Factory class for BTree global indexers. Implements theGlobalIndexerFactoryinterface and createsBTreeGlobalIndexerinstances from configuration options.LazyFilteredBTreeReader— Lazy-loading, file-filtering BTree global index reader for multi-file scenarios. UsesBTreeFileMetaSelectorto prune candidate files based on predicate conditions before lazily instantiating per-file readers on demand. Supports reader caching and parallel execution.New Tests
btree_compatibility_test.cpplazy_filtered_btree_reader_test.cpp