Skip to content

feat(storage): isolate zset random and scan reads by group - #436

Merged
thweetkomputer merged 1 commit into
mainfrom
zset-sampling-intent-reads
Nov 6, 2026
Merged

thweetkomputer merged 1 commit into
mainfrom
zset-sampling-intent-reads

Conversation

@thweetkomputer

@thweetkomputer thweetkomputer commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Grouped ZRANDMEMBER now samples ranks before taking IS and S on the selected ordered pages; ZSCAN takes S on the member leaves covering its cursor/count window. Both revalidate their plan after waits and retain a common logical cut through reply construction. A changed random population falls back once to key S. Scan keeps its full ordered-page compatibility path when persisted and public cursor seeds differ.

Validation: Debug builds and nine focused E2E tests pass, covering selective read/write overlap, conflict wait/wakeup, random semantics and OOM, scan cursor/mutation/promotion/recovery behavior, and the existing range concurrency cases.

Based directly on main; this PR contains only random and scan reads.

Summary by CodeRabbit

  • Improvements
    • ZRANDMEMBER and ZSCAN can now run alongside updates to unrelated sorted-set groups, while preserving consistent results for the data they read.
    • Concurrent reads remain protected when updates affect the same data being read.

@coderabbitai

coderabbitai Bot commented Nov 5, 2026 •

Copy link
Copy Markdown

Explain Change →

Caution

Review failed

The pull request is closed.

Walkthrough

Grouped sorted-set reads now route eligible ZRANDMEMBER and ZSCAN operations through concurrent page reads. The read path locks selected pages or member-index leaves, validates read state, and refreshes data under those locks. Architecture documentation and concurrency and recovery tests cover the behavior.

Changes

Grouped sorted-set sampling reads

Layer / File(s) Summary
Read eligibility and lock planning
src/storage/engine/sorted_set_api.cpp, src/storage/engine/impl.h, src/storage/engine/grouped/zset.cpp
Eligible random and scan operations can use the concurrent read path. The path prepares random ranks and selects grouped lock IDs for sampled ordered pages or scan-window member leaves.
Locked reads and validation
src/storage/engine/grouped/zset.cpp, src/storage/engine/impl.h, docs/architecture/09-grouped-collections.md, tests/grouped/zset_write_e2e_test.cpp
Random reads and scans refresh data under selected locks and validate population and routing. Documentation describes the lock windows, and tests cover concurrent writes, returned values, and recovery.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Possibly related PRs

  • eloqdata/lavik#408: Adds the member-leaf ZSCAN path and different-seed fallback that this change extends with locking.
  • eloqdata/lavik#431: Adds prepared page publication and count merging; this change validates reads while unrelated group publications can proceed.
  • eloqdata/lavik#432: Adds IX point writes and locked-group refresh used by these reads.

Suggested reviewers: liunyl

Merge Risk: Low · up to f7c4a

The change is mergeable with bounded owner awareness: uncommon storage API usage can produce scan failures during concurrent writes or false out-of-memory errors before memory-limit initialization. Normal server initialization avoids the latter. Both localized fixes are recommended, alongside stronger revalidation coverage.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A rabbit checks the pages with care
And holds the leaves that fall in range
A distant group can still make tracks
The scores stay true through read and write
Then home I hop, with locks in tow

@thweetkomputer
thweetkomputer force-pushed the zset-range-intent-reads branch from 819ecde to b30065a Compare November 5, 2026 11:12
@thweetkomputer
thweetkomputer force-pushed the zset-sampling-intent-reads branch 2 times, most recently from 9898b1b to add7fa9 Compare November 5, 2026 12:35
@thweetkomputer
thweetkomputer force-pushed the zset-range-intent-reads branch from b30065a to 5681af9 Compare November 5, 2026 12:35
@thweetkomputer
thweetkomputer force-pushed the zset-sampling-intent-reads branch from add7fa9 to 950850d Compare November 6, 2026 05:07
@thweetkomputer
thweetkomputer force-pushed the zset-range-intent-reads branch from 5681af9 to 54d48dd Compare November 6, 2026 05:07
@thweetkomputer
thweetkomputer force-pushed the zset-sampling-intent-reads branch from 950850d to d05481d Compare November 6, 2026 06:13
@thweetkomputer
thweetkomputer force-pushed the zset-range-intent-reads branch 2 times, most recently from 88784f8 to 2d57e86 Compare November 6, 2026 06:49
@thweetkomputer
thweetkomputer force-pushed the zset-sampling-intent-reads branch from d05481d to 6cb62a6 Compare November 6, 2026 06:49
@thweetkomputer
thweetkomputer force-pushed the zset-range-intent-reads branch from 2d57e86 to 9b90efd Compare November 6, 2026 07:38
@thweetkomputer
thweetkomputer force-pushed the zset-sampling-intent-reads branch 2 times, most recently from 7129426 to a22b186 Compare November 6, 2026 07:53
@thweetkomputer
thweetkomputer force-pushed the zset-range-intent-reads branch 2 times, most recently from 17551b7 to fe76457 Compare November 6, 2026 08:18
@thweetkomputer
thweetkomputer force-pushed the zset-sampling-intent-reads branch from a22b186 to c567408 Compare November 6, 2026 08:18
@thweetkomputer
thweetkomputer force-pushed the zset-range-intent-reads branch from fe76457 to c94dfd7 Compare November 6, 2026 08:22
@thweetkomputer
thweetkomputer force-pushed the zset-sampling-intent-reads branch 2 times, most recently from 5e13c6a to 0e4a0ad Compare November 6, 2026 09:00
Base automatically changed from zset-range-intent-reads to main November 6, 2026 09:04
@thweetkomputer
thweetkomputer force-pushed the zset-sampling-intent-reads branch from 0e4a0ad to f7c4a01 Compare November 6, 2026 09:08
@thweetkomputer
thweetkomputer merged commit eaad37a into main Nov 6, 2026
6 checks passed
@thweetkomputer
thweetkomputer deleted the zset-sampling-intent-reads branch November 6, 2026 09:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant