Skip to content

feat(CuckooFilter): add cf.del command - #3567

Draft
nagisa-kunhah wants to merge 3 commits into
apache:unstablefrom
nagisa-kunhah:feature/cf-del
Draft

feat(CuckooFilter): add cf.del command#3567
nagisa-kunhah wants to merge 3 commits into
apache:unstablefrom
nagisa-kunhah:feature/cf-del

Conversation

@nagisa-kunhah

@nagisa-kunhah nagisa-kunhah commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

part of: #3552

Summary

Add RedisBloom-compatible CF.DEL key item support for Cuckoo Filter.

CF.DEL deletes one matching fingerprint occurrence and returns 1 when a slot is cleared, or 0 when the key/item
is not found. Duplicate inserts require duplicate deletes, matching RedisBloom behavior.

Design

The command layer adds cf.del as a write command and delegates deletion to CuckooChain::Delete.

Deletion loads the Cuckoo Filter metadata, hashes the item, generates the fingerprint, then searches sub-filters from
newest to oldest. The first matching slot in either candidate bucket is cleared, and only one occurrence is removed.

After a successful delete, metadata size is decremented and num_deleted_items is incremented. When the chain has
more than one sub-filter and accumulated deletes exceed 10% of the remaining item count, an internal compact pass is
triggered. Compact tries to move fingerprints from newer sub-filters into older ones, removes fully compacted latest
sub-filters, and deletes their persisted page keys to avoid stale data if the chain expands again later.

Page key construction is shared through small Cuckoo page helpers so compact cleanup uses the same encoding as normal
page access.

This pr is This PR was written using codex and GPT-5.5

@nagisa-kunhah nagisa-kunhah changed the title feat: del feat(CuckooFilter): add cf.exists and cf.del command Jul 26, 2026
@nagisa-kunhah nagisa-kunhah changed the title feat(CuckooFilter): add cf.exists and cf.del command feat(CuckooFilter): add cf.del command Jul 26, 2026
@nagisa-kunhah

Copy link
Copy Markdown
Contributor Author

Hi @jihuayu, while implementing CF.DEL, I found that the automatic compaction path may introduce serious worker-blocking and OOM risks. Currently, it scans all logical buckets of a sub‑filter and loads every page into CuckooPageCache, including zero‑filled pages for missing keys. Thus, a large sparse sub‑filter can make a single CF.DEL perform work and allocate memory proportional to its logical capacity.

We seek feedback on whether compaction should be part of this PR. Two options:

  1. Land CF.DEL without automatic compaction for now.
    Only remove fingerprint and update metadata. Semantics are preserved, but deleted slots and extra sub‑filters remain, causing space and lookup amplification.

  2. Move compaction to bounded, incremental background maintenance.
    Use Kvrocks’ TaskRunner (used by AsyncCompactDB, AsyncScanDBSize, etc.) with cursor‑based scanning. Process a bounded number of actually stored pages per slice, commit a bounded write batch, release cache and locks, and requeue if more work remains. Requires actual‑page iteration, bounded cache/batch, deduplication, synchronization, and crash‑safe progress tracking.

@jihuayu

jihuayu commented Aug 3, 2026

Copy link
Copy Markdown
Member

@nagisa-kunhah Let's leave this out for now. We can implement background compaction in a follow-up PR. Thanks.

PS: when the key does not exist, RedisBloom returns a Not found error, while this PR currently returns success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants