Repository navigation
Conversation
… duplicate scan for hashes past the k-th estimate took &mut self and merge took the other sketch as &mut KMV without mutating either, so a caller holding a shared state had to clone it (k hashes) to read an estimate or to merge it in. Both now borrow shared; &mut call sites still compile. approx_memory_bytes reports the struct plus the hash slots the heap has allocated, so callers stop guessing the footprint. (The heap reserves up to 1024 slots and grows toward k, so neither the retained count nor k alone is the allocation.) insert_by_hash scanned all k retained hashes for a duplicate before offering a hash to the heap, so every steady-state insert cost O(k). A full sketch now turns away a hash at or above its largest retained one first; the heap rejected those anyway, so the retained set and every estimate are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
KMVmakes shared readers pay for copies and its own inserts payO(k):estimatetakes&mut selfandmergetakes the other sketch as&mut KMV, though neither mutates. A caller holding a state behindArcor&(a pane shared by several windows) must clone allkhashes to read an estimate or to merge it in. ASAPPlanner's executor does both on every read and every merge.insert_by_hashscans allkretained hashes for a duplicate before offering a hash to the heap, so once the sketch is full every insert costsO(k), even though almost every hash is above the k-th smallest and the heap would turn it away.k * 8bytes, assuming the heap reserves allkslots up front. It reserves at most 1024 and grows.What
estimate(&self)andmerge(&mut self, other: &KMV<H>). Existing&mutcall sites still compile.approx_memory_bytes(&self): the struct plus the hash slots the heap has allocated.CommonHeap::allocated_slots(), the heap's allocated element slots, which the memory report reads.How
The early return in
insert_by_hashis exactly the caseCommonHeap::pushalready ignored for a full max-heap (new >= root). The memory report reads the heap'sVeccapacity through the new accessor.Key code interfaces added / modified
src/sketches/kmv.rssrc/common/structures/heap.rsUsage, end to end (checked by a scratch test at the head commit):
Before this PR
Release build,
k = 4096, 2,000,000 distinct inserts: 2.43 us per insert. Reading or merging a shared state required a clone of up tokhashes.After this PR
The same insert loop: 37 ns per insert. Shared states are read and merged by reference.
Verification
estimate_and_merge_borrow_inputs_shared(an immutable state's estimate; merging an immutable sketch equals the union build) andmemory_report_covers_the_retained_hashes(k = 3000: the report covers every retained hash, grows from empty to full, and does not move on a duplicate or a hash ofu64::MAX).src/sketches/kmv/wire.rs,tests/pbt/kmv.rs,tests/e2e/experimental.rsandtests/e2e/wire.rsdrop their now-needlessmut.cargo fmt --all -- --check; clippy with-D warningsfor--all-features, default features,experimentalandocto-runtime;cargo test --all-features --locked(every target) passes.main'skmv.rsand this branch's, removed before commit.Limitations
mergestill panics on a differentk, as before.