Skip to content

feat: own Float64 weighted CMS and CountSketch kernels in sketchlib - #160

Closed
zzylol wants to merge 1 commit into
mainfrom
feat/weighted-frequency-kernel
Closed

zzylol wants to merge 1 commit into
mainfrom
feat/weighted-frequency-kernel

Conversation

@zzylol

@zzylol zzylol commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Before this PR

ASAPPlanner #462 implemented Float64 weighted CMS/CountSketch matrices and candidate heaps inside its physical-operator crate. The algorithms depended on Planner runtime values and could not be reused directly from sketchlib.

After this PR

WeightedFrequency owns the matrix, hashing, estimation, typed candidate heap, merge, memory estimate and versioned snapshot logic in sketchlib. It exposes primitive typed tuple identities without a dependency on Planner, physical operators or execution scopes. ASAPPlanner retains only its parameter/value/trait adapter.

For example, two updates of 0.125 and 0.25 to the same identity produce a 0.375 estimate without integer rounding. CMS accepts nonnegative weights; CountSketch accepts signed weights and uses sign-corrected median estimates at positive odd depth. Candidate retention does not establish TopK membership completeness.

Existing ASAP-WFREQ-1 bytes are preserved. This is separate from integer sketch and cross-language interchange formats. API documentation and a runnable example are included.

Validation

  • Seven focused tests passed, including migrated algorithm regressions, typed tuples, signed zero, overflow atomicity, candidate capacity and two golden snapshots generated by ASAPPlanner before extraction.
  • Weighted-frequency documentation example passed.
  • Strict Clippy for library and tests passed; formatting and whitespace checks passed.
  • ASAPPlanner #462 pins this commit; its 303 physical-library tests passed against the published git revision, including typed summary and Planner-to-DAG TopK tests. Its strict workspace Clippy also passed.

Companion: ProjectASAP/ASAPPlanner#462

@zzylol

zzylol commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Closing: main now provides f64 Count-Min and Count Sketch with heaps (#232), ASAPv1 serialization as the only format (#241) and TupleKey group-by keys (#237), so WeightedFrequency's own kernel and ASAP-WFREQ-1 format are redundant. The remaining gaps (shape and weight checks, an atomic checked f64 update, a full clear, opt-in rescoring of retained candidates, and estimated_memory_bytes) move to new PRs on main's heap sketches. The old head is kept on backup/feat/weighted-frequency-kernel-20261020-161432.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant