Skip to content

feat(weighted_frequency): shape and weight checks without a state, and clear in place - #226

Closed
zzylol wants to merge 1 commit into
feat/weighted-frequency-point-estimatefrom
fix/weighted-frequency-shape-and-weight-checks
Closed

zzylol wants to merge 1 commit into
feat/weighted-frequency-point-estimatefrom
fix/weighted-frequency-shape-and-weight-checks

Conversation

@zzylol

@zzylol zzylol commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Why

A caller that sizes a WeightedFrequency from configuration has to reject a bad shape or a bad weight before it builds or updates a state. Today the only check is to try: new allocates width * depth cells, and update mutates. So ASAPPlanner's executor restates the kernel's rules itself: an odd CountSketch depth and positive dimensions in its plan validation, its cost-evidence check and its family mapping, and "finite, nonnegative for CMS" in its input validation. It also stores (algorithm, width, depth, capacity) next to every state only to rebuild an empty one at each window.

What

  • WeightedFrequency::check_shape(algorithm, width, depth): accepts exactly the shapes new builds, without allocating.
  • WeightedFrequency::check_weight(&self, weight): accepts exactly the weights update takes, without touching the state.
  • WeightedFrequency::clear(&mut self): empties the counters and candidates in place, keeping the shape and allocation.

new, update and from_bytes now call the two checks, so the rules live in one place. No behavior changes for existing callers.

Stacked on #225 (and so on #160). Retarget to main after those merge.

How

The shape test that new and from_bytes each inlined moves into check_shape; the weight test at the top of update moves into check_weight. clear zero-fills cells and clears the candidate heap.

Key code interfaces added / modified

src/sketches/weighted_frequency.rs

impl WeightedFrequency {
    /// Checks a shape without allocating it: positive width and depth, a
    /// representable width * depth, and an odd CountSketch depth (the median
    /// needs an odd row count). Err(Invalid) is bad input. Used before a
    /// caller accepts a configuration; `new` accepts exactly these shapes.
    pub fn check_shape(algorithm: FrequencyAlgorithm, width: usize, depth: usize) -> Result<(), WeightedFrequencyError>;   // new

    /// Checks a weight `update` would accept: finite, and nonnegative for CMS.
    /// Err(Update) is bad input. Used to reject a sample before it reaches the state.
    pub fn check_weight(&self, weight: f64) -> Result<(), WeightedFrequencyError>;   // new

    /// Empties the counters and candidates in place, keeping the algorithm,
    /// shape and allocation. Used to reuse a state for the next window.
    pub fn clear(&mut self);   // new
}

Usage, end to end (checked by a scratch test at the head commit):

use asap_sketchlib::{FrequencyAlgorithm, FrequencyIdentity, WeightedFrequency};

// 1. A planner sized `topk by (service) (2, sum_over_time(m[5m]))` as a CountSketch with a
//    50-key heap, 7500 x 85. Before accepting that plan, the executor checks the shape.
//    `shape` is Ok: 85 rows is odd, so the median is defined. Nothing is allocated.
let shape = WeightedFrequency::check_shape(FrequencyAlgorithm::CountSketch, 7500, 85);
// `even` is what a 7500 x 84 plan gets back: Err(Invalid("invalid weighted frequency dimensions")).
let even = WeightedFrequency::check_shape(FrequencyAlgorithm::CountSketch, 7500, 84);

// 2. At ingestion, `state` is one population's summary (service="a"): 7500 x 85 cells, 50 candidates.
let mut state = WeightedFrequency::new(FrequencyAlgorithm::CountSketch, 7500, 85, 50)?;
// `sample` is one raw sample of series {instance="2"}: its value is the weight. The executor
// checks it first, so a NaN or (for CMS) a negative sample is refused as bad input.
let sample = 3.0;
state.check_weight(sample)?;
state.update(&[FrequencyIdentity::Utf8("2".into())], sample)?;

// 3. The window closes. `top` is what the query reads back: [([Utf8("2")], 3.0)].
let top = state.topk(2);
// `clear` readies the same allocation for the next window; no shape needs remembering.
state.clear();

Before this PR

A caller either built a state to learn whether a shape was valid (allocating width * depth cells) or copied the rules. ASAPPlanner copied them in four places and kept the shape beside each state to rebuild it.

After this PR

The caller asks the kernel. ASAPPlanner's companion PR deletes its copies and calls check_shape, check_weight and clear.

Verification

  • New tests, failing to compile before the change: shape_and_weight_checks_match_construction_and_update (seven shapes, including even CountSketch depth, zero dimensions and usize::MAX width, agree with new; five weights agree with update for both algorithms) and clear_empties_in_place_and_keeps_the_shape (a cleared state equals a fresh one byte for byte).
  • cargo fmt --all -- --check; cargo clippy --workspace --all-targets --all-features --locked -- -D warnings; cargo test --all-features --locked weighted and the WeightedFrequency doc test pass.
  • ASAPPlanner's asap-physical-operators (212 tests) and asap-aware-mapping (581 tests) pass against a pin that includes this commit.

Limitations

The error strings are unchanged. The checks return the existing WeightedFrequencyError variants (Invalid for shapes, Update for weights).

…d clear in place

check_shape accepts exactly the shapes new builds (positive width and
depth, a representable cell count, an odd CountSketch depth), and
check_weight exactly the weights update takes (finite; nonnegative for
CMS). A caller can reject a configuration or an input before it builds
or touches a state, instead of repeating these rules. new, update and
from_bytes now call them, so the rules live in one place.

clear empties the counters and candidates in place and keeps the shape,
so a caller can reuse a state for the next window without remembering
its dimensions.
@zzylol

zzylol commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Closing: main now provides f64 Count-Min and Count Sketch with heaps (#232), ASAPv1 serialization as the only format (#241) and TupleKey group-by keys (#237), so WeightedFrequency's own kernel and ASAP-WFREQ-1 format are redundant. The remaining gaps (shape and weight checks, an atomic checked f64 update, a full clear, opt-in rescoring of retained candidates, and estimated_memory_bytes) move to new PRs on main's heap sketches. The old head is kept on backup/fix/weighted-frequency-shape-and-weight-checks-20261020-161432.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant