Repository navigation
fix(planner): price state by its size; size KLL for δ - #682
Merged
Merged
Conversation
zzylol
force-pushed
the
stack/fix-exact-state-cost-kll-delta
branch
from
October 13, 2026 21:07
a9d18fc to
c43586e
Compare
zzylol
added a commit
that referenced
this pull request
Oct 14, 2026
The stage viewer guide and the Example 3 acceptance spec described the ties #682 removes: Theta and KMV costing the same in Example 2, KLL and DDSketch tying in 3a (P122) and 3b (P2). They now give the selections with state priced by its size and no Theta candidate: 3a selects P109 (KLL for q1 and q2, exact for q3-q5), 3b selects exact P1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Oct 14, 2026
The stage viewer guide and the Example 3 acceptance spec described the ties #682 removes: Theta and KMV costing the same in Example 2, KLL and DDSketch tying in 3a (P122) and 3b (P2). They now give the selections with state priced by its size and no Theta candidate: 3a selects P109 (KLL for q1 and q2, exact for q3-q5), 3b selects exact P1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/fix-exact-state-cost-kll-delta
branch
from
October 14, 2026 17:09
a47471a to
3b2f31e
Compare
…s size Exact quantile, top-k, distinct-count and entropy state keeps every value of its group, so is priced at one value per input row of the group; fixed exact accumulators stay one value per group. KLL state is sized by k, and retained bytes are now priced, so a sketch smaller than the exact state it replaces is cheaper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion order A valid candidate that ties the selected one was reported as "costlier: X vs X"; it now says it tied with the selected candidate and lost on enumeration order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
KLL was sized and certified only at the empirical 99% calibration, so every KLL missed a target with delta < 0.01. Below 0.01, k is now sized by the Karnin-Lang-Liberty bound's sqrt(ln(1/delta)) dependence anchored at that calibration, and the params record the delta, so the guarantee reports it. delta >= 0.01 keeps the calibrated sizing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…es not compose The three logical_candidates test helpers skipped a candidate whose composition failed, so an assertion over every candidate could pass on none. They now panic with the candidate and its error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Exact L2 keeps one counter per distinct value of its group, like distinct count and entropy, so it is priced as growing state, not one value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A DDSketch was priced at a fixed 1 KiB. It now keeps one 8-byte count per bin over an assumed nine-decade value range, ceil(ln(1e9) / ln(gamma)) bins, capped by the bins over every value asap_sketchlib can index. Exact quantile state over small groups is now cheaper than the sketch and is selected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e (0, 1) A hand-built failure probability of 0, a negative value, 1 or more, NaN or infinity gave an infinite or NaN rank-error bound; it now gives none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eir parameters Plan selection priced every HLL, KMV, Theta and UnivMon state at a fixed 1 KiB, so a distinct-count Theta and KMV candidate tied whatever their k, and exact distinct-count state was compared against a constant. State is now sized from the sketch's parameters, as asap_sketchlib allocates it: HLL one byte per register (2^precision), KMV k 64-bit hashes, Theta an Apache DataSketches update table of 2k 64-bit hashes (asap_sketchlib has no Theta sketch), and UnivMon per layer a rows x cols Count-Sketch of 8-byte counters plus a heap of 24-byte entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The DDSketch bin count, and so its priced state, is proportional to the decades of values it spans; the doc comment now says the 1e9 range is an assumption, not workload evidence, and how the count scales with it. A test pins the documented 1,037 bins at alpha = 0.01. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n contract A KLL sized for delta < 0.01 scales the empirical 99% calibration by the KLL bound's delta dependence. Its guarantee now names that model apache_datasketches_kll_empirical_99_delta_scaled_a9b42755072b, so it is not conflated with the measured calibration's contract. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eta sketch Pass 1 offered Theta for distinct counts, but asap_sketchlib has no Theta sketch to execute it. summary_candidates no longer lists it; it can be added back once the library supports it. The Theta-specific state size (a 2k-hash update table) goes with it; a hand-built Theta is sized like KMV's k hashes. In Example 2, the Theta candidate is gone and the candidates after it are renumbered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A UnivMon row was priced as one update. asap_sketchlib updates layers 0..=b, with b geometric in the row's hash, so a row reaches 2 - 2^(1 - layers) layers in expectation; each updates one counter per sketch row and its heavy-hitter heap. The per-row update cost is now that expectation x (rows + 1), consistent with the parameter-based state size. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 3 priced KLL state as about 3k items, but asap_sketchlib allocates compute_max_capacity(k, 8) items up front: max(ceil(k * (2/3)^i), 8) over 61 levels, 1,206 items at k = 269 rather than 807. It also silently caps k at 26,602 (MAX_CACHEABLE_K), so sizing could produce, and a guarantee claim, a k that never runs. - Stage 3 sizes KLL state as 8 bytes x compute_max_capacity(min(k, 26_602), 8). - KLL_MAX_K = 26_602 in asap_types. kll_k saturates there (a tighter eps gets that k, whose guarantee then misses the target, as other saturated sizings do), a hand-built k above it gives no guarantee, and the KLL kernel capability check and factory reject it. At epsilon = delta = 0.01, quantile(0.99, rate(x[5m])) now selects DDSketch (8.3 KB) over KLL (9.6 KB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… Gaussian quantile ratio Below delta = 0.01, the KLL rank-error bound was the empirical 99% calibration scaled by sqrt(ln(1/delta) / ln(100)). It is now scaled by z(1 - delta/2) / z(1 - 0.01/2), the two-sided standard normal quantile ratio, which grows faster as delta shrinks (1.277 vs 1.225 at delta = 0.001) and so is the conservative choice. Sizing and the guarantee use the same factor: at epsilon = 0.01, delta = 0.001, k goes from 331 to 345. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ty they were sized for SketchParams::Kll keeps failure_probability, but it records what k was sized for, not the sketch that is built. Equality now ignores it, so two KLLs with the same k are structurally equal (summary sharing) and match the same offline evidence in EmpiricalEvidenceProvider::lookup. The inequality arm lists every variant, so a new variant fails to compile until it is compared. The field doc also says how a hand-built Some(delta >= 0.01) is read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three logical_candidates should_panic tests use a query that does not compose because of #672; once it is fixed they must switch to another candidate that does not compose. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stage viewer guide and the Example 3 acceptance spec described the ties #682 removes: Theta and KMV costing the same in Example 2, KLL and DDSketch tying in 3a (P122) and 3b (P2). They now give the selections with state priced by its size and no Theta candidate: 3a selects P109 (KLL for q1 and q2, exact for q3-q5), 3b selects exact P1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…al lower tail delta_scale took z(1 - delta/2), which loses delta to rounding as it nears f64's epsilon: NaN below about 2.2e-16 and slightly low at 1e-15. It now uses the equal -z(delta/2), which the lower-tail approximation computes accurately. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Theta is no longer offered because asap_sketchlib has no Theta sketch. Updates only the statements that listed it as a candidate: the Example 2 acceptance spec (Pass 1 options, now 4 x 2 x 2 = 16; plan selection; the distinct-count invariant), the strategy-extension guide, the pre-ASAP IR tuple-cardinality note and the primitive schema's candidate table and category map. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/fix-exact-state-cost-kll-delta
branch
from
October 14, 2026 17:50
3b2f31e to
ff2c6f0
Compare
zzylol
added a commit
that referenced
this pull request
Oct 14, 2026
Plan selection no longer charges a computation for the memory it holds while it runs (#682's per-computation peak-memory term, including the growing exact state of a query-time aggregate). Memory is charged only as state a materialized node keeps between computations: kept bytes times cost_per_retained_byte_second. The kept state is the node's output, or the values an exact aggregate keeps per group when that is more, so a kept exact quantile or distinct count is still sized by what it keeps. Peak capacity belongs to cost model 2 (#683). Every term is now a per-second cost; the per-computation memory coefficient is ignored. The tests that compared exact state against sketches now keep the state at query time, where its size is charged.
zzylol
added a commit
that referenced
this pull request
Oct 14, 2026
Plan selection no longer charges a computation for the memory it holds while it runs (#682's per-computation peak-memory term, including the growing exact state of a query-time aggregate). Memory is charged only as state a materialized node keeps between computations: kept bytes times cost_per_retained_byte_second. The kept state is the node's output, or the values an exact aggregate keeps per group when that is more, so a kept exact quantile or distinct count is still sized by what it keeps. Peak capacity belongs to cost model 2 (#683). Every term is now a per-second cost; the per-computation memory coefficient is ignored. The tests that compared exact state against sketches now keep the state at query time, where its size is charged.
zzylol
added a commit
that referenced
this pull request
Oct 14, 2026
…-expression top-k heaps, counter-sample weights, per-second plan-selection cost (#692) fix(logical-asap-optimization): weight Top-K heap sketches by the ranked value Pass 1's Top-K sketch update weighted every heap by the sample value. Over SQL rows that column is the one named `value`, or else the only numeric column: an Int64 key made the heap candidates fail to compose, and a key named `value` would have weighted the heap by the key. The update now reads the column the top-k ranks (the aggregate's measure over grouped rows, the sample value over per-series rows), the same column exact top-k sorts by. `ranked_column` moves from Stage 2 into the shared realization helpers so both stages use it. feat(logical-asap-optimization): Pass 2 identical-expression rule as Stage 3 candidates Stage 1 Pass 2 now applies the identical-expression rule of Algorithm 2. For each composed Pass 1 candidate it finds the sharing groups (two or more nodes with identical input and computation semantics that are legal to share) and adds one candidate per non-empty subset of the groups, keeping the independent candidate. A subset that leaves out a group that the other shared groups already merge gives the same candidate as the larger subset, so it is listed once. Nothing is capped. The variants are ordinary candidates for Stage 2 and Stage 3. PromQL rows that carry the full series identity now declare (identity, timestamp) as a unique key, which makes scans and range selectors legal to share. Plan selection treats totals that differ only by floating-point rounding as a tie and keeps the earlier candidate. A shared NotMaterialized input is charged once per consumer, so it ties with the independent plan. Example 1 gets 9 logical candidates (3 Pass 1 x (1 + 2 Pass 2)) and Example 3a gets 972. stage_pipeline's --max-combinations still counts Pass 1 combinations; its document lists every Stage 1 candidate. The Example 1 and scalar-queries viewer fixtures are regenerated. refactor(logical-asap-optimization): move the keyed-additive heap rule into the realization helpers Pass 1's whole-expression heap reuses the legacy search's keyed-additive rule, so the rule, its result types and schema_column_ref move from replacement.rs into realization.rs, which both use. Pure move. feat(logical-asap-optimization): whole-expression top-k heaps and counter-sample weights Pass 1 gains a whole-expression candidate kind. A top-k over a per-item sum or count that is itself a target (one consumer, no filter or HAVING, realizable by the keyed-additive rule) has its Count-Min and CountSketch heap candidates replaced by whole-expression ones: one heap over the inner aggregate's input, keyed by the ranked item and weighted by the summed value. It absorbs the inner target, which is then not computed. The outer top-k's partition keys are moved onto the input the heap reads. Example 1's Q2 gets exactly three candidates: exact, Count-Min + heap and CountSketch + heap. Enumeration skips choices that differ only in an absorbed target, and PerComputationCandidates::combinations counts the same set. Count-Min needs non-negative weights. Pass 1 does not read the data workload: when a heap's weights are raw samples of metric M, or plain sums of them, it records the open requirement WeightDomain::SamplesOfMetric{M}. Rate, other arithmetic and other functions end it, and no type is inferred from a `_total` name. DataWorkload gains metric_types (declared Prometheus types). Plan selection admits Count-Min over such weights only when M is declared a counter; an unconfirmed requirement counts as unmet. The planner pipeline's selected plan carries the confirmed requirement as the proof NonNegativeWeightProof::CounterSamples, since the executor reads no data workload. Example 1 declares http_requests_total a counter, so its Count-Min heap is now rejected only because a top-k output has no guarantee; without the declaration it is rejected on its weights. Raw-sample precompute now checks the whole-expression heaps. The Example 1 viewer fixture is regenerated. feat(plan-selection): price candidates per second of wall time Plan selection now prices a candidate per second, so queries with different recurrences add up, and takes each query's recurrence: - A NotMaterialized node is charged once per consumer at the sum of their rates (a shared NotMaterialized input still costs what reading it separately does). - A MatAtQuery node is computed once at the combined rate of the queries reaching it; queries at the same interval run together. - A MatAtIngestion node is priced over one second of arriving rows, once per second. - One-off, scheduled and unknown queries are spread over horizon_s. ResourceCalibration gains horizon_s (default 3600 s) and cost_per_retained_byte_second, the price of state kept between computations. Every materialized node's kept state is charged per byte per second: twice its output at ingestion time (the window being built and the completed one), once at query time. Memory held only during a query gets no per-second charge. select_plan, run_stage1_to_stage3 and run_stage1_to_stage3_from_pass1 take the recurrences; the planner pipeline and stage_pipeline read them from the workload. The cost model is described in the stage3_plan_selection module docs. The viewer fixtures are regenerated. fix(plan-selection): charge memory only as kept state per second Plan selection no longer charges a computation for the memory it holds while it runs (#682's per-computation peak-memory term, including the growing exact state of a query-time aggregate). Memory is charged only as state a materialized node keeps between computations: kept bytes times cost_per_retained_byte_second. The kept state is the node's output, or the values an exact aggregate keeps per group when that is more, so a kept exact quantile or distinct count is still sized by what it keeps. Peak capacity belongs to cost model 2 (#683). Every term is now a per-second cost; the per-computation memory coefficient is ignored. The tests that compared exact state against sketches now keep the state at query time, where its size is charged. feat(plan-selection): bind the sign of confirmed counter samples An exact sum or mean of relative-error values keeps the relative bound only over non-negative values. When the folded values are raw samples of one metric, or plain sums of them (an exact sum accumulator, a renaming, or a heap estimate whose weights carry the SamplesOfMetric requirement), Stage 1 now names the unknown sign by that metric, values_non_negative[M], and gates the bound on it. Plan selection binds it to 1 when the data workload declares M a counter; a gauge, an undeclared metric, rate, other arithmetic or a name such as _total leave it unbound, so the target stays unmet. Other folds keep the bare values_non_negative, which nothing binds. feat(devtools): limit Stage 1 candidates, Pass 2's included, while enumerating stage_pipeline's --max-combinations (default 10^4) now counts Stage 1 candidates as they are enumerated, Pass 2's identical-expression candidates included, and stops with an error as soon as the count passes the limit; it never truncates. Exit status 3 (the viewer server's 422) is unchanged. The library stays uncapped: Stage 1 is exposed as a lazy iterator (stage1_logical_asap_optimization) and Stage 2 and Stage 3 run over the candidates a caller enumerated (run_stage2_and_stage3); run_stage1_to_stage3_from_pass1 enumerates them all. planning_stages takes the limit as an Option. perf(logical-asap-optimization): enumerate distinct Pass 2 candidates directly The identical-expression rule tried all 2^g subsets of the sharing groups and dropped those that repeat a candidate. With nested groups most subsets repeat (14 groups: 16 384 subsets for 130 candidates). Sharing a set of groups also shares the groups it leaves as one node; that closure maps each subset to its candidate, and the distinct candidates are its closed sets. They are now enumerated directly (Next Closure: one closure per step and group), then put in the same binary-counter order, so the output is unchanged. Twenty nested groups give their 20 candidates quickly; a brute-force test checks the same candidates on small cases. Sharing groups are computed once per candidate, and a Pass 2 candidate names its shared groups by kind and number (Scan #1, TimeRange #2), so repeated kinds stay distinguishable. The viewer fixtures are regenerated. fix(plan-selection): every consumer gets the selected plan with confirmed weights The counter-sample requirement Stage 3 confirms was made a proof only in the planner pass, so run_stage1_to_stage3's selected_logical and the selected candidate's roots still carried the open requirement, which the executor rejects. Stage 3 now puts the plan on its output: Selection::plan is the selected candidate's roots with confirmed weights, and Stage1ToStage3Run::selected_logical returns the selected logical roots the same way. The pass reads Selection::plan. fix(logical-asap-optimization): an open weight requirement does not certify Count-Min estimate_has_bound treated any weight domain but UnknownOrSigned as proven, so a Count-Min estimate over weights with the open SamplesOfMetric requirement got its bound as if the weights were proven non-negative. Only NonNegative now counts. Such an estimate instead gets its local bound gated on the metric's sign, values_non_negative[M], which plan selection binds to 1 when M is declared a counter and leaves unbound otherwise. test: Example 1's shared-input totals tie with the independent one up to rounding Shows that the tie tolerance is needed: the shared-input candidates' totals differ from the independent candidate's in the last bits, and plan selection reports them as tied and keeps the independent one. Also drops stale wording in the acceptance helpers. test(logical-asap-optimization): Pass 2 candidate labels and roots agree Applying a candidate's label (the groups it shares) gives the same roots the direct enumeration returned, sharing exactly those groups. fix(plan-selection): the sign gate opens only at 1 The sign of one metric's samples gates a relative bound as a factor; a binder supplying 0 would have read as a perfect bound. Plan selection now binds a sign statistic only to 1 (proven) and leaves any other value unbound. docs: update the Example 1, 2 and 3 selections this PR changes The Example 3 acceptance spec and the stage viewer guide now give 3a's 972 candidates and its all-KLL selection, 3b's KLL selection tied with DDSketch, the per-second totals, and the shared-input plans that tie with the independent one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
An audit of
mainfound that Stage 3 could not tell exact state from sketch state by cost, and that KLL sizing was wrong outside its default δ:quantile(0.99, rate(x[5m])), exact, KLL and DDSketch tied, and exact won only by enumeration order.asap_sketchlibcannot execute.costlier: X vs X.logical_candidatestest helpers silently skipped candidates that did not compose.What
asap_sketchliballocates it (k = 269 → ~9.6 KB). DDSketch is sized from α over an assumed nine-decade range (α = 0.01 → 1,037 bins, ~8.3 KB). HLL, KMV and UnivMon are sized from their parameters.1e-7/byte of peak operator memory, the same as a scanned byte. These numbers are illustrative, not calibrated.tied with the selected candidate Pn at X; lost on enumeration order.KLL_MAX_K= 26,602. A hand-built k above the cap, or afailure_probabilityoutside (0, 1), gives no guarantee, and the kernel factory rejects the oversized k.SketchParams::Kllgainsfailure_probability: Option<f64>. It is serde-default and skipped whenNone, and ignored by equality and evidence lookup. A δ-scaled KLL names its own guarantee contract.logical_candidateshelpers panic with<id> does not compose: <error>instead of skipping.extend-asap-aware-mapping.md,pre-asap-ir.mdandasap-primitive-schema.md.How
Sizing is in
stage3_plan_selection.rs, followingasap_sketchlib's allocation, for exampleKLL::init_internal→compute_max_capacity(k, 8)andbottom_layer_for_hashfor UnivMon layers. KLL δ scaling and the cap are inaccuracy/estimators/kll.rsandpost_asap/sketch.rs. Assumptions are recorded in the module doc: rows spread evenly over groups, and any row may be distinct.Before / After
main)quantile(0.99, rate(x[5m])), ε = δ = 0.01by (job)Verification
ff2c6f03:testandformat-and-lintpassed.cargo fmt --checkandcargo clippy -D warningsclean.cargo nextest run --workspace: 1608 passed, 50 skipped.test_render.pyOK;test_server.py11 OK.verify_metricsql_external_consumer.shOK atc43586e5. It was not rerun after the later commits, which touch no MetricsQL code.main. Examples:exact_quantile_state_costs_more_than_a_sketch_over_large_groups,kll_state_is_sized_as_asap_sketchlib_allocates_it,ddsketch_bins_span_nine_decades,kll_k_is_capped_at_the_library_maximum,kll_candidate_meets_a_failure_probability_below_one_percent,a_tie_is_reported_as_lost_on_enumeration_order,stage3_plan_selection_has_no_theta_candidate, andlogical_candidates_fails_on_a_candidate_that_does_not_compose(one per crate).examples/planner-layering-example1.jsonandexamples/scalar-queries.json. Only Stage 3 costs change.Limitations and follow-ups
Human review — do not complete with an agent