feat(planner): collapse topk(sum by (l) (sum_over_time)) into one CMS heap

[milindsrivastava1997]

Before this PR

topk(k, sum by (l) (f(m[w]))) had no single-sketch plan.

• MILP and --planner saved: they rejected it with no supported query pattern matched. That includes the docs(planner): plan the AutoSketch vs. ASAPQuery planner evaluation (§6.3) #777 classic workload's 4 topk(32, sum by (label_0) (sum_over_time(data[w]))) queries, so the saved plan's cms-heap-topk-fastpath-vector2d deployment couldn't be converted.
• Legacy planner: it planned an exact sum by (l) anchor with a query-time topk.

After this PR

When f is linear under both sum and topk (sum_over_time, count_over_time), the query collapses into one CountMinSketchWithHeap keyed by l. The weighting is sum for sum_over_time and count for count_over_time, and no query-time stage runs.

• promql_utilities: build_topk_of_sum_patterns() matches the three-level shape. The inner sum is collected as inner_aggregation. topk's output labels, and the labels a topk without (...) buckets over, now come from the inner sum instead of all labels. Every other shape is unchanged, because only these patterns produce the inner token.
• Planner: build_patterns includes the new patterns, so both the legacy and MILP planners emit the heap. The MILP Raqe is a global TopKByValue (grouping []) with k from the outer topk. Its aggregated labels are l.
• Engine: registers the same patterns, so it matches the planned query and answers from the heap.

Behavior change: the legacy planner no longer plans this shape as exact sum plus query-time topk. It's now approximate (CMS) and uses less memory. #744 described this collapse as the optimization.

Still unsupported (scope agreed on #827): sum by (job) (rate(data_total[w])) and topk(32, sum by (label_0) (rate(data_total[w]))). Both need per-series deltas of the cumulative counter before grouping (#496). They still fail loudly.

Validation

• New tests:
  • promql_utilities: output and bucket labels for by, without, and no inner modifier.
  • Planner patterns: sum_over_time and count_over_time match; rate shapes and a non-sum inner reducer don't.
  • Planner integration: one heap with aggregated labels [job], planned subquery equal to the query, empty query-time pipeline.
  • MILP: the Raqe described above.
  • Engine e2e (e2e_topk_of_sum_ranks_groups_from_one_planned_heap): planner config → precompute → query. The data makes the per-series top (c = 4) differ from the per-job top (api = 2 + 3). The test fails without the engine change.
• cargo test for promql_utilities, asap_types, asap_planner and query_engine_rust: all pass. clippy adds no warnings and cargo fmt is clean.
• --planner saved end to end: I merged this branch with feat(planner): asap-planner --planner saved writes a sketch-bench eval plan without re-solving #828 in a scratch worktree and ran it on sketch-bench's synthetic-templatesclassic-shared1 (ASAP, cpu, cheapest), with only the 8 rate queries removed (46 of 54). It wrote 6 aggregations, versus 5 in feat(planner): asap-planner --planner saved writes a sketch-bench eval plan without re-solving #828 with all 12 removed. The new one is CountMinSketchWithHeap sum, depth 3, width 1024, heap 46080, aggregated [label_0], matching the plan's deployment. All 4 sum_over_time topk queries route to it.

Part of #827. Blocks #824.

🤖 Generated with Claude Code

[milindsrivastava1997]

MILP: topk-of-sum Raqes are indistinguishable from other topk Raqes on the same metric, so one deployment can serve queries needing different heap keys.

build_raqe sets grouping_labels to topk_by_labels or [], and Raqe carries no aggregated/key labels. topk(32, sum by (label_0) (sum_over_time(data[15m]))) and topk(5, sum by (job) (sum_over_time(data[15m]))) (or a plain topk(5, sum_over_time(data[15m]))) all become TopKByValue, grouping [], same metric/lookback/filter. The solver can put them on one deployment, and the roll-up guard in plan_to_planner_output only compares grouping_labels, so it passes. aggregation_config then takes aggregated_labels from items.first(), so the heap is keyed by [label_0] and the by (job) query is routed to it and returns wrong results with no error. Before this PR every topk Raqe with the same grouping needed the same keys, so sharing was safe. Fix: add the key labels (req.grouping_labels) to the Raqe identity, or reject a deployment whose items disagree on them.

[milindsrivastava1997]

topk by (l) over an inner sum uses l as heap buckets even when the inner sum already removed l.

topk by (instance) (3, sum by (job) (sum_over_time(m[5m]))): in Prometheus every input series has no instance, so there is a single bucket and the result is the global top 3 jobs. The Without branch now intersects with topk_input_labels, but the By branch still returns [instance]. set_subpopulation_labels then plans grouping [instance] and aggregated [job], so precompute builds one heap per raw instance value holding per-(instance, job) partial sums. The query gets per-instance partial sums instead of per-job totals. Fix: intersect the by labels with topk_input_labels(match_result, all_labels).

[milindsrivastava1997]

The MILP prices and sizes the topk-of-sum CMS heap from the grouping [] data shape, not from the cardinality of the inner sum by keys.

require_shapes and the accuracy curves look up data_shape[raqe.grouping_labels], which is [] for every global topk. The CMS for topk(32, sum by (label_0) ...) holds one key per label_0 value, not one per series. The accuracy/cost estimate, and so the chosen depth/width/heap, is the same as for a per-series heap. That can over-provision a few-key heap or under-provision a many-key heap without any warning.

[milindsrivastava1997]

Doc comments still say topk output labels are always all labels.

get_topk_by_labels's doc (line ~288) still says get_spatial_aggregation_output_labels "always returns all labels for topk regardless of any modifier", and the engine's extract_topk_param doc says topk is "excluded from every collapsable spatial-of-temporal pattern". Both are wrong after this PR and will mislead the next person who changes the topk label logic.

[milindsrivastava1997]

build_topk_of_sum_patterns adds a third copy of the temporal-function list and the pattern-building code that the planner and engine already duplicate.

planner/patterns.rs and simple_engine/mod.rs each hard-code the same [Rate, Increase, SumOverTime, ...] list and the same collapsable aggregation(function(matrix_selector)) builder. This PR moves only the new pattern into promql_utilities (and into query_logics/parsing, a module for parsing helpers, not pattern builders), leaving three lists to keep in sync. Moving the full ONE_TEMPORAL_ONE_SPATIAL and TOPK_OF_SUM builders into one shared promql_utilities function would let the planner and engine share them, so they can't drift apart on which shapes they accept.

[milindsrivastava1997]

Changed comments keep issue-number references, which AGENTS.md (pulled in by CLAUDE.md) forbids.

The repo CLAUDE.md says "See AGENTS.md for process conventions". AGENTS.md says: "Code comments should not include git history, or issue numbers or historical facts." The rewritten comments still carry (#714) (query_requirements.rs:49) and (collapsable, see #508) (query_requirements.rs:101).