feat(planner): MILP roll-ups for exact sum, count, min and max (#852)

* feat(promql-compliance): let a suite supply fixed planner output

Add an optional suite-level planner_output_dir holding streaming_config.yaml
and inference_config.yaml. When set, the runner validates both files at load
time, skips generating the planner input, and adds a Compose override that
parks the planner behind an inactive profile and mounts the directory where
the query engine reads planner output. This lets the harness exercise engine
configs the planner does not produce yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(promql-compliance): add roll-up suite with fixed planner output

The rollup suite reads KLL, DDSketch, Sum and MinMax aggregations grouped by
(service, endpoint), over tumbling and sliding windows, through queries
grouped by (service, endpoint), by (service) and with no grouping. The
dataset includes an endpoint that starts mid-range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(planner): MILP roll-ups for KLL, DDSketch and HLL

A MILP deployment grouped by G can serve a query grouped by Q ⊊ G. For
single-subpopulation sketches (DatasketchesKLL, DDSketch, HLL) the engine
merges across G-keys at query time, so the planner now writes the
query_config pointing at the G-grouped aggregation instead of refusing.

aggregation_config takes its labels from the deployment's grouping rather
than the first served item, so a shared deployment is written at G
whichever query comes first. Other families' roll-ups stay
RollupUnsupported, and a query grouping outside the deployment's is a new
GroupingOutsideDeployment error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(planner): MILP roll-ups for exact sum, count, min and max

The MILP emits exact-sum/min/max as keyed MultipleSum/MultipleMinMax,
which the engine doesn't roll up itself. Regrouping their final values is
exact, so a roll-up from grouping G to Q now writes a query_config whose
planned_subquery is the query with its outer aggregation rewritten to
`by (G)` (on the AST, which also covers `without`) and whose
query_time_aggregations is one Sum/Min/Max stage `by (Q)`, or `all` when
Q is empty. Count regroups with Sum. exact-increase and cms-heap roll-ups
stay RollupUnsupported.

Adds an end-to-end engine test that such a config over a MultipleSum
keyed by (instance, job) answers `sum by (job)` exactly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(query-engine): move the exact roll-up e2e test off a shared port

It and the topk-of-sum e2e test both bound port 19429, so running them in
parallel failed one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(planner): build exact roll-up stages from the query's aggregation

Find the outer aggregation through parentheses, build its query-time
stage with aggregation_stage (count read as a sum of counts), and pick
regroupable deployments by capability alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(planner): refuse exact roll-ups for binary-query leaves and SQL

An exact roll-up needs a query-time stage. The engine runs a binary
query's arms without one, and the SQL path doesn't run them at all, so
those queries now get RollupUnsupported with the reason, instead of a
plan the engine can't answer or a misleading UnregroupableQuery.
Sketch roll-ups need no stage and stay allowed for binary-query leaves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(planner): share the roll-up type list and check groupings up front

- AggregationType::merges_across_groups() is now the one list of types a
  roll-up can merge; the engine and the MILP output both use it, and the
  refusal message is built from it.
- A query grouping outside its deployment's is refused before any
  aggregation is built, so a param error can't hide it.
- A top-k heap takes its buckets from the query's own topk by labels.
- Tests: HLL roll-up from a SQL COUNT(DISTINCT) query, a saved-plan DD
  roll-up that succeeds, and widen_to_instance_job sets the grouping.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(planner): check the saved plan's roll-up stage and planned subquery

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(query-engine): e2e checks for min, max, count and all roll-ups

Each reads a keyed (instance, job) aggregation and regroups it at query
time, checking exact instant and range values: min and max from
MultipleMinMax, count by (job) and sum of count_over_time from a keyed
count, and sum() reducing a keyed sum to one series.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(planner): gate exact roll-ups on the aggregation type; test unregroupable and without

- An exact roll-up needs a MultipleSum or MultipleMinMax aggregation, so
  the check reads the aggregation type instead of repeating the
  (variant, capability) mapping.
- Test that a query with no outer aggregation is UnregroupableQuery.
- Engine e2e test for a without stage over the keyed output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(promql-compliance): harden the fixed planner output override

- Check docker compose >= 2.24.4 (needed for !override) before using it,
  and document it.
- Clear COMPOSE_PROFILES so an inherited value can't re-enable the planner.
- Bind the fixed directory with long syntax, so a ':' in its path is safe.
- Only skip planner-input generation when a Compose project is used.
- LoadSuite rejects a relative planner_output_dir and validates an absolute
  one, so every loaded suite holds an absolute, checked path.
- The checked-in fixture test also checks aggregation ids, metrics and
  grouping labels; an engine test parses the fixed configs with the
  engine's own loaders.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Author: milindsrivastava1997

asap-planner-rs/src/optimizer/milp.rs

@@ -602,7 +602,7 @@ metrics:
         names.iter().map(|s| s.to_string()).collect()
     }
 
-    fn cost(sketch: &str, accuracy: &[(&str, f64)]) -> AtomicCostEntry {
+    pub(crate) fn cost(sketch: &str, accuracy: &[(&str, f64)]) -> AtomicCostEntry {
         AtomicCostEntry {
             sketch: sketch.into(),
             sketch_config: serde_json::json!({"algorithm": sketch, "params": {}}),