Skip to content

asap-planner: --planner milp for SQL #834

Closed

Description

@milindsrivastava1997

Part of #753.

asap-planner --planner milp only accepts --query-language promql (run_milp in asap-planner-rs/src/main.rs). The MILP core (solve_milp, Raqe, workload facts, saturation curves) is language-neutral; PromQL is hard-wired only at the input (ControllerConfig → extract_aqes), the labels (metrics: hints) and the output (promql::generator YAML).

Scope

  • --query-language sql only (not elastic_sql).
  • Workload facts for SQL are written manually.

Design

  • Metric key: table.value_column, so each value column gets its own value_range/shapes in the facts file. Its labels are the table's metadata_columns; MILP hints are built from tables:.
  • Requirements extraction: planner-side. Factor the parse/match/top-k detection out of SQLSingleQueryProcessor::get_streaming_aggregation_configs (planner/sql.rs) so the legacy SQL planner and the MILP share it. The engine is untouched: MILP output writes each query's aggregation explicitly.
  • Top-k: allowed only when GROUP BY covers every metadata column. The ranked key is then the series identity, which is sketch-bench's model (same as PromQL topk(k, x)); k comes from LIMIT. Otherwise refused: two top-k queries keyed on different columns would both be grouping [] and need different shapes.
    -- supported: ranks (host, region) = whole series
    SELECT host, region, SUM(latency) AS s FROM requests WHERE ... GROUP BY host, region ORDER BY s DESC LIMIT 5;
    -- refused: ranks hosts summed over regions
    SELECT host, SUM(latency) AS s FROM requests WHERE ... GROUP BY host ORDER BY s DESC LIMIT 5;
  • AVG: nothing special; it maps to [Sum, Count], which the MILP already rejects (MultipleStatistics).
  • Code shape: language-neutral core taking (items, hints), with promql_items(&ControllerConfig) and sql_items(&SQLControllerConfig) front ends. Output keeps the shared roll-up check, dedup and aggregation config; only the final YAML emit and the table_name/value_column fields branch by language.
  • Cleanup: NoCleanup, same as MILP PromQL (planner: MILP plan → InferenceConfig should use ReadBased cleanup instead of NoCleanup #800 fixes both). aggregate_cleanup, windowing, sketch_parameters rejected.
  • CLI: --clickhouse-url/--clickhouse-database rejected with --planner milp (as --prometheus-url is); metadata_columns must be listed in the config.
  • rollup_labels left empty, as in MILP PromQL output.

Verification

  • PromQL-twin equivalence: the same workload as SQL on a table and as PromQL on a metric with the same labels and facts. Assert identical deployments, and configs equal apart from the SQL table/value_column fields.
  • Engine round-trip: the emitted SQL configs route each query to its planned aggregation (like plan_round_trips_into_engine_configs).

Stages

  1. Pure refactor to the neutral core; existing PromQL tests unchanged and passing.
  2. SQL front end + SQL output, with the tests above.
  3. CLI wiring in main.rs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions