Skip to content

feat(asap-tools): run the e2e planner through the rqe-optimizer MILP (planner.use_optimizer) #830

Open

Description

@milindsrivastava1997

experiment_run_e2e and experiment_run_clickhouse start asap-planner without --planner, so they always use the legacy planner. Add a Hydra option to run --planner milp, which uses sketch-bench's rqe-optimizer.

Config (config/config.yaml):

planner:
  use_optimizer: true     # --planner milp
  saturation_dir: null    # remote path; atomic costs read from <dir>/optimizer_cost/rqe_atomic_costs.json (rqe_optimizer::saturation::COST_TABLE)
  workload_facts: null    # remote path to the workload-facts YAML
  w_cpu: null             # null = rqe-optimizer default
  w_mem: null

Behavior when use_optimizer=true:

  • ControllerService passes --planner milp --workload-facts --atomic-costs --saturation-dir [--w-cpu] [--w-mem].
  • Bare metal uses the remote paths as given.
  • Containerized: the compose template adds read-only mounts (/app/saturation, /app/input/workload_facts.yaml). The sketchdb-controller image has to be rebuilt by hand so it includes --planner milp.
  • aggregate_cleanup and sketch_parameters are left out of controller_input.yaml, because MILP rejects them (milp.rs:249) and the MILP picks sketch parameters itself.
  • These fail fast in Python:
    • saturation_dir or workload_facts is unset
    • controller.punting=true
    • controller.auto_discover_labels=true
    • windowing is set
    • experiment_run_clickhouse is used, since SQL is unsupported by MILP. Pass planner.use_optimizer=false.

Known limitation: MILP output uses NoCleanup (milp_output.rs:159), so optimizer runs keep every aggregate in memory until #800 lands.

Tests:

  • tests/test_controller_service.py: CLI args and compose mounts for both modes.
  • Validation tests for each fail-fast case, including the ClickHouse script in tests/test_experiment_run_clickhouse.py.

Related: #790, #800.

Activity

  1. added a commit that references this issue on Oct 12, 2026
    542e4cf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions