planner: rqe-optimizer calls diverge from sketch-bench eval (version pin, minimize vs minimize_usage_cost, accuracy, cost tables) #836

Part of #790 (and ProjectASAP/sketch-bench#143).

The planner (asap-planner-rs) calls sketch-bench's rqe-optimizer differently from the sketch-bench eval (rqe-optimizer/examples/autosketch_vs_asap.rs) and the study scripts. So the plans ASAP deploys end to end are not the plans the eval reports.

1. Library version

Cargo.toml says branch = "main", but the lockfile pins the rev. ASAPQuery is missing these sketch-bench changes:

• sketch-bench#188: milp::minimize_usage_cost (doesn't exist at the pinned rev).
• sketch-bench#190: finer-grouped deployments that can be merged may serve coarser RAQEs. This changes the candidate set (candidates.rs) and the merge and query costs in analytical_cost_model.rs (merge count, roll-up memory).
• sketch-bench#201 / calculate_start_timestamp_sql multiplies duration by scrape_interval, making window scrape_interval× too wide #202: Hydra candidates and cost rows. Only reachable with allow_undeployable_families = true.

2. MILP entry point and objective

The two objectives can choose different plans even at the same weights.

3. Candidates and accuracy

• Undeployable families: the eval builds candidates with build_all_candidates(.., allow_undeployable = true, ..), per stream, so Hydra is allowed. ASAPQuery passes false (main.rs).
• Accuracy: ASAPQuery calls SaturationCurves::accuracy(raqe, d, &facts) once. The eval's worse_read reads at the smallest covered group (covers_share) and, if there is one, at the largest, then keeps the worse value. ASAPQuery has no coverage concept, so its accuracy can look better than the eval's.
• Cost rows: the eval keys cost rows per stream (w.costs[stream]). ASAPQuery uses one flat table for every metric.

4. Atomic-cost tables

• study_saturation.py --phase optimizer-cost writes optimizer_cost/rqe_atomic_costs.json. Both ASAPQuery (feat(asap-tools): run the e2e planner through the rqe-optimizer MILP (planner.use_optimizer) #830, misc.py COST_TABLE) and the eval read this file, so they agree on the input. ✅

• scripts/export_atomic_costs.sh writes out/atomic_costs.json for the legacy greedy grid. It is not a valid MILP input:

  • Uniform data with 1e5 keys, where the study uses Zipf(1.1) over 1e4 keys or Pareto.
  • Configs CMS 512–2048, KLL k=200/500, HLL 12/14, where the study uses CMS 256–16384, KLL 50/200/800, HLL 12/14/16.
  • It has no cms-heap-topk heap rows and no dd or exact-* rows.

  --atomic-costs accepts it without complaint.

5. Local-only behaviour

A local sketch-bench change, not on main, adds an opt-in RQE_CLAMP_BELOW_FIRST_N to saturation.rs::curve_error_at. It returns the first point's error below the curve's first N instead of None. Eval runs with that variable set see small-N accuracy that ASAPQuery's git-pinned build never does, and ASAPQuery reports those RAQEs as unservable.

To decide / do

• Bump the pin (cargo update -p rqe-optimizer) and fix any API breakage.
• Choose the objective: call minimize_usage_cost (matches the eval) or keep minimize (enforces the per-RAQE latency SLA). Or make the eval report minimize too.
• Decide whether ASAPQuery should adopt coverage-aware accuracy (worse_read) or the eval should drop it for ASAP.
• Guard --atomic-costs against the greedy atomic_costs.json, or document that it is not an input.
• Land RQE_CLAMP_BELOW_FIRST_N in sketch-bench, or drop it.