Skip to content

test: select plans through plan_stages instead of global_selection - #629

Closed
zzylol wants to merge 1 commit into
stack/cleanup-4-ir-binsfrom
stack/cleanup-5-port-selection-tests
Closed

zzylol wants to merge 1 commit into
stack/cleanup-4-ir-binsfrom
stack/cleanup-5-port-selection-tests

Conversation

@zzylol

@zzylol zzylol commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Stack: #574 → #620 → #618 → #621 → #627 → #625 → #628 → #632 → #634 → #616 → #617 → #622 → #624 → #629 → #630 → #631 → #633 → #635 → #636 → #637

Problem

Many tests used the legacy candidate_selection::global_selection (plus assemble_selected_dag) only to get a selected DAG to execute or inspect (Q52). That kept the legacy selection alive and tested plans the planner no longer selects.

Changes

These tests now take the plan the #509 stage pipeline selects (plan_stages, one root, run once over a continuously ingested source, with the built-in models). The selection goes through a small selected_dag(root, accuracy) helper in each test crate's existing shared module:

  • executor/tests/common (reference-executor capabilities)
  • integration-tests/tests/executor_models
  • frontend-promql/tests/support.rs
  • the cfg(test) fixture in executor/src/physical_planner/candidates.rs, which calls plan_stages inline

Ported: executor deployment_computation, planspace_series_identity_heap, precompute_candidates, promql_fallback, tumbling_pane_merge, physical_planner/candidates.rs; frontend-promql count_planning, univmon_candidates; integration-tests operator_sharing, precompute_raw_samples, sql_to_physical; planner summary_sharing.

Legacy search calls (search_workload*, ASAPStrategies) that these files use for other assertions are untouched. They belong to the legacy-search deletion (Q52 step 7).

Assertions that only held for legacy behaviour

  • deployment_computation::per_series_average_divides_stored_sum_by_count: Stage 1 has no avg → sum/count rewrite (that was the legacy AvgToSumOverCountStrategy), so avg_over_time now selects the raw PromQL fallback. The test now plans sum_over_time(m[5m]) / count_over_time(m[5m]), which still divides stored per-series sum and count states and drops the name. The overflow assertion is dropped: it checked the avg accumulator's checked division, which a Stage 1 plan never reaches (inf / n is inf, not an error).
  • promql_fallback::histogram_quantile_selection_keeps_the_exact_fallback: the legacy search kept histogram_quantile whole. Stage 3 keeps it exact (no sketch, still asserted for both targets), but under sum by (le, job) it selects an exact Sum accumulator below the quantile. This whole-expression fallback harness can't bind that plan, so only the raw-bucket query is executed (still 1.75).
  • count_planning / univmon_candidates: "the chosen candidate never has missing accuracy evidence" becomes "the selected plan contains no Hydra state" and "no UnivMon state", the families whose candidates lacked evidence.
  • operator_sharing::each_side_of_union_all_holds_a_summary_estimate: now #[ignore]d with the same reason as the other Pass 2 sharing and Stage 1 coverage parity with the retired MajorPass #580 ignores. Stage 3 selects the raw plan, because query-time summaries never cost less until Stage 2 plans materialization.
  • planner/tests/summary_sharing::certified_frequency_evaluations_share_one_univmon_state: deleted. It ran the legacy search and selection with a UnivMon-preferring cost model, and its stage-pipeline counterpart frequency_moments_share_one_univmon_in_the_stage_pipeline already covers the behaviour. PREFER_UNIVMON goes with it.

Stacked on #624.

Test plan

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace: 1610 passed, 0 failed, 14 ignored (after the rebase on main d4869a7; was 1607 passed, 14 ignored)

@zzylol
zzylol force-pushed the stack/cleanup-5-port-selection-tests branch from 67ba774 to d5313d6 Compare October 5, 2026 03:07
@zzylol
zzylol force-pushed the stack/cleanup-4-ir-bins branch from 45fde63 to 4761c3e Compare October 5, 2026 03:07
@zzylol
zzylol force-pushed the stack/cleanup-4-ir-bins branch from 4761c3e to 0ba3310 Compare October 5, 2026 06:21
@zzylol
zzylol force-pushed the stack/cleanup-5-port-selection-tests branch from d5313d6 to 01d3e54 Compare October 5, 2026 06:21
@zzylol
zzylol force-pushed the stack/cleanup-5-port-selection-tests branch from 01d3e54 to d53dbf7 Compare October 8, 2026 16:47
@zzylol
zzylol force-pushed the stack/cleanup-4-ir-bins branch from 0ba3310 to c47d1f1 Compare October 8, 2026 16:47
@zzylol
zzylol force-pushed the stack/cleanup-5-port-selection-tests branch from d53dbf7 to e16b054 Compare October 10, 2026 14:32
Tests that used the legacy global_selection only to obtain a selected
DAG to execute or inspect now take the plan the #509 stage pipeline
selects, through a small per-crate selected_dag helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the stack/cleanup-4-ir-bins branch from 52190e0 to e23bb15 Compare October 11, 2026 17:11
@zzylol
zzylol force-pushed the stack/cleanup-5-port-selection-tests branch from e16b054 to 2ef57fe Compare October 11, 2026 17:11
zzylol added a commit that referenced this pull request Oct 13, 2026
…e MajorPass (#674)

* feat(planner): run the planning stages as the planner pipeline; retire MajorPass

plan_stages runs logical ASAP-aware optimization (Pass 1 and composition
of every combination, capped), physical ASAP-aware optimization and plan
selection over the frontends' roots, and returns every stage's output.
StagePipeline runs it behind OptimizationPass and is the facade default;
the stage_pipeline devtool uses it (its fixture is unchanged). The
combination count saturates instead of overflowing.

summary_sharing: sharing tests are ignored until Pass 2 (#580); the
UnivMon legacy-search test is deleted; scalar and combined-quantile tests
run on plan_stages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): logical and physical ASAP-aware optimization import no cost model

From #582: only plan selection prices plans, so the Pass 1 / composition
and physical ASAP-aware optimization modules may not import cost_model,
recurrence, analytical_cost, physical_operator_statistics or
plan_selection (paths or re-exported names) outside tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): compose a per-series avg over rows that carry the series identity

The per-series sum keeps the input's labels, so with
$promql_series_identity after the sample value its last column is the
identity, and composition rejected the only exact option ("non-Float64
per-series avg"). The planner pipeline then had no candidate for
avg_over_time. Read the summed value by its role instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(executor): take plans and candidates from the planner pipeline

From #629/#633: executor tests take the plan plan_stages selects
(selected_dag) or its composed logical candidates (logical_candidates)
instead of the legacy search and global selection.

- deployment_computation, precompute_candidates, the candidates.rs
  fixture: ported.
- planspace_series_identity_heap: keeps the selection check; the four
  tests of the legacy whole-root proposals are deleted.
- weighted_topk_binding: ported to Pass 1 heap candidates (Count Sketch
  only over rates: no sign proof). The tests of legacy-only candidate
  generation are deleted. Ignored, with reasons: the grouped weighted
  binding (whole-expression heap, #589) and the two rate-heap executions
  (top-k output schema, #672).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(frontend-promql): take plans and candidates from the planner pipeline

From #629/#633, on support helpers over plan_stages (plan_stages,
selected_dag, logical_candidates, logical_plan):

- count_planning: exact counts select a Count accumulator; count
  updates are read from the composed Count-Min / Count Sketch
  candidates. The Hydra test is ported and ignored: Pass 1 offers no
  Hydra for a grouped count yet (#605).
- univmon_candidates: the synthetic accuracy model is gone; UnivMon
  candidates come from Pass 1 (the shared count at an approximate
  target), the uncalibrated ones carry no guarantee and plan selection
  does not select them.
- promql_corpus, metrics_observability: totality over Pass 1 and
  composition (logical_plan) instead of the legacy strategies; the
  metrics corpus reports composition errors under
  METRICS_OBSERVABILITY_REPORT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(frontend-sql): take corr's exact plan from the planner pipeline

From #633: the plan plan_stages selects replaces retain_exact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): the batch SUM example reads one exact accumulator per query

Under the planner pipeline each query's SUM is replaced by its exact
accumulator state; sharing identical producers is Pass 2 (#580).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(planner): stage identifiers carry number and name; enumerate every candidate

User decisions (2026-10-13):
- stage modules stage2_physical_asap_optimization and
  stage3_plan_selection; tests stage1_logical_asap_optimization.rs and
  stage2_physical_asap_optimization_sql.rs; stage_pipeline keys
  stage0_query_language_frontends, stage1_logical_asap_optimization,
  stage2_physical_asap_optimization, stage3_plan_selection (fixture
  regenerated: only the keys change);
- enumeration and selection are never capped: plan_stages enumerates
  every combination; the devtool's --max-displayed (was
  --max-candidates) caps only the listing;
- code docs say "query-language frontends".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(integration): take plans and candidates from the planner pipeline

From #629/#633, on asap_integration_tests::planning (plan_stages,
selected_dag, logical_candidates):

- sql_to_physical, operator_sharing, materialization_e2e,
  precompute_raw_samples, summary_coverage_e2e,
  promql_numeric_regressions: ported.
- cse.rs: deleted; it pinned the legacy search's memo groups (sharing
  identical sub-DAGs is Pass 2, #580).
- Ignored, with reasons: the UNION ALL summary test (plan selection picks
  the exact plan until materialization, #604), arithmetic over one
  selector (#580), the unit-weight heap over raw samples (#589).
- precompute_raw_samples no longer requires a Count Sketch heap over raw
  samples (#589); materialization_e2e takes the quantile's sketch
  candidate, since plan selection picks the exact plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(devtools): sketch_coverage reports Pass 1's summary candidates

From #633: per lowered query, the summary candidates Pass 1 offers,
instead of the legacy explain_replacements. The CSE-shareable metric is
dropped (sharing is Pass 2). Over every corpus at epsilon 0.01 and a 1 s
ingestion interval: 145/2183 lowered queries get a summary candidate, and
Pass 1 rejects none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(frontend-promql): the Kubernetes mixin aggregates over topk compose (ignored, #672)

Both corpus queries that do not compose aggregate over a topk output,
which the frontend declares as [keys.., topk_k] with a Utf8 placeholder
and no value column; the outer sum's accumulator has no value to read.
They compose once the top-k output carries the ranked value (#672).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(devtools): run_planning_stages runs the library's plan_stages

One pipeline implementation: the devtool's run_planning_stages (#577)
validates the frontends' roots and every composed candidate, and
otherwise is plan_stages with the built-in models. Its candidate types
are the library's. The Example 1 fixture is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol

zzylol commented Oct 13, 2026

Copy link
Copy Markdown
Contributor Author

Closing as part of the upper-stack consolidation (G1): its ports of the legacy callers onto the planner pipeline are in #674. Closed with the author's approval.

@zzylol zzylol closed this Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant