Repository navigation
Conversation
This was referenced Oct 5, 2026
zzylol
force-pushed
the
stack/cleanup-7-stage1-helpers
branch
from
October 5, 2026 06:21
a35b447 to
c41b508
Compare
zzylol
force-pushed
the
stack/cleanup-8-port-legacy-callers
branch
from
October 5, 2026 06:21
94a6d35 to
b0b37f1
Compare
This was referenced Oct 5, 2026
Closed
zzylol
force-pushed
the
stack/cleanup-8-port-legacy-callers
branch
2 times, most recently
from
October 10, 2026 14:32
cf6c3a0 to
f5d217c
Compare
…ge 1 The sketch_coverage devtool and the frontend, integration and executor tests that drove ASAPStrategies / search_workload / retain_exact now take their candidates from Stage 1 (Pass 1 inventory and composition) or the stage pipeline. Tests that only pinned legacy search behaviour are deleted, among them integration-tests/tests/cse.rs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol
force-pushed
the
stack/cleanup-8-port-legacy-callers
branch
from
October 11, 2026 17:11
f5d217c to
adc3baf
Compare
3 tasks
zzylol
added a commit
that referenced
this pull request
Oct 13, 2026
…e MajorPass (#674) * feat(planner): run the planning stages as the planner pipeline; retire MajorPass plan_stages runs logical ASAP-aware optimization (Pass 1 and composition of every combination, capped), physical ASAP-aware optimization and plan selection over the frontends' roots, and returns every stage's output. StagePipeline runs it behind OptimizationPass and is the facade default; the stage_pipeline devtool uses it (its fixture is unchanged). The combination count saturates instead of overflowing. summary_sharing: sharing tests are ignored until Pass 2 (#580); the UnivMon legacy-search test is deleted; scalar and combined-quantile tests run on plan_stages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(planner): logical and physical ASAP-aware optimization import no cost model From #582: only plan selection prices plans, so the Pass 1 / composition and physical ASAP-aware optimization modules may not import cost_model, recurrence, analytical_cost, physical_operator_statistics or plan_selection (paths or re-exported names) outside tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * fix(planner): compose a per-series avg over rows that carry the series identity The per-series sum keeps the input's labels, so with $promql_series_identity after the sample value its last column is the identity, and composition rejected the only exact option ("non-Float64 per-series avg"). The planner pipeline then had no candidate for avg_over_time. Read the summed value by its role instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(executor): take plans and candidates from the planner pipeline From #629/#633: executor tests take the plan plan_stages selects (selected_dag) or its composed logical candidates (logical_candidates) instead of the legacy search and global selection. - deployment_computation, precompute_candidates, the candidates.rs fixture: ported. - planspace_series_identity_heap: keeps the selection check; the four tests of the legacy whole-root proposals are deleted. - weighted_topk_binding: ported to Pass 1 heap candidates (Count Sketch only over rates: no sign proof). The tests of legacy-only candidate generation are deleted. Ignored, with reasons: the grouped weighted binding (whole-expression heap, #589) and the two rate-heap executions (top-k output schema, #672). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-promql): take plans and candidates from the planner pipeline From #629/#633, on support helpers over plan_stages (plan_stages, selected_dag, logical_candidates, logical_plan): - count_planning: exact counts select a Count accumulator; count updates are read from the composed Count-Min / Count Sketch candidates. The Hydra test is ported and ignored: Pass 1 offers no Hydra for a grouped count yet (#605). - univmon_candidates: the synthetic accuracy model is gone; UnivMon candidates come from Pass 1 (the shared count at an approximate target), the uncalibrated ones carry no guarantee and plan selection does not select them. - promql_corpus, metrics_observability: totality over Pass 1 and composition (logical_plan) instead of the legacy strategies; the metrics corpus reports composition errors under METRICS_OBSERVABILITY_REPORT. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-sql): take corr's exact plan from the planner pipeline From #633: the plan plan_stages selects replaces retain_exact. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(planner): the batch SUM example reads one exact accumulator per query Under the planner pipeline each query's SUM is replaced by its exact accumulator state; sharing identical producers is Pass 2 (#580). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(planner): stage identifiers carry number and name; enumerate every candidate User decisions (2026-10-13): - stage modules stage2_physical_asap_optimization and stage3_plan_selection; tests stage1_logical_asap_optimization.rs and stage2_physical_asap_optimization_sql.rs; stage_pipeline keys stage0_query_language_frontends, stage1_logical_asap_optimization, stage2_physical_asap_optimization, stage3_plan_selection (fixture regenerated: only the keys change); - enumeration and selection are never capped: plan_stages enumerates every combination; the devtool's --max-displayed (was --max-candidates) caps only the listing; - code docs say "query-language frontends". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(integration): take plans and candidates from the planner pipeline From #629/#633, on asap_integration_tests::planning (plan_stages, selected_dag, logical_candidates): - sql_to_physical, operator_sharing, materialization_e2e, precompute_raw_samples, summary_coverage_e2e, promql_numeric_regressions: ported. - cse.rs: deleted; it pinned the legacy search's memo groups (sharing identical sub-DAGs is Pass 2, #580). - Ignored, with reasons: the UNION ALL summary test (plan selection picks the exact plan until materialization, #604), arithmetic over one selector (#580), the unit-weight heap over raw samples (#589). - precompute_raw_samples no longer requires a Count Sketch heap over raw samples (#589); materialization_e2e takes the quantile's sketch candidate, since plan selection picks the exact plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): sketch_coverage reports Pass 1's summary candidates From #633: per lowered query, the summary candidates Pass 1 offers, instead of the legacy explain_replacements. The CSE-shareable metric is dropped (sharing is Pass 2). Over every corpus at epsilon 0.01 and a 1 s ingestion interval: 145/2183 lowered queries get a summary candidate, and Pass 1 rejects none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * test(frontend-promql): the Kubernetes mixin aggregates over topk compose (ignored, #672) Both corpus queries that do not compose aggregate over a topk output, which the frontend declares as [keys.., topk_k] with a Utf8 placeholder and no value column; the outer sum's accumulator has no value to read. They compose once the top-k output carries the ranked value (#672). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW * refactor(devtools): run_planning_stages runs the library's plan_stages One pipeline implementation: the devtool's run_planning_stages (#577) validates the frontends' roots and every composed candidate, and otherwise is plan_stages with the built-in models. Its candidate types are the library's. The Example 1 fixture is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Closing as part of the upper-stack consolidation (G1): its ports of the legacy callers onto the planner pipeline are in #674. Closed with the author's approval. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack: #574 → #620 → #618 → #621 → #627 → #625 → #628 → #632 → #634 → #616 → #617 → #622 → #624 → #629 → #630 → #631 → #633 → #635 → #636 → #637
Rebased:
univmon_candidates.rskeeps #621's assertion that only L2 is certified (RelativeValue), on the Stage 1 candidates.Problem
The legacy search (
ASAPStrategies,search_workload*,explain_replacements,retain_exact) still had callers outside logical-optimizer: one devtool and a set of frontend, integration and executor tests. It cannot be deleted (Q68 (a), step 7c) until they move to Stage 1 or are deleted.Changes
One decision per file. Shared helpers:
stage1_candidates(root)inexecutor/tests/common/mod.rsandfrontend-promql/tests/support.rscomposes every Pass 1 choice for one query (up to 4096) and skips those that do not compose.stage1_plan(root)in the frontend-promql support composes the first summarized choice, or else the pass-through choice.devtools/src/bin/sketch_coverage.rsenumerate_local_logical_candidatesper lowered query. It prints the sketch alternatives Pass 1 offers for each query (a Hydra alternative prints asHydra(<algorithm>)), and per corpus the fraction of lowered queries with at least one. The CSE-shareable metric is dropped: Stage 1's identical-expression sharing is a workload variant, not a per-query replacement. Over every corpus: 135/2178 lowered queries get a sketch alternative at ε = 0.01.frontend-promql/tests/observability/promql_corpus.rs,metrics_observability.rsstage1_plan). Pass 1 must accept every lowered query, and the query must compose to a candidate. Totals: docs 19 summarized / 28 unchanged / 0 errors; testdata 354 / 721 / 0; every metrics-observability corpus 0 errors. A choice can legitimately fail to compose, for example a Top-K heap over a closed schema or a summary over two sources. Selection skips such choices, so the probe does not count them as errors.frontend-promql/tests/count_planning.rsgrouped_count_offers_hydra_but_selection_keeps_per_group_state). Its input carries the series identity: Hydra hashes a non-null item, and PromQL labels are nullable.frontend-promql/tests/univmon_candidates.rsAccuracyModelis gone. The shared count is lowered at an approximate target, because Pass 1 offers no UnivMon for an exact count (the legacy search did). Stage 1 nodes carry no guarantee, so the "no calibrated bound" check usesDefaultAccuracyModel::local_guarantee.integration-tests/tests/promql_numeric_regressions.rsplantakes each target's first alternative other than pass-through; the weighted-heap test takes the heap that absorbs the inner aggregate. Dropped: the node-guarantee assertions (Stage 1 sets none; the ratio-certification gap is listed in #623) and the Top-K evidence provider. The two-metric guarded expression becameavg_over_time(a[5m]) + avg_over_time(a[5m] offset 5m): Pass 1 offers no summary over two sources. A signed sum's Count-Min heap is now checked to carry no non-negative proof, not to be absent; Stage 3 rejects it.integration-tests/tests/precompute_raw_samples.rsintegration-tests/tests/cse.rspass2::identical_expressions's tests.frontend-sql/tests/pearson_corr.rscorr. The exact-guarantee assertion is dropped.executor/tests/weighted_topk_binding.rs#[ignore]), direct per-series rate heap with dynamic labels, maintained rate heap. These run on Count Sketch only, because Stage 1 proves no sign for rate values and the executor rejects a Count-Min heap without one. The evaluation-timestamp assertions are dropped, because Stage 1's heap evaluation outputs no timestamp column. Deleted:physical_binding_does_not_impose_an_accuracy_acceptance_policy(it only varied legacy evidence),spatial_topk_exposes_signed_heap_candidate_over_complete_snapshot(legacycurrent_series_topk_candidatesonly), anddirect_rate_topk_exposes_heap_candidates_with_complete_series_identity(a closed catalog schema with no series identity has no Top-K item in Stage 1).executor/tests/planspace_series_identity_heap.rsselection_never_commits_a_series_identity_heap(already onplan_stages). Deleted the four tests of the legacy search's whole-root proposals.executor/tests/precompute_candidates.rsexecutor/tests/deployment_computation.rscount(up)candidate comes from Stage 1.executor/tests/current_series_heap.rs,deployment_computation.rs(population_dag),precompute_candidates.rs(population_topk_*),frontend-sql/tests/maintained_population.rs,frontend-promql/tests/maintained_population_horizon.rsMaintainedPopulationStrategy::candidatedirectly, not the search. The plan for 7c is to keepcandidateand drop only itsReplacementStrategyimpl.Findings: Stage 1 gaps these ports exposed (not fixed here)
topk by(job)(k, sum by(service, job)(rate(...)))) keeps the outer top-k's partition keys. Those keys index the inner aggregate's output, but the heap is built over the raw rates. Hereby(job)became key 0 =ts, and the output schema losesjob. The legacy search remapped the keys.planner_weighted_topk_binds_at_either_deployment_phaseis#[ignore]d with this reason.Stacked on #631.
Test plan
cargo fmt --all --checkcargo clippy --workspace --all-targets --all-features --locked -- -D warningscargo test --workspace --locked: 1508 passed, 0 failed, 15 ignored (after the rebase on main d4869a7; was 1484 passed, 23 ignored)cargo run -p asap-devtools --bin sketch_coverage -- --data-ingestion-interval-ms 1000runs🤖 Generated with Claude Code