refactor(asap-aware-mapping)!: delete the legacy replacement search and CostModel

[zzylol]

Why

The planner runs Stage 0 → Pass 1 → Pass 2 → Stage 2 → Stage 3 (docs/design_docs/proposals/planner-layering.md). It no longer calls the legacy replacement search or the CostModel the search ranked with. That legacy core is about 21k lines that every refactor still has to compile around, and it blocks the crate layout (G3, #572).

The rule is to delete a legacy piece only once the stages implement it. Most of the core is replaced. Some of it is not (see the table). The user approved deleting the whole legacy core anyway, as an explicit exception, on 2026-10-16. The capabilities it loses are tracked in #735, #703 (Hydra) and #681 (maintained-population top-k).

What

• Deleted modules of asap-aware-mapping:

  • replacement.rs: ASAPStrategies and its realization engine, CandidatePlanningInputs, new_with_planning_inputs*, nested_layers_target, the workload search, CandidateLogicalASAPDAGs / GlobalSelection, SharedSubDAGStrategy, Matcher, retain_exact, posterior_aware_size_params, sketch_state_bytes;
  • the strategies: grouping.rs (HydraGroupingStrategy), rollup.rs, rewrite.rs, topk_reuse.rs, exact_composition.rs, accuracy/reconciliation.rs;
  • explanation.rs;
  • cost_model.rs: CostModel, DefaultCostModel and the CSE share/recompute and exact-composition cost hooks;
  • recurrence.rs: the legacy decide and its rate types.

  feat(asap-aware-mapping)!: price sketch nodes from measured operation costs in Stage 3 #734 already deleted the physical-plan and empirical cost models, which were built on CostModel.

• Public interfaces trimmed, each because only the deleted code read it:

  • PlanningModels.cost and with_cost; PlanningModels::new takes (accuracy, evidence);
  • AccuracyModel::exact_operation_rule;
  • AccuracyEvidenceProvider::{topk_max_distinct_items, propagation_stats};
  • PropagationStats::{hydra_shared_grid_collision_bound, hydra_shared_grid_failure_probability};
  • realization::RealizationError;
  • the internal FunctionRules.accumulator field;
  • OperandDomain::supports_ddsketch;
  • MaintainedPopulationStrategy's ReplacementStrategy impl. candidate stays: executor and frontend tests build population DAGs with it, and Offer maintained-population top-k candidates in Pass 1 #681 moves it into Pass 1.

• Not changed: what the stages plan. Candidate sets, selections and costs are the same; Examples 1–4 and stage_pipeline pass unchanged.

• Tests: tests of deleted code go with it. Tests of capabilities the stages have are moved onto the stage code: 24 new or extended tests, and three integration-test files that now drive the stages instead of the search.

How

Every remaining reference was a type, a re-export or a doc link. PlanningModels held a &dyn CostModel that no stage read. AccuracyModel named ExactOperation in a method only ExactCompositionStrategy called. function_rules returned a struct whose accumulator half only the legacy engine read; it becomes approximate_input_composition(intent) -> Option<CompositionOperator>. realize_keyed_additive_summary_input returned a three-way result whose "unsupported" reason only the legacy rule table read; it returns Option<PhysicalSummaryInput>, with the reasons kept as comments.

The integration tests promql_to_post_asap.rs, sql_to_post_asap.rs and exact_composition.rs used ASAPStrategies::replacements and global_selection. They now take a Stage 1 logical candidate (planning::logical_candidates), or Stage 3's selection when they need evidence. Assertions follow the stages' shapes:

• Stage 1 restores names with a Project;
• a state column is named state;
• an exact max over an estimate is an exact Max state;
• a join predicate keeps its written orientation.

Where the stages behave differently by design, the test pins the new behaviour or is dropped. Examples: an unsupported arm of a binary op stays exact while the other arm keeps its accumulator, and on(job) matching keeps both accumulators.

Capability → replacement

Legacy capability	Replaced by (file: function)
Per-intent summary families and exact accumulators (realizations_for_intent, sketch_realizations)	per_computation_candidates.rs: computation_candidates_for_intent; realization.rs: summary_candidates
Sizing (CostModel::size_params)	realization.rs: default_size_params → accuracy::estimators::size_params; Pass 1 sized
Recursive realization, retain_exact, construct_summary_*, finalize_*, evaluation, compose_guarantee	per_computation_candidates.rs: substitute_measure, compose_logical_candidate, statistic, composed_guarantee / recompose_guarantees; target check in stage3_plan_selection.rs: accuracy_check
Summary inputs (value/frequency, keyed heap, current series)	per_computation_candidates.rs: measure_input, summary_update, whole_expression_input; realization.rs: realize_keyed_additive_summary_input
Budget split across nested layers (nested_layers_target, per_input_target)	per_computation_candidates.rs: budget_shares + sized; δ in stage3_plan_selection.rs: resize_for_failure_probability
avg_over_time and avg as sum/count (realize_temporal_average, AvgToSumOverCountStrategy)	per_computation_candidates.rs: ExactAccumulator([Sum, Count]) in substitute_measure / sql_avg
sum∘sum fusion (SemanticEquivalentRewriteStrategy)	per_computation_candidates.rs: add_nested_sum_targets / nested_sum
Exact top-k over per-series values	Pass 1 exact choice + stage2_physical_asap_optimization.rs: implement_topk
Binary ops over realized operands; DDSketch quantile ratio	Pass 1 rewrite + arithmetic_guarantee; stage3_plan_selection.rs: domain_bound with the division operand domains
Shared sub-DAG CSE (SharedSubDAGStrategy)	asap_aware_cse.rs: identical_expression_pass2_candidates
One summary sized for the strictest reader (AccuracyReconciliationStrategy, strictest_sibling_accuracy)	asap_aware_cse.rs: summary_capability_pass2_candidates; accuracy::estimators::size_for_all_readers
Group-by rollup (RollupStrategy, is_legal_rollup_source)	asap_aware_cse.rs: rollup_candidates, rollup_pass2_candidates
Top-k limit reuse (TopKLimitReuseStrategy)	asap_aware_cse.rs: topk_reuse_candidates, topk_reuse_pass2_candidates
Read-time exact fold over a summary evaluation (ExactCompositionStrategy, query time)	per_computation_candidates.rs: compose_logical_candidate
Search, global selection, ranking (search_workload*, GlobalSelection, rank_group, CostModel::{rank_candidates, estimate_cost, candidate_cost})	pass/stage1_to_stage3.rs: run_stage1_to_stage3; stage3_plan_selection.rs: select_plan
CSE share vs recompute and recurrence decisions (cse_* hooks, recurrence::decide, RecurrenceProfile, CostRate, Horizon)	stage2_physical_asap_optimization.rs: materialization_assignments; stage3_plan_selection.rs: computations, price, evaluation_rate
Exact-composition cost rates	stage3_plan_selection.rs: price
Replacement explanations (explain_replacements)	per-stage reasons: PruneReason, PrunedCandidate, Rejection.reason; devtools stage_pipeline
Matcher (trait, no impl)	asap_types::post_asap::FamilyCapability::answers; DeploymentCapabilities (#730)
CostModel::summary_support_evidence	already replaced by DeploymentCapabilities.supported_summaries (#730)
Evidence-backed physical costing (PhysicalPlanCostModel), measured costs (EmpiricalCostModel), ResourceCalibration	already deleted by #734 (stage3_plan_selection::measured_operation_costs); the rest is tracked in #733

Not replaced (user-approved exception, 2026-10-16):

Lost capability	Tracked in
Hydra grouping (HydraGroupingStrategy)	#703 (G7b)
Maintained-population top-k as a candidate (MaintainedPopulationStrategy in the search)	#681
Exact operation maintained at ingestion under a summary (OperationPlacement::Maintenance, value-operation hooks)	#735 item 1
Extension intents (CostModel::realize_extension / evaluation_extension)	#735 item 2, #548
sum∘count, min∘min, max∘max fusion	#735 item 3
Summary maintained over finalized per-window rates	#735 item 4
Reset-aware counter proof for a CMS top-k over a rate	#735 item 5
Top-k membership guarantee from a distinct-items bound	#735 item 6
UnivMon for an exact count	#735 item 7
512 MiB sketch-state cap	#735 item 8
Expected-case sizing (posterior_aware_size_params, unwired)	#735 item 9
No checked division inside a maintained state (maintenance_exact_values)	#735 item 10
DDSketch indexable-bounds and max_samples check (supports_ddsketch)	#735 item 11

Stays, and why:

• PlanningModels.accuracy and .evidence: Stage 3 reads them (accuracy.satisfies, evidence.operand_domain, evidence.output_guarantee).
• MaintainedPopulationStrategy::candidate: see What (Offer maintained-population top-k candidates in Pass 1 #681).
• storage_io.rs, physical_handoff_cost.rs and query_physical_lowering.rs: unwired, with their own tests, kept for Plan selection: price storage I/O, handoffs and cache, from deployment evidence with provenance #733.
• PropagationStats' topk_* fields: the TopKSelection composition rule reads them. Nothing sets them since propagation_stats is gone (Capabilities lost with the legacy replacement search: maintenance-time exact operations, extension intents, nested folds, rate-maintained summaries and guards #735 item 6).

Key code interfaces added / modified

crates/asap-aware-mapping/src/pass/mod.rs

/// The models and settings the planner stages read. The type stays; this PR removes `cost`.
pub struct PlanningModels<'a> {
    pub cost: &'a dyn CostModel,                                   // removed: no stage read it
    /// Decides whether a candidate's accuracy guarantee meets its query's target; built-in DefaultAccuracyModel by default.
    pub accuracy: &'a dyn AccuracyModel,                           // unchanged
    /// Deployment facts the guarantees need (operand domains, observed guarantees); NoAccuracyEvidence by default.
    pub evidence: &'a dyn AccuracyEvidenceProvider,                // unchanged
    /// The deployment's plan selection settings: cost model, resource speed, non-repeating query horizon.
    pub plan_selection: &'a PlanSelectionSettings,                 // unchanged
    /// The summaries the executor supports and what it retains; default DeploymentCapabilities::default.
    pub deployment_capabilities: &'a DeploymentCapabilities,       // unchanged
}

impl<'a> PlanningModels<'a> {
    /// These accuracy and evidence models with the default plan selection settings and deployment.
    pub fn new(accuracy: &'a dyn AccuracyModel, evidence: &'a dyn AccuracyEvidenceProvider) -> Self; // was new(cost, accuracy, evidence)
    pub fn with_cost(self, cost: &'a dyn CostModel) -> Self;         // removed
}

crates/asap-aware-mapping/src/accuracy/mod.rs, accuracy/evidence.rs

/// The deployment-extensible accuracy algebra. The trait stays; this PR removes one method.
pub trait AccuracyModel {
    fn exact_operation_rule(&self, operation: &ExactOperation) -> Option<CompositionOperator>; // removed: only ExactCompositionStrategy called it
    /// The guarantee of reading `query` out of a summary of `family` over an exact input.
    fn local_guarantee(&self, family: &FieldDataType, query: &SketchStatistic) -> Option<AccuracyGuarantee>; // unchanged
    /// Compose input guarantees with a local one under `op`; Err when no rule applies.
    fn propagate(&self, op: &CompositionOperator, inputs: &[AccuracyGuarantee], local: Option<&AccuracyGuarantee>, stats: &PropagationStats) -> Result<AccuracyGuarantee, AccuracyError>; // unchanged
    /// Whether `guarantee` meets `target`; read by Stage 3's accuracy check.
    fn satisfies(&self, guarantee: &AccuracyGuarantee, target: &AccuracyTarget) -> bool; // unchanged
}

/// Typed planning-time evidence. The trait stays; this PR removes two methods.
pub trait AccuracyEvidenceProvider {
    fn topk_max_distinct_items(&self, expression: &OperatorNode) -> Option<u64>; // removed: only the legacy weighted top-k read it
    /// Enforced domain of `operand`'s true values; Stage 3 binds products, quotients and DDSketch ratios with it.
    fn operand_domain(&self, operand: &OperatorNode) -> Option<OperandDomain>; // unchanged
    /// The deployment's empirical accuracy for a candidate's whole output; Stage 3 accepts an unbounded output only with it.
    fn output_guarantee(&self, output: &QueryRoot) -> Option<AccuracyGuarantee>; // unchanged
    fn propagation_stats(&self, op: &CompositionOperator, family: &FieldDataType, query: Option<&SketchStatistic>) -> PropagationStats; // removed: only the legacy engine called it
}

/// Statistics a propagation rule may consult; every field defaults to unknown. The type stays.
pub struct PropagationStats {
    /// Certified ranges of a quotient's true numerator and denominator.
    pub division_operand_domains: Option<[OperandDomain; 2]>,      // unchanged
    /// Provenance of supplied evidence.
    pub evidence_provenance: Vec<GuaranteeSource>,                 // unchanged
    /// Whether every input value is non-negative.
    pub values_non_negative: Option<bool>,                         // unchanged
    /// The statistic an unknown sign is named by.
    pub sign_statistic: Option<String>,                            // unchanged
    /// Rows an exact aggregation folds.
    pub input_row_count: Option<u64>,                              // unchanged
    /// The statistic an unknown row count is named by.
    pub row_count_statistic: Option<&'static str>,                 // unchanged
    /// Lower confidence bound of the kth selected top-k item.
    pub topk_selected_lower_bound: Option<f64>,                    // unchanged
    /// Greatest upper confidence bound among excluded top-k items.
    pub topk_excluded_upper_bound: Option<f64>,                    // unchanged
    /// Union-bound failure probability of the margin certificate's intervals.
    pub topk_interval_failure_probability: Option<f64>,            // unchanged
    pub hydra_shared_grid_collision_bound: Option<f64>,            // removed: only HydraGroupingStrategy read it
    pub hydra_shared_grid_failure_probability: Option<f64>,        // removed: only HydraGroupingStrategy read it
}

impl OperandDomain {
    pub(crate) fn supports_ddsketch(&self, alpha: f64) -> bool;    // removed: only the legacy ratio realization read it (#735 item 11)
}

crates/asap-aware-mapping/src/lib.rs, realization.rs, maintained_population.rs, function_rules.rs

// removed modules: replacement, grouping, rollup, rewrite, topk_reuse, explanation,
// exact_composition, accuracy::reconciliation, cost_model, recurrence, with every re-export of them
// (ASAPStrategies, search_workload*, CandidateLogicalASAPDAGs, GlobalSelection, CostModel,
// DefaultCostModel, HydraGroupingStrategy, explain_replacements, RecurrenceProfile, ...).

/// Errors of the legacy construction path. The type is removed: only the legacy engine returned it.
pub enum RealizationError {
    Schema(SchemaDerivationError),                                  // removed
    Accuracy(AccuracyError),                                        // removed
    PhysicalRealization(&'static str),                              // removed
    ExecutionDataState(ExecutionDataStateError),                    // removed
    ExactOperationSchema(ExactOperationSchemaError),                // removed
}

/// The input columns a summary of `intent` reads; None when a tuple leg is outside the schema.
pub(crate) fn summarised_input(intent: &AggIntent, child_schema: &Schema) -> Option<SummaryInputExpr>; // was Result<_, RealizationError>
/// The keyed heap update for TopK(Count/Sum GROUP BY key) over the raw stream; None when it does not apply or is unsupported.
pub(crate) fn realize_keyed_additive_summary_input(intent: &AggIntent, family: &FieldDataType, output_reduction: &Reduction, child: &Rc<OperatorNode>) -> Option<PhysicalSummaryInput>; // was PhysicalSummaryInputRuleResult (removed)

/// The composition operator that propagates an approximate input's guarantee through `intent`; None fails closed.
pub(crate) fn approximate_input_composition(intent: &AggIntent) -> Option<CompositionOperator>; // was function_rules(intent) -> Option<FunctionRules { accuracy, accumulator }>

/// Builds the maintained-population sub-DAG of one root. The type stays.
pub struct MaintainedPopulationStrategy {
    roots: Vec<Rc<OperatorNode>>,                                   // unchanged (private): the workload roots a population is shared across
}
impl MaintainedPopulationStrategy {
    pub fn new(roots: &[Rc<OperatorNode>]) -> Self;                          // unchanged
    pub fn candidate(&self, root: &Rc<OperatorNode>) -> Option<Rc<OperatorNode>>; // unchanged
}
impl ReplacementStrategy for MaintainedPopulationStrategy {             // removed: the trait is deleted
    fn matches(&self, target: &TargetSubDAG<'_>) -> bool;                    // removed
    fn replacements(&self, target: &TargetSubDAG<'_>) -> Vec<ReplacementSubDAG>; // removed
}

Usage (end to end, run at the head commit): a deployment with a DDSketch ratio query and an enforced value range plans with the two-argument PlanningModels::new.

// The deployment's evidence: every latency sample lies in [1, 100]. Stage 3 reads it to bound the ratio.
struct LatencyBetween1And100;
impl AccuracyEvidenceProvider for LatencyBetween1And100 {
    fn operand_domain(&self, _: &OperatorNode) -> Option<OperandDomain> {
        Some(OperandDomain { lower: 1.0, upper: 100.0, max_samples: 1000, contract: "source enforces [1, 100]".into() })
    }
}
// The query as the user writes it, lowered at ε = 0.01: the IR Stage 1 plans.
let query = lower_promql("quantile_over_time(0.9, latency[5m]) / quantile_over_time(0.5, latency[5m])",
                         AccuracyTarget::max_error(0.01))?;
// Its workload entry: a 5% relative-error target, checked by Stage 3.
let entry = QueryWorkloadEntry {
    requirements: QueryRequirements {
        accuracy: AccuracyTarget::max_error(0.05).accuracy_metric(AccuracyMetric::Relative),
        max_latency_ms: None,
    },
    ..Default::default()
};
// The planner's models: built-in accuracy algebra plus the deployment's evidence (no cost model argument any more).
let models = PlanningModels::new(&DefaultAccuracyModel, &LatencyBetween1And100);
// Stage 1 → Stage 3 over the one query: 15 logical and 15 physical candidates.
let run = run_stage1_to_stage3(vec![(0, QueryRoot::Operator(query))], &[entry], &DataWorkload::default(), models)?;
// Stage 3's selection: P15, a DDSketch ratio. Accepted lines read, e.g., "P15: metric relative; accepted relative (DDSketch)".
// P2, P4–P9 and P11 are rejected: "the output has no guarantee, and the deployment has no evidence that it meets target ...".
let selection = run.stage3_plan_selection?;

Before this PR

The planner already planned only through the stages. Next to them sat the legacy search, its strategies and the CostModel trait. Only their own tests and three integration-test files reached them. PlanningModels carried a cost: &dyn CostModel that nothing read. AccuracyModel and AccuracyEvidenceProvider had methods that only the legacy engine called.

After this PR

asap-aware-mapping holds the stages and the models they read. PlanningModels::new(accuracy, evidence) has no cost model. Planning results are unchanged. The capabilities only the legacy core had are listed in #735, #703 and #681.

Also deleted because only the legacy core used them: asap_types::post_asap::lift_plain, ExactOperationSchemaError, and the test helpers test_support::agg_per_entity and time_and_export.

Verification

Run on the rebased head, on top of main de19eba (#737):

• cargo fmt --all -- --check: clean.
• cargo clippy --all-targets --all-features --locked -- -D warnings on asap-aware-mapping, asap-integration-tests, asap-types, asap-planner and asap-devtools: clean. cargo check --tests on asap-physical-operators, asap-frontend-promql and asap-frontend-sql: clean.
• asap-aware-mapping, the touched modules (accuracy, realization, function_rules, per_computation_candidates, maintained_population, analytical_cost, pass, asap_aware_cse) plus tests/cost_independence.rs and tests/per_computation_candidates.rs: 225 passed.
• asap-integration-tests, promql_to_post_asap, sql_to_post_asap, exact_composition, stage1_logical_asap_optimization, planner_layering_example1–4, materialization_e2e and operator_design_examples: 135 passed. Examples 1–4 pin the selections and costs, unchanged.
• asap-planner e2e_plan, plan_stages and summary_sharing, and asap-devtools stage_pipeline: 42 passed.
• Frontend and executor tests that build population DAGs or check UnivMon candidates (maintained_population, maintained_population_horizon, univmon_candidates, current_series_heap, deployment_computation, precompute_candidates): 37 passed.
• asap-types ir::cse, workload and query_time: 63 passed.

Not run, per the current rule: workspace clippy and the full workspace suite (GitHub CI runs them after merge).

Found while moving the tests. These are not fixed here: they are pre-existing and this PR changes no planning behaviour.

• avg without (instance) (m) composes with a closed schema while the pre-ASAP schema is open.
• On rows that carry the series-identity column, whole_expression_input refuses grouped top-k fusion because the schema is closed.
• The sum/count rewrite of SQL avg over an Int64 column does not compile physically (summary update type is unsupported by its family).

Old drafts this PR fully supersedes: #635 (delete the legacy search and CostModel). #622 (delete the physical-plan cost model) is superseded by #734.

[zzylol]

If you remove the CostModel, where are my defined 3 cost models,and how are they used by physical optimization stage?

[zzylol]

Your three cost models are untouched. #736 removed a different, older thing with the same name.

What was removed: the legacy pub trait CostModel in cost_model.rs, with DefaultCostModel. It belonged to the old replacement search. It ranked candidates, sized sketches and made CSE share/recompute decisions, and nothing on the stage pipeline used it any more.

Where your three cost models are: in Stage 3, crates/asap-aware-mapping/src/stage3_plan_selection.rs, added in #723:

pub enum CostModel {
    UsageOverTime(ResourceUnitCost),        // cost model 1
    ProvisionedCapacity(ResourceUnitCost),  // cost model 2
    ClusterCapacity(ClusterLimits),         // cost model 3
}
pub struct PlanSelectionSettings {
    pub cost_model: CostModel,              // default: UsageOverTime(ResourceUnitCost::illustrative())
    pub resource_speed: ResourceSpeed,
    pub non_repeating_query_horizon_seconds: f64,
    pub measured_operation_costs: ...,
}

The deployment picks one through PlanningModels::with_plan_selection(&settings), or stage_pipeline --cost-model ….

How the stages use them:

• Stage 2 (physical ASAP optimization) never reads a cost model. It only enumerates the legal physical candidates: each summary build NotMaterialized, MatAtIngestion, MatAtQuery or MatAtQueryKeptAcrossEvaluations. It does this without ranking them, so the same Stage 2 output can be judged under any of the three models.
• Stage 3 (plan selection) measures each physical candidate once as a ResourceUsage, in CPU ops/s, scanned bytes/s, average and peak memory, disk, peak cores and per-query latency. Then select_plan applies settings.cost_model:
  • cost model 1 prices usage over time;
  • cost model 2 prices the peaks;
  • cost model 3 enforces the cluster limits and picks by ClusterObjective, or returns the Pareto front.

So the physical-optimization stage feeds the cost models but doesn't use them. That keeps Stage 2 independent of how a deployment pays.