refactor(ir): remove the legacy QueryExpr and SummaryExpr IRs; move docs and DAG viewer to the unified IR

[zzylol]

Why

After the cutover (#542) and the move to the crate roots (#665), the legacy QueryExpr and SummaryExpr IRs have no live consumers, but they are still compiled, documented and drawn by the DAG viewer.

What

• Remove the legacy IRs. The pre-ASAP QueryExpr DAG with its resolver, canonicalizer and CSE, and the post-ASAP SummaryExpr / PostAsapDAG with its CSE and its own validation errors, are deleted: pre_asap/{query_expr, cse, canonicalize, resolve, schema_resolver} and post_asap/{expr, cse, post_asap_dag}. Parameter types (GroupKeys, Reduction, Source, …) move to ir/operator_properties.rs (pre_asap re-exports them), and their users are migrated. QueryExpr paths are dropped from execution_data_state, maintained_population, agg_intent, column_resolution, scalar_type_rules and pre_asap/schema; with_promql_series_identity keeps only its OperatorNode version in ir/schema_support.rs.
• Remove unused API around execution data states: the functions produced_data_state, validate_execution_data_states, validate_execution_data_states_at, assigned_child_data_state, plain_schema and exact_operation_output_schema, the struct ExecutionDataStateAssignment, the enum ExecutionDataStateEdge, the error variant AmbiguousKeepPreAsap, and the unused Source::data_model and ExecutionTiming::is_query_time.
• Docs describe the unified operator IR instead of the removed legacy IRs: export docs point at ir::physical_export (compile_physical_asap_dag, PhysicalASAPDAG, PhysicalASAPDAGDocument) and name the wire version by its constant (PHYSICAL_ASAP_DAG_WIRE_VERSION) rather than a number; physical-compile-coverage.md and planner-vocabulary-migration.md drop the removed Fallback payloads; links to deleted files in the decision docs are retargeted; stale cse / HAVING / post_asap references in code comments are fixed. ASAPQuery backend names (post_asap_readout.rs, ExactReadout) and the old strategy names stay in the migration tables, since they name what callers migrate from.
• DAG viewer uses the unified operator kinds and names OperatorNode. tools/dag-viewer/generate-sample.sh passes --data-ingestion-interval-ms and --default-cost, and dag.example.json is regenerated. crates/devtools/tests/viewer_contract.rs builds one instance of every operator variant and checks the viewer's kinds against their real kind_name() strings.
• SQL collection UDFs. sql/collection_planning.rs becomes collection_udfs.rs (CollectionUdf, CollectionFunction, MAP_UDF_NAME), with a module doc saying these are planning-only DataFusion UDFs for ClickHouse's Map / List / Struct functions that DataFusion never executes (from the feat(sql): lower SQL to unified operator and scalar IR #539 review).

How

Every remaining user of a legacy type is moved to its ir counterpart, then the type is deleted. This is IR and tooling cleanup with no #509 stage logic. Tests that covered behavior still present are ported to the IR: maintenance_binary_accepts_only_well_typed_series_identity (ir::timing), exports_summary_over_summary_as_typed_precompute_edges and physical_node_ids_serialize_in_deterministic_binding_order (physical_export.rs).

No planner behavior change is intended. Two kinds of visible change remain:

• Series-identity rule restored. The migration to ir::timing had loosened the maintenance-binary check, so a column named after the series identity was accepted with a Timestamp type. It must again be non-null Utf8 (InvalidMaintenanceBinary otherwise); the ported test caught this.
• Error messages and error types:
  • ConflictingTiming: "shared node is assigned conflicting timings: {first} and {second}".
  • UntimedNode: adds "; apply lifecycle timings before export" (refactor(planner): replace the summary lifecycle with #509 materialization options #667 changes it to "materialization timings").
  • UnimplementedOperator: "{operator} has no execution timing contract yet".
  • InvalidMaintainedPopulation: says "evaluation" instead of "readout".
  • ReadoutUnderMaintenance is folded into EvaluationUnderMaintenance { edge, child }.
  • ExactOperationSchemaError::{Schema(QueryExprError), Derivation} are merged into Schema(SchemaDerivationError).
  • AmbiguousKeepPreAsap is removed.

Key code interfaces added / modified

crates/types/src/pre_asap/mod.rs

// removed modules: canonicalize, cse, query_expr, resolve, schema_resolver (the legacy pre-ASAP QueryExpr IR)
// removed: pub enum QueryExpr<C: ColState = ColumnId> { /* ... */ }
// removed: pub enum QueryExprError { /* ... */ }
// removed: pub fn canonicalize(expr: QueryExpr) -> QueryExpr
// removed: pub fn share_common_sub_dags<Id>(roots: Vec<(Id, QueryExpr)>) -> Vec<(Id, Rc<QueryExpr>)>
// removed: pub fn structural_hash(node: &QueryExpr, cache: &mut HashCache) -> u64
// removed: pub fn dag_node_count(root: &QueryExpr) -> usize
// removed: resolve_root, ResolveDAGError, SchemaResolver, SchemaCatalog, UsageDerivedCatalog
// removed: resolve_expr, output_schema_for_aggregate (column_resolution)
// removed: struct_field_type, element_access_type (scalar_type_rules)
// removed: pub fn with_promql_series_identity(root: &QueryExpr) -> Result<QueryExpr, String>  (schema; the OperatorNode version in ir::schema_support stays)

/// Operator parameter types, now defined in `ir::operator_properties` and re-exported here so existing
/// `pre_asap::` imports keep working.
pub use crate::ir::operator_properties::{   // was re-exported from query_expr
    AtModifier, BinaryOpKind, ColState, ConcatDiscriminatorKey, DataModel, GroupKeys, GroupSide,
    InfoMatcher, JoinKind, PromQLVectorSetOpKind, Reduction, RelationalSetOpKind, SampleKind,
    Source, TimeShift, VectorGrouping, VectorMatch, VectorMatchKind, WindowFrame, WindowFrameBound,
    WindowFrameOffset, WindowFrameUnits, WindowFuncKind,
};
/// The one canonical output-schema derivation for an `Aggregate`, shared by the IR and HAVING resolution.
pub use crate::ir::aggregate_schema::aggregate_output_schema;   // was from query_expr
/// Resolve name-based `ColumnRef`s to positional `ColumnId`s against a `Schema` (front-end name resolution);
/// `ResolveError` reports unknown, ambiguous, or wildcard references (bad input).
pub use column_resolution::{resolve_column_ref, resolve_column_refs, ResolveError};

crates/types/src/ir/operator_properties.rs

// moved here from pre_asap::query_expr
/// The column-reference type operator parameters are generic over: positional `ColumnId` once bound,
/// name-based `ColumnRef` before.
pub trait ColState: Clone + std::fmt::Debug + PartialEq + Serialize + for<'de> Deserialize<'de> {}
/// A grouping key set: kept labels for `by(...)`, excluded labels for `without(...)` (the kept set is resolved at run time).
pub struct GroupKeys<C = ColumnId> { /* ... */ }
impl<C> GroupKeys<C> {
    /// An empty key set: a global, ungrouped operation.
    pub fn none() -> Self;
    /// Group by exactly these columns.
    pub fn by(keys: Vec<C>) -> Self;
    /// Group by every label except these (PromQL `without`); only `Aggregate` produces this form.
    pub fn without(keys: Vec<C>) -> Self;
    /// Whether this is a `without(...)` exclusion grouping.
    pub fn is_without(&self) -> bool;
    /// The named keys: kept labels for `by`, excluded labels for `without`.
    pub fn keys(&self) -> &[C];
}
/// What kind of computation an `Aggregate` performs, decided once when the node is built rather than
/// inferred downstream from its shape.
pub enum Reduction<C = ColumnId> { /* ... */ }
/// A caller-proven compound unique key for `Concat` output: a per-branch discriminator plus an inner key.
/// Not verified; a wrong claim yields a wrong `unique_keys`.
pub struct ConcatDiscriminatorKey<C: ColState = ColumnId> { /* ... */ }
/// The identity of a `Scan`'s leaf: a time-series metric or a table. The schema rides on `Scan.schema`.
pub enum Source { /* ... */ }
/// Which data model a `Source` or `AggIntent` operates over: time series, tabular, or any.
pub enum DataModel { /* ... */ }
/// PromQL per-selector `offset` and `@` modifiers; moves when a selector is evaluated without changing its schema.
pub struct TimeShift { pub offset_ms: i64, pub at: Option<AtModifier> }
/// PromQL vector-match modifier: `on`/`ignoring` labels plus optional `group_left`/`group_right`.
pub struct VectorMatch { pub kind: VectorMatchKind, pub labels: Vec<String>, pub grouping: Option<VectorGrouping> }
/// A label matcher on the info-metric side of `PromqlInfoEnrich`, applied against the info metric, not the input schema.
pub struct InfoMatcher { pub label: String, pub op: CompareOpKind, pub value: String }
/// A SQL window function's `ROWS`/`RANGE BETWEEN ... AND ...` frame; `GROUPS` is rejected at lowering.
pub struct WindowFrame { pub units: WindowFrameUnits, pub start_bound: WindowFrameBound, pub end_bound: WindowFrameBound }
/* BinaryOpKind, JoinKind, RelationalSetOpKind, PromQLVectorSetOpKind, WindowFuncKind, SampleKind,
   AtModifier, VectorMatchKind, VectorGrouping, GroupSide, WindowFrame{Offset, Units, Bound} */

crates/types/src/post_asap/mod.rs, post_asap/execution_data_state.rs, post_asap/maintained_population.rs

// removed modules: cse, expr, post_asap_dag (the legacy post-ASAP SummaryExpr IR and its export; ir::physical_export replaces it)
// removed: SummaryExpr, SummaryNode, ExactOperation, ValueOperation, CandidateCompleteness, BinaryOperator
// removed: share_common_summary_sub_dags
// removed: PostAsapDAG, PostAsapDAGDocument, PostAsapDAGNode, PostAsapDAGEdge, PostAsapNodeId, PostAsapDAGValidationError
// removed: pub fn compile_post_asap_dag(root: &Rc<SummaryNode>) -> Result<PostAsapDAG, ExecutionDataStateError>
// removed: pub const POST_ASAP_DAG_WIRE_VERSION: u32 = 6
// removed: produced_data_state, validate_execution_data_states, validate_execution_data_states_at,
//          assigned_child_data_state, plain_schema, exact_operation_output_schema,
//          ExecutionDataStateAssignment, ExecutionDataStateEdge, ExecutionTiming::is_query_time

/// A plan-construction-time data-state violation, typed so a strategy can fall back on the variant it expects
/// and tests can assert why a plan was rejected.
pub enum ExecutionDataStateError {
    /// The maintained-population maintenance/evaluation contract is invalid; the plan is rejected.
    InvalidMaintainedPopulation,
    /// A query-time value was placed beneath a maintained summary, which may consume only update-path values
    /// or exact accumulator state. The plan is malformed.
    EvaluationUnderMaintenance { edge: &'static str, child: ExecutionDataState },   // ReadoutUnderMaintenance folded in
    /// Any other edge whose child data state the parent does not accept, e.g. sketch state fed into a query-time operator.
    IllegalChildDataState { edge: &'static str, child: ExecutionDataState },
    /* ... */
    // removed: AmbiguousKeepPreAsap { .. }
}

/// Why an exact operator's output schema could not be derived.
pub enum ExactOperationSchemaError {
    /// The input carries summary state, not plain columns.
    NonPlainInput,
    /// Schema derivation of the operator failed.
    Schema(#[from] SchemaDerivationError),   // was Schema(#[from] QueryExprError) and Derivation(#[from] SchemaDerivationError)
}

// removed: impl MaintainedPopulation<QueryExpr> { pub fn matches_input(&self, input: &QueryExpr) -> bool; pub fn supports(..) }
// removed: impl CurrentSeriesInput { pub fn matches_input(&self, input: &QueryExpr) -> bool }
impl MaintainedPopulation<crate::ir::OperatorNode> {
    /// Whether this maintained population can answer the statistic: quantiles only if enabled and `q` is
    /// finite, TopK only up to `max_k`; Sum, Count and Average always.
    pub fn supports(&self, evaluation: &PopulationStatistic) -> bool;
}

crates/frontend-sql/src/sql/mod.rs, collection_udfs.rs

/// Planning-only DataFusion UDFs so the SQL planner resolves ClickHouse `map`, `mapConcat`, `arrayElement` and
/// `tupleElement`; DataFusion never executes them.
mod collection_udfs;   // renamed from collection_planning: planning-only DataFusion UDFs for ClickHouse Map / List / Struct
/// The name ClickHouse `map(...)` is planned under, since DataFusion reserves `map`; lowering restores `map`.
pub(super) const MAP_UDF_NAME: &str = "asap_map_construct";   // renamed from MAP_PLANNING_NAME
/// Registers the collection UDFs on a DataFusion session before SQL planning.
pub(super) fn register(context: &SessionContext);

tools/dag-viewer/node-style.js

// Operator kinds (`Operator::kind_name`), checked by devtools/tests/viewer_contract.rs
const KIND_CATEGORY_JSON = `{
  "Scan": "data",
  // SQL `VALUES` rows, or the one empty row of a `SELECT` without `FROM`.
  "Values": "data",                      // new
  /* ... */
  "SummaryMerge": "summary",
  // Reads an exact accumulator's state as its finalized value, before query-time operators.
  "FinalizeExactAccumulator": "summary", // new
  // Maintains a full declared population, including membership changes.
  "MaintainPopulation": "summary",       // new
  // Reads an aggregate or TopK prefix from a maintained population.
  "EvaluatePopulation": "summary",       // new
  // A named extension operator over one child.
  "Extension": "summary"                 // new
}`;
// removed kinds: PromqlScalarBridge, EvalTimestamp, CurrentTimestamp, PromqlScalarFromVector,
//                RelationalJoin, KeepPreAsap, SummaryBinaryOp, ValueOperation

Before this PR

Two IRs were compiled side by side; docs and the viewer described the legacy one.

After this PR

asap-types has one IR (ir::OperatorNode, exported as PhysicalASAPDAG), and the docs and the viewer describe it. Callers of the removed functions and error variants listed above must use the ir timing pass and the merged variants.

Evidence

Not applicable (removal; no planner behavior change). crates/devtools/tests/viewer_contract.rs (new) checks that the viewer knows every unified operator kind the export emits.

Verification

• Unit and integration tests: tests that lived in deleted legacy files are removed with them, except the three ported tests above; planner_vocabulary.rs is migrated off SchemaResolver.
• Series identity: maintenance_binary_accepts_only_well_typed_series_identity covers the restored rule (a Timestamp-typed series-identity column is rejected).
• Viewer: crates/devtools/tests/viewer_contract.rs (new; viewer_categorizes_exactly_the_exported_node_kinds) and the tools/dag-viewer render tests.
• Other checks on the PR head: cargo fmt --check; workspace all-target Clippy with -D warnings; cargo test --workspace (1,635 passed, 0 failed, 2 ignored); dag-viewer test_render (31 passed); tools/verify_metricsql_external_consumer.sh OK.

Architectural decisions

• The summary maintenance lifecycle is removed in its own PR (refactor(planner): replace the summary lifecycle with #509 materialization options #667), because replacing it with the Stage 2 materialization options changes when summaries are materialized, which is a product-semantics change, not a cleanup.

Limitations and follow-up

• Next PR (refactor(planner): replace the summary lifecycle with #509 materialization options #667): remove the summary maintenance lifecycle and replace it with the docs: propose workload-wide planning, summary sharing, and materialization #509 Stage 2 materialization options (NotMaterialized / MatAtIngestion / MatAtQuery).
• Reorganize crates and modules by #509 stages and the #511 unified IR #572: arrange asap-types modules by docs: define unified operators and SQL/PromQL scalar boundaries #511 section (pre_asap / post_asap go away).

Stack: tracker #528 · main → #542 → #665 → #666 → #543 → #667 → #561.

Human review — do not complete with an agent

• The MVP boundary is correct.
• New conceptual layers or public interfaces are necessary.
• The before/after description matches the intended product behavior.
• Human reviewer:
• Decision and rationale:

[zzylol]

Follow-up: the module/crate reorganization by #509 stages and the #511 IR (removing pre_asap / post_asap and splitting asap-aware-mapping) is tracked in #572 and starts after this cleanup lands.