Skip to content

feat(ir): derive summary coverage as definition + selection - #646

Merged
zzylol merged 7 commits into
mainfrom
stack/528-02c-coverage-derivation
Oct 11, 2026
Merged

zzylol merged 7 commits into
mainfrom
stack/528-02c-coverage-derivation

Conversation

@zzylol

@zzylol zzylol commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Closes #570.

Why

A summary state's schema says what kind of state it is (KLL, k=200, by job), not what it summarizes. #560 (merged) makes SummaryMerge structurally valid, but structure cannot tell a KLL over latency from one over size, nor show that two inputs share no row. This PR derives what each summary state covers from its sub-DAG and uses it to decide whether a merge is valid. The design is in #573 (§4.2), based on Goldstein & Larson's view matching (SIGMOD 2001).

Before this PR (main + #560): coverage was declared by whoever built the node and never checked against the sub-DAG.

A = KLL(latency) over Filter(region = 'us', Scan t)   declared {region: eu}   ← wrong
B = KLL(latency) over Filter(region = 'us', Scan t)   declared {region: us}
SummaryMerge(A, B) → accepted; every US row counted twice

After this PR: nothing is declared. Both states derive selection {region ∈ {us}} over the same definition, so the merge fails with PossibleOverlap.

What

Coverage = definition (what the state computes) + selection (which output rows of that computation it took):

KLL(latency) by[job] over Filter(region = 'us' AND latency < 100, Scan t)
  definition: KLL(latency) by[job] over Scan t
  selection:  [{ region ∈ {us}, latency ∈ (−∞, 100) }]

KLL(value) over TimeRange(1m, range) over TimeShift(2m) over Scan m{job="api"}
  definition: KLL(value) over Scan m
  selection:  [{ job ∈ {api}, relative time (−3m, −2m] }]
  • Selection: a predicate conjunct is lifted when it passes Filter, a range TimeRange, a TimeShift without @, and direct-column Project items up to the SummaryAgg (including SummaryAgg.filter and Scan.predicates), and it is a value set (=, !=, IN, NOT IN, OR of equalities) or an interval (<, <=, >, >=) on one column. Columns are identified by (table, name) as the SummaryAgg reads them, so shipping.region and billing.region stay distinct, and a Project renames (region AS r gives r ∈ {us}). A column whose (table, name) is not unique in the agg child (two items aliased k) is not lifted, and value sets compare literals by type, so 1 and 1.0 are never proven different. One range TimeRange(w) over TimeShift(s) gives the relative window (−(s+w), −s]; PromQL ranges are left-open.
  • Definition: the SummaryAgg with what was lifted removed. Everything else (arithmetic, regex, rate, instant selectors, …) stays in it as a residual, so states that differ there do not merge.
  • Merge: valid only when all inputs have structurally equal definitions (ignoring timing and guarantee, so ingestion-time and query-time panes can merge) and pairwise disjoint selections. Its coverage is the shared definition and the union of the selections; boxes that differ in one dimension join when the union is again one constraint (touching time windows, touching value ranges, value sets of one column), as in the paper's one range per column; gaps stay separate boxes. Merges nest.

Not in this PR (each follows when a caller needs it; see #573 §4.2.2 and §5.6): SummaryMerge { group_by } (rollup), family-specific overlap rules (HLL may overlap; #592), hash-partition constraints, lifting through Aggregate/window/rate partition columns, and SQL timestamp intervals (the IR has no timestamp literal yet).

Key code interfaces

summary_coverage.rs, node.rs

impl OperatorNode {
    /// `Some` for a SummaryAgg and a valid SummaryMerge; derived on first use.
    pub fn coverage(&self) -> Option<&SummaryCoverage>;
}

impl SummaryCoverage {
    /// The single entry point for every summary node.
    pub fn derive(node: &OperatorNode) -> Result<Self, CoverageError>;
}

pub struct SummaryCoverage {
    pub definition: Rc<OperatorNode>,
    pub selection: Vec<SelectionBox>,
}

pub struct SelectionBox {
    pub columns: BTreeMap<ColumnIdentity, Constraint>,
    pub relative_time: Option<(Bound<i64>, Bound<i64>)>,
}

pub enum Constraint {
    In(Vec<ScalarValue>),
    NotIn(Vec<ScalarValue>),
    Interval { lower: Bound<ScalarValue>, upper: Bound<ScalarValue> },
}

pub enum CoverageError { NotSummary, EmptyMerge, DefinitionMismatch, PossibleOverlap }
  • OperatorNode::new and validate_structure reject an invalid SummaryMerge; the derived coverage is cached on the node, so it is not derived again.
  • The node's coverage field is now a private cache: ignored by equality, skipped by serde, emptied on clone.
  • Removed: with_coverage, requires_coverage, CoverageError::Missing, and the coverage fields of CSE keys and FlatNode.

Tests

crates/types/tests/summary_coverage.rs covers each rule: filters and scan predicates lifting, population and value-range merges, overlap rejection, different inputs and different scan schemas, qualified columns, time panes (join, gap, overlap), touching value ranges joining, residuals, instant selectors, nested merges, timing, a forged merge caught by validate_structure, and three regressions from an independent review (ambiguous column names, literals of different types, partly lifted scan predicates), each confirmed failing before the fix. design_doc_worked_example reproduces the SQL worked example of #573 §4.2.2 (renaming Project, Scan.predicates, residual on an expression, SummaryAgg.filter). Tests that start from SQL/PromQL query strings follow in #542, the first PR where the planner lowers a query into a SummaryAgg. Workspace tests, cargo fmt and workspace/all-targets Clippy with warnings denied pass.


Base: main · Next: #539 · Tracker: #528 · Design: #573

🤖 Generated with Claude Code

@zzylol
zzylol force-pushed the stack/528-02b-merge-structure branch from 52b4003 to 8f58489 Compare October 6, 2026 20:45
@zzylol
zzylol force-pushed the stack/528-02c-coverage-derivation branch from 996294c to 764088d Compare October 6, 2026 20:48
@zzylol
zzylol marked this pull request as ready for review October 6, 2026 20:48
@zzylol
zzylol requested a review from Selvomega October 6, 2026 20:49
zzylol added a commit that referenced this pull request Oct 6, 2026
…low multi-source summaries

Review of #560/#646 found four problems:

1. Population names came from each node's schema, which `with_schema` may
   rename. A scan whose `tier` column is named "region" made `tier = 'eu'`
   read as `{region: eu}`, so a merge with a real `{region: us}` state was
   accepted and double-counted. Columns are now named from the Scan
   operator's own schema, and a path that renames a field leaves the
   population unknown.
2. For the same reason a merge could mix states of different columns
   (`Named("latency")` reading `size` on a renamed scan). An unknown
   population only merges with the same input, so this is rejected too.
3. `OperatorNode::map_children` dropped a SummaryAgg's coverage, so
   rebuilding a merge (e.g. in canonicalize) failed. A rebuild now keeps the
   declared time bounds and reads source and population again.
4. A SummaryAgg over two sources (a join, an IN subquery over another
   table) could never validate. It now carries no coverage and cannot be
   merged.

Adds summary_coverage_derivation.rs; the four regression tests fail
before this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol changed the title feat(ir): check summary coverage source and population against the subtree feat(ir): read summary coverage source and population from the subtree Oct 6, 2026
Comment thread crates/types/src/ir/summary_coverage.rs Outdated
Comment thread crates/types/src/ir/summary_coverage.rs Outdated
Comment thread crates/types/src/ir/summary_coverage.rs Outdated
Comment thread crates/types/src/ir/node.rs Outdated
zzylol added a commit that referenced this pull request Oct 6, 2026
…low multi-source summaries

Review of #560/#646 found four problems:

1. Population names came from each node's schema, which `with_schema` may
   rename. A scan whose `tier` column is named "region" made `tier = 'eu'`
   read as `{region: eu}`, so a merge with a real `{region: us}` state was
   accepted and double-counted. Columns are now named from the Scan
   operator's own schema, and a path that renames a field leaves the
   population unknown.
2. For the same reason a merge could mix states of different columns
   (`Named("latency")` reading `size` on a renamed scan). An unknown
   population only merges with the same input, so this is rejected too.
3. `OperatorNode::map_children` dropped a SummaryAgg's coverage, so
   rebuilding a merge (e.g. in canonicalize) failed. A rebuild now keeps the
   declared time bounds and reads source and population again.
4. A SummaryAgg over two sources (a join, an IN subquery over another
   table) could never validate. It now carries no coverage and cannot be
   merged.

Adds summary_coverage_derivation.rs; the four regression tests fail
before this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the stack/528-02c-coverage-derivation branch from b727827 to 34c43f0 Compare October 6, 2026 21:49
Comment thread crates/types/src/ir/summary_coverage.rs Outdated
zzylol added a commit that referenced this pull request Oct 7, 2026
…sketches

SummaryMerge no longer derives or checks coverage: of_merge,
UnknownInput and MergeOutputMismatch are removed and summary_coverage.rs
matches main. Coverage for all summary nodes will be derived by one
function in #646. The heap-based sketch restriction is also dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
SummaryCoverage now records which columns a state summarizes as well as
which rows: `input` (the SummaryAgg update expression) and `group_by`
(its reduction). with_coverage rejects a SummaryAgg declaration whose
columns differ from the node's own (ColumnMismatch), and SummaryMerge
requires coverage on every input (UnknownInput) with identical columns.
merge_disjoint checks columns too. summary_input_data is removed. A
nested SummaryMerge carries no coverage until #646, so it is rejected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
SummaryCoverage records which columns a state summarizes (input,
group_by) as well as which rows. Update the SummaryAgg and SummaryMerge
examples to #560: merges compare coverage columns, carry no coverage
until #646, and merged_coverage is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
…sketches

SummaryMerge no longer derives or checks coverage: of_merge,
UnknownInput and MergeOutputMismatch are removed and summary_coverage.rs
matches main. Coverage for all summary nodes will be derived by one
function in #646. The heap-based sketch restriction is also dropped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
SummaryCoverage now records which columns a state summarizes as well as
which rows: `input` (the SummaryAgg update expression) and `group_by`
(its reduction). with_coverage rejects a SummaryAgg declaration whose
columns differ from the node's own (ColumnMismatch), and SummaryMerge
requires coverage on every input (UnknownInput) with identical columns.
merge_disjoint checks columns too. summary_input_data is removed. A
nested SummaryMerge carries no coverage until #646, so it is rejected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the stack/528-02b-merge-structure branch from a5eb728 to 7ce0660 Compare October 7, 2026 14:55
@zzylol
zzylol force-pushed the stack/528-02c-coverage-derivation branch from 34c43f0 to fea4cc0 Compare October 7, 2026 15:02
@zzylol zzylol changed the title feat(ir): read summary coverage source and population from the subtree feat(ir): derive summary coverage from the subtree with one SummaryCoverage::derive Oct 7, 2026
@zzylol zzylol changed the title feat(ir): derive summary coverage from the subtree with one SummaryCoverage::derive feat(ir): derive summary coverage from the subtree with SummaryCoverage::derive Oct 7, 2026
zzylol added a commit that referenced this pull request Oct 7, 2026
Mark the summary-operator part as implemented, show the SummaryCoverage
fields, list merge_disjoint/validate as private to derive, and replace
the dropped same-input merge rule and "checked declarations" with what
#646 does: an unreadable population has no coverage and cannot merge, and
coverage is never written by hand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 7, 2026
Remove the coverage columns (input, group_by), check_columns and the
merge's coverage checks. SummaryMerge now only checks structure: at least
one input, every input is State with one state column and an identical
schema. Whether a structurally valid merge is semantically valid (same
computation, disjoint selections) is decided by summary coverage in #646,
following the design in #573.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
@zzylol
zzylol force-pushed the stack/528-02c-coverage-derivation branch from 3721101 to ebd2a93 Compare October 7, 2026 23:07
zzylol added a commit that referenced this pull request Oct 7, 2026
…e interface in #646

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
@zzylol zzylol changed the title feat(ir): derive summary coverage from the subtree with SummaryCoverage::derive feat(ir): derive summary coverage as definition + selection Oct 7, 2026
zzylol and others added 2 commits October 9, 2026 15:50
Coverage is derived from the node, never declared. For a SummaryAgg,
predicate conjuncts that lift through Filter, a range TimeRange, a
TimeShift without @ and direct-column Project items, and are a value set
or an interval on one column, form the selection; everything else stays
in the definition (the SummaryAgg with the selection removed). A
TimeRange(w) over TimeShift(s) gives the relative window (-(s+w), -s].

A SummaryMerge is valid only when its inputs have structurally equal
definitions (ignoring timing and guarantee) and pairwise disjoint
selections; OperatorNode::new and validate_structure enforce it. Its
coverage is the shared definition and the union of the selections,
joining adjacent windows and value sets.

OperatorNode.coverage becomes a private cache behind coverage():
ignored by equality, skipped by serde, emptied on clone. with_coverage,
requires_coverage and CoverageError::Missing are removed, as are the
coverage fields of CSE keys and FlatNode.

Design: docs/design_docs/proposals/asap-primitive-schema.md §4 (#573).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
…ections

From an independent review of the derivation:
- a column whose (table, name) is not unique in the agg child's schema
  cannot be named in a selection, so its conjuncts stay in the definition;
- value sets compare literals by typed order, so 1 and 1.0 (or NaN) are
  never proven different;
- a partly lifted Scan predicate is rebuilt with only its residual.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
@zzylol
zzylol force-pushed the stack/528-02c-coverage-derivation branch from ebd2a93 to 1bbd2a0 Compare October 9, 2026 15:58
zzylol added a commit that referenced this pull request Oct 9, 2026
Column identity follows the SummaryAgg's input schema (a Project renames
and drops the table); mark §6.1/§6.6 as #646 and per-family overlap as #592;
panes come from window composition (Pass 2); add IRate; three selection
shapes; current SummaryEstimate top-k output; time row of the definition
table; drop unchecked paper section numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol and others added 2 commits October 9, 2026 21:23
Point the module docs at #573 §4.2 and note that a Project renames
columns in their identity.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
OperatorNode::new and validate_structure derived a SummaryMerge's coverage
to check it and dropped the result, so coverage() derived it again. Store
it in the cache instead. Also correct two comments (Project qualifier,
absolute time needs timestamp literals).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 9, 2026
Merge coverage is cached at construction (#646); full per-type overlap
table from #592; k mismatch is a schema rejection; #579 top-k shape;
tighter cost bounds; walk-order caveat for combining conditions; Project
qualifier; absolute time marked later; Example B scope; one-walk wording;
define by[job] and closed schema; link Pass 2; window notation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
The #573 worked example (§4.2.2) is now a SQL query; build the DAG the SQL
frontend lowers it to (WHERE folded into Scan.predicates, no time window).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 10, 2026
…s them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
Value ranges on one column now join like touching time windows, so every
selection dimension follows Goldstein & Larson's one-range-per-column form.
The joined range stays explicit: neither input took NULL rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
Comment thread crates/types/src/ir/node.rs Outdated
Comment thread crates/types/src/ir/summary_coverage.rs
Review of #646:
- The coverage cache could go stale: `operator` and `schema` are public,
  so a node edited in place (e.g. through `Rc::make_mut`) kept the
  coverage of its old value. Drop the cache; `coverage()` derives on each
  call and returns an owned value.
- A `Filter`, `TimeRange` or `TimeShift` whose schema renames columns was
  removed from the definition once everything was lifted out of it, taking
  the rename with it, so `tier = 'eu'` named `region` looked disjoint from
  `region = 'us'`. The walk now stops at such a node, which stays in the
  definition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
@zzylol
zzylol merged commit 66a9952 into main Oct 11, 2026
3 checks passed
@zzylol
zzylol deleted the stack/528-02c-coverage-derivation branch October 11, 2026 16:21
zzylol added a commit that referenced this pull request Oct 12, 2026
…export

Coverage is derived from the node (#646), so timed copies no longer carry
it and PhysicalASAPDAGNode drops its coverage field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
…ion) (#573)

* docs: design for ASAP primitive schema and summary semantics

One design document for the edge schema of ASAP primitives: fields that carry
summary state, the summary semantics a schema must preserve, coverage, and
per-operator examples. Related docs keep a summary and link to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: restore the hand-written sections before the examples

Keep the author's sections 1–4 as written; the operator examples (§5) and
key code interfaces (§6) stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: a top-k readout returns selected rows

Follows #579: the top-k sketch readout derives one row per selected item
instead of a packed Utf8 column, matching the exact Sort → Limit shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: explain why summary coverage is a node field, not part of the schema

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: summary coverage records rows and columns

SummaryCoverage records which columns a state summarizes (input,
group_by) as well as which rows. Update the SummaryAgg and SummaryMerge
examples to #560: merges compare coverage columns, carry no coverage
until #646, and merged_coverage is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: summary coverage as definition + selection, based on Goldstein-Larson view matching

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: the 5.1 example's time index is a Timestamp column

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: absolute time is a timestamp-column interval; match the coverage interface in #646

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: ambiguous column names and literal types in selections

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: fill §1 links and operator list, §2 terminology, §3 summary type terms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: point links into asap-primitive-schema.md at its current sections

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: list the levels of a summary type as a table

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: list supported query intents per summary type

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: structure §4 with summary table, rule tables and bullet lists

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: rename §4.2 heading

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: explain the residual condition of view matching

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: define the three parts of a view's WHERE with examples

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: define equivalence classes, ranges and residuals

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: move the Goldstein & Larson mapping after §4.2 as §4.3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: say which paper requirement unions of views relax

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: SummaryMerge is implemented on main since #560

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: §4.7 leaves the deployment runtime, such as SDS, out of coverage

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: rewrite §4.1 as a worked G&L example and §4.2–§4.5 in plain terms

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: worked examples for §4.4 and §4.5; drop the §5.8 recap table

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: draw the §5 examples as diagrams and timelines

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: reword how coverage is computed in §4.4

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: explain the selection derivation in §4.4 as goal and steps

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: move the Goldstein & Larson mapping into a footnote of §4.2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: clearer wording for Rule 1

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: draw §5 examples as bottom-up operator trees

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: draw §4.4 and §5 examples as DAGs with shared nodes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: rename step 3 in §4.3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: phrase step 3 outcomes as conditions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: plainer Rule 1 and Rule 2 details with reasons and examples

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: §4.3 derives the definition first, then the selection

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: name the code interface in the §4.5 heading

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: move the coverage code interface to §6.5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: §6 intro points to §6.5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: merge §4.2 and §4.3 (why two parts, then derivation); fold §4.4 into §5.6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: refer to the downstream deployment runtime, not a specific backend

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: same wording in the §2 layout row

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: leave the SDS definition as a TODO for a separate doc

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: make §5 easier to read with one question table per operator

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: explain every struct and variant field in §6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: label §5 operator descriptions as operator definitions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: explain every function in §6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: describe SketchKind.category as the aggregation intents it answers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: explain what §6.3 update input and readouts are for, with examples

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: rename §6.2 to summary-family parameters

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: show how the §6 code interfaces connect

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: order §6 subsections from the DAG node inward

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: add the cost of deriving coverage (§4.2.3)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: fix correctness issues found in review

Column identity follows the SummaryAgg's input schema (a Project renames
and drops the table); mark §6.1/§6.6 as #646 and per-family overlap as #592;
panes come from window composition (Pass 2); add IRate; three selection
shapes; current SummaryEstimate top-k output; time row of the definition
table; drop unchecked paper section numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: fix second-review findings

Merge coverage is cached at construction (#646); full per-type overlap
table from #592; k mismatch is a schema rejection; #579 top-k shape;
tighter cost bounds; walk-order caveat for combining conditions; Project
qualifier; absolute time marked later; Example B scope; one-walk wording;
define by[job] and closed schema; link Pass 2; window notation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: fix §1 grammar; start every example from its PromQL or SQL query

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: restore the author's wording in §1, §3 and §4 openings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: show ASAPOp generic over its child reference, as on main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: describe the union and instant-selector rules as #646 implements them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: touching value ranges join; instant selectors join the later lifting rules

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: PromQL comparisons by lowering; constants from folded casts

A vector/scalar comparison lowers to a Filter above rate and moves to the
selection; a bool comparison is a Project and a vector/vector comparison a
BinaryOp, both kept in the definition. The SQL front end folds numeric casts
of literals as DataFusion does, so `latency < 100` reads as a constant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* docs: only a Project may rename on the way to the selection

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 12, 2026
…export

Coverage is derived from the node (#646), so timed copies no longer carry
it and PhysicalASAPDAGNode drops its coverage field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
Cut today's planner over to the unified OperatorNode IR. The planner calls
the unified frontends (`asap_frontend_*::unified`) and returns OperatorNode /
QueryRoot; ParsedWorkload keeps scalar roots; Pass 1 (ASAPStrategies), the
identical-sub-DAG sharing, selection, DAG assembly and lifecycle costing all
run on OperatorNode; and PlanOutput exposes the whole workload DAG.
Planned summary states derive their coverage from the node (#646).

The legacy physical planner read the planner's QueryExpr output, so it is
removed here together with its expression compiler and raw sources; every
caller uses the native compiler from #541 (`unified_physical_planner`,
`unified_sources`), which reads the PhysicalASAPDAG export. Operator names
follow the evaluation vocabulary (readout -> evaluation).

The MetricsQL frontend input carries the planning clock and lowers through
`lower_metricsql_query_workload`, so bare selectors get the workload's
ingestion-interval lookback (#540).

The legacy frontend modules and the QueryExpr/SummaryNode IRs stay compiled
but are no longer re-exported from post_asap. The next PR moves the unified
modules to the crate roots; a later one removes the legacy IRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
…export

Coverage is derived from the node (#646), so timed copies no longer carry
it and PhysicalASAPDAGNode drops its coverage field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
Cut today's planner over to the unified OperatorNode IR. The planner calls
the unified frontends (`asap_frontend_*::unified`) and returns OperatorNode /
QueryRoot; ParsedWorkload keeps scalar roots; Pass 1 (ASAPStrategies), the
identical-sub-DAG sharing, selection, DAG assembly and lifecycle costing all
run on OperatorNode; and PlanOutput exposes the whole workload DAG.
Planned summary states derive their coverage from the node (#646).

The legacy physical planner read the planner's QueryExpr output, so it is
removed here together with its expression compiler and raw sources; every
caller uses the native compiler from #541 (`unified_physical_planner`,
`unified_sources`), which reads the PhysicalASAPDAG export. Operator names
follow the evaluation vocabulary (readout -> evaluation).

The MetricsQL frontend input carries the planning clock and lowers through
`lower_metricsql_query_workload`, so bare selectors get the workload's
ingestion-interval lookback (#540).

The legacy frontend modules and the QueryExpr/SummaryNode IRs stay compiled
but are no longer re-exported from post_asap. The next PR moves the unified
modules to the crate roots; a later one removes the legacy IRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 12, 2026
…#541)

* feat(ir): restore execution timing assignment for physical plans

The logical export is phase-free. Physical planning still needs the lifecycle
timing expansion; bring it back unchanged except for carrying coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ir): export timed plans as PhysicalASAPDAG

The logical export carries no execution timing. Physical planning output
needs it: reuse the logical payloads and node ids, and add each node's and
edge's data state plus window compatibility, as the earlier post-ASAP export
did. Coverage is carried through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(runtime): compile unified operator and scalar graphs

* feat(ir): give physical ASAP DAGs one root per batch query

Match the logical export: a batch is one DAG whose roots are its queries,
with shared sub-DAGs exported once. The runtime compiles all roots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: integrate with #614 (DataFusion 54): let chrono resolve to DF 54's version

DataFusion 54 requires chrono ^0.4.44, so the runtime crate's exact
=0.4.39 pin no longer resolves; keep 0.4.39 as the minimum.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(runtime): read Operator<NodeId> payloads from the physical ASAP DAG

Follow #537: the physical DAG now carries each node's operator as
Operator<NodeId> (from ir::flat) instead of the removed wire mirror types.
The runtime matches Operator::NonASAP / Operator::ASAP directly, and
restore() rebuilds the in-memory DAG with map_children instead of a
field-by-field conversion. Node ids are plain usize.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(ir): follow derived summary coverage in timing and physical export

Coverage is derived from the node (#646), so timed copies no longer carry
it and PhysicalASAPDAGNode drops its coverage field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(runtime): address #541 review findings

- timing: a SummaryAgg the lifecycle assignment recomputes at query time
  now forces the SummaryAgg consuming it to query time
  (`forces_query_time` reads the assignment), instead of failing with
  EvaluationUnderMaintenance.
- physical export: `PhysicalASAPDAG::validate` requires each node's
  incoming edges to be exactly its operator's inputs with their roles, so
  swapped Left/Right edges cannot reorder `a - b` (EdgeRoleMismatch).
- physical export: a DAG that fails validation (an empty batch, a
  hand-timed plan) is an `InvalidPhysicalDAG` error instead of a panic.
- precompute: only a TimeRange over (a TimeShift over) a time series Scan
  takes the raw-sample input contract, as before the port; a table range
  needs a stored population.
- compile: an input binding below a node the PromQL fallback evaluates
  is an error; the fallback reads raw series slots, so it was silently
  ignored. A binding below another bound input is still superseded by
  it, as in `cut`.
- compile: drop the scalar-literal binary branch; nothing filled its map.
- PromQL fallback: clamp's `NOT (max < min)` is now the IR `Filter`
  #540 emits; the fallback runs a filter on scalars alone, and the
  pointwise projection no longer filters on its own.
- tests: `raw_series_contract_is_explicit` asserts that a consumed bare
  selector is instant selection (its comment said the opposite) and that
  a binding below the fallback is rejected; new `unified_precompute` test.
- PromQL fallback: unary minus drops the metric name like every other
  pointwise projection, following #540 and the Prometheus 3.15 baseline
  (#511); pointwise_projection_names_and_dynamic_parameters expects it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(runtime): remove the PromQL fallback evaluator

ASAPPlanner generates candidates and computes their cost, or says why a
query has none. Executing what the native operators cannot run belongs
to the deployment runtime (ASAPQuery backend), not to the Planner.

Remove unified_physical_planner::promql_fallback and its tests, and the
fallback paths in compile: raw-series input slots, the fallback lowering
and the below-fallback binding check. A node without a native lowering
now fails compilation with Error::Invalid.

Reject Planner node IDs above u32::MAX, which could collide with helper
IDs, and error instead of debug-asserting when a node needs more than
2^16 helpers. Document that forces_query_time also forces an unaligned
per-series PromQL BinaryOp to query time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Oct 12, 2026
Cut today's planner over to the unified OperatorNode IR. The planner calls
the unified frontends (`asap_frontend_*::unified`) and returns OperatorNode /
QueryRoot; ParsedWorkload keeps scalar roots; Pass 1 (ASAPStrategies), the
identical-sub-DAG sharing, selection, DAG assembly and lifecycle costing all
run on OperatorNode; and PlanOutput exposes the whole workload DAG.
Planned summary states derive their coverage from the node (#646).

The legacy physical planner read the planner's QueryExpr output, so it is
removed here together with its expression compiler and raw sources; every
caller uses the native compiler from #541 (`unified_physical_planner`,
`unified_sources`), which reads the PhysicalASAPDAG export. Operator names
follow the evaluation vocabulary (readout -> evaluation).

The MetricsQL frontend input carries the planning clock and lowers through
`lower_metricsql_query_workload`, so bare selectors get the workload's
ingestion-interval lookback (#540).

The legacy frontend modules and the QueryExpr/SummaryNode IRs stay compiled
but are no longer re-exported from post_asap. The next PR moves the unified
modules to the crate roots; a later one removes the legacy IRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW
zzylol added a commit that referenced this pull request Oct 13, 2026
* refactor(planner): plan one shared unified workload DAG

Cut today's planner over to the unified OperatorNode IR. The planner calls
the unified frontends (`asap_frontend_*::unified`) and returns OperatorNode /
QueryRoot; ParsedWorkload keeps scalar roots; Pass 1 (ASAPStrategies), the
identical-sub-DAG sharing, selection, DAG assembly and lifecycle costing all
run on OperatorNode; and PlanOutput exposes the whole workload DAG.
Planned summary states derive their coverage from the node (#646).

The legacy physical planner read the planner's QueryExpr output, so it is
removed here together with its expression compiler and raw sources; every
caller uses the native compiler from #541 (`unified_physical_planner`,
`unified_sources`), which reads the PhysicalASAPDAG export. Operator names
follow the evaluation vocabulary (readout -> evaluation).

The MetricsQL frontend input carries the planning clock and lowers through
`lower_metricsql_query_workload`, so bare selectors get the workload's
ingestion-interval lookback (#540).

The legacy frontend modules and the QueryExpr/SummaryNode IRs stay compiled
but are no longer re-exported from post_asap. The next PR moves the unified
modules to the crate roots; a later one removes the legacy IRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* feat(planner): plan and export standalone scalar queries

A workload entry whose root is a standalone scalar (`scalar(quantile(0.99,
rate(x[5m])))`, `scalar(sum(up)) + 1`) was copied through unplanned. Per
#511 §2.3 it has no query node and reads operator sub-DAGs by reference;
per operator-sharing §3 those sub-DAGs get the same validation and accuracy
assessment as operator roots.

MajorPass now searches, selects, assembles and lifecycle-plans each
distinct operator sub-DAG a scalar root reads, against the scalar's entry
and accuracy target, alongside the operator roots, and rebuilds the scalar
over the selected sub-DAGs. PlanOutput::scalar_plans holds their lifecycle
plans. PhysicalASAPDAG gains `scalar_roots`: each standalone scalar query
as a `ScalarExpr<PhysicalASAPNodeId>` keyed by its position in the batch,
so query i is the scalar root keyed i, or else the next of `roots`.
Validation checks positions and treats scalar references as roots for
existence, cycles and reachability. Wire version 9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): plan non-pass-through scalar references exact

A standalone scalar hands its entry's accuracy target to its operator
reference only when it passes that one reference through (`scalar(v)`, or
arithmetic/comparisons with literals). Otherwise each reference is planned
exact: independently approximate references leave a combination such as a
difference of two quantiles unbounded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): check scalar plans in the contract; remove dead code

check_contract now verifies that `scalar_plans` holds one plan per
distinct reference of each scalar root, in order and keyed by its entry.
The scalar_roots doc says keys are batch positions, equal to entry
indices because the contract requires dense indices. A validation test
covers duplicate and descending scalar-root keys. The never-constructed
VectorScalar/ScalarVector operand modes and the unused
ParsedWorkload::new/entries are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): lower scalar(v) in Project/Filter expressions as scalar inputs

`a / scalar(b)` and `a > scalar(b)` failed costing with
UnsupportedQueryOperator. A Project or Filter now takes one
PromqlVectorToScalar input per distinct operator reference its
expressions read, after its row input; its statistics carry one
`scalar_inputs` edge per such input.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): reuse a top-k only over the same partitions

TopKLimitReuseStrategy offered a global top-10 as the source of a per-job
top-5. It now requires equal `partition_by`, and the maintained
population recognizer requires the limit to partition like its sort.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* refactor(planner): tell logical rewrites apart by provenance

`is_logical_rewrite` inferred a logical rewrite from the replacement's
shape (no guarantee, no ASAP operator). It is now
`ReplacementSubDAG::is_logical_rewrite`, read from the candidate's
provenance (LogicalRewrite, CseShare, CseRecompute,
AccuracyReconciliation); the free function is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): return an error when search input already holds ASAP operators

The search entry points asserted that no workload root contains an ASAP
operator. They now return `SearchError::AsapInput`; MajorPass reports it
as `OptimizeError::Search`, and the explanation entry points pass it on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): walk operator nodes read by scalar expressions

The explanation report located only relational children, so a target
read through `scalar(...)` in a predicate or projection had no
breadcrumb; it now visits every operator input, as target discovery does.
`frontier_from_timing` also puts an ingestion-time node a standalone
scalar root reads on the frontier, like an operator root.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(ir): do not promote a filtered count ranking to a heavy-hitter TopK

The IR canonicalize pass ignored measure filters, so `COUNT(*) FILTER
(WHERE ...)` or `COUNT(col)` over a nullable column ranked by ORDER BY
... LIMIT became a heap sketch over every row. It now keeps Sort + Limit
for a filtered measure (the legacy #466 guard), and the keyed heap-sketch
input rule refuses a filtered inner aggregate. The legacy
does_not_promote_a_filtered_count_ranking test is ported to the IR and
through the SQL front end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* Revert "fix(planner): plan non-pass-through scalar references exact"

Combinations of approximate values may read summaries: each scalar
reference is planned against the entry's accuracy target again. Composing
the guarantee of such a combination is left to #575 and Stage 3's
accuracy model.

This reverts commit c44ecda.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): rank top-k only when the Limit partitions like its Sort

A global Limit over a partitioned Sort keeps k rows overall, not k per
group, so neither the heavy-hitter promotion nor the current-series heap
candidates apply to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(ir): export one ScalarRef edge per distinct referenced node

Cost lowering reads a node referenced several times from one operator's
expressions once; the export now matches, deduping by node id (one id
per pointer).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): cover scalar inputs, top-k partitions, scalar plan contract, provenance

Project/Filter with two scalar references and one read twice lower and
cost; the maintained population requires the Limit to partition like its
Sort; the keyed heap input refuses a filtered count; check_contract
rejects scalar plans out of order, under another entry, or extra; a
guarantee-less summary realization is not a logical rewrite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): a difference of two scalar quantiles reads summaries

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): plan structurally identical scalar references once

Front ends build identical references as separate nodes and planning
merges them, so pointer-distinct references left the rebuilt scalar with
one reference but two scalar plans, failing the contract. References are
now distinct by structure. Document that each reference keeps its summary
candidates and that the combined guarantee is composed later (#669, #575).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* test(planner): reject a per-job Limit over a global Sort; doc cleanups

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

* fix(planner): read exact-state query roots through their finalize

MajorPass assembled roots with assemble_selected_dag, so a root whose
selected candidate is an exact state came out as a bare SummaryAgg. The
lifecycle plan now finalizes its root after costing (the finalize wraps
the shared state, so sharing and cost evidence stay), scalar references
included since scalar(v) reads values. Query-root callers and tests use
assemble_selected_query.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W7qG9aFyPij5uWsyAJCxDW

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol mentioned this pull request Oct 13, 2026
2 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check declared summary coverage population against subtree filters

2 participants