feat(dag-viewer): plan through stage_pipeline in the live viewer; delete dag_export

[zzylol]

Why

The dag-viewer's live path (server.py, generate-sample.sh, the Query planner's "Plan selected workload" button) still ran the legacy dag_export bin. That bin uses the legacy search, not the planner on main (Stage 1 → Stage 3, run_stage1_to_stage3). This PR moves the live path onto stage_pipeline. Once nothing needs dag_export and asap_types::{dag_export, cost}, it deletes them.

What

Capability mapping (dag_export → stage_pipeline/viewer)

dag_export capability	Now provided by
Plan ad-hoc PromQL from the editor (--promql, --epsilon, --data-ingestion-interval-ms)	server.py → stage_pipeline --promql … --epsilon … --interval-ms …
Plan ad-hoc SQL over tables declared in the editor (--sql, --table-schema)	new stage_pipeline --sql … --table '<json>' (same column types; no built-in tables, the editor sends its declared ones)
Mixed SQL + PromQL workloads in one plan	stage_pipeline takes --promql/--sql in any order. Each query is lowered in its own language, then planned as one workload (planning_stages::run_planning_stages_over).
Query names (--name)	stage_pipeline --name after each query sets workload.queries[].id, which defaults to q<n>. The editor sends each row's name through /api/plan.
Pre-ASAP DAG (JSON)	stage0_query_language_frontends.dag. It is still written when Pass 1 fails: Stage 1 then carries error and no candidates, nothing is selected, and the viewer shows the Stage 0 lanes with the error in the candidate table header.
Post-ASAP / physical DAG (JSON)	stage2_physical_asap_optimization.candidates[].dag (PhysicalASAPDAG), with the Stage 3 selected plan
Per-node cost annotations	stage3_plan_selection.costs[id].per_node; the viewer labels each post-ASAP node cost X [× multiplier] and shows the entry in the node detail
Workload cost and the winner (replacing the old baseline/benefit comparison)	a Stage 3 candidate table built from stage3_plan_selection (stageCandidateRows): the selected candidate, the other costed ones cheapest first, then the rejected and Stage 2 pruned ones with their reasons, then the Stage 1 candidates that did not compose (composition error) with their errors. The table header (stageCandidateHeader) gives the candidate count, the shown count when capped, and the Stage 3 selection error when there is one. The editor status line shows the selected id, total and candidate count.
Pre/post view in the viewer, multi-query union	stage-workload.js turns a stage document into the viewer's per-query pre (Stage 0) / post (selected Stage 2 plan) DAGs, using stage node ids as workload_node_id so shared nodes merge. A standalone scalar query (scalar(v), time(), 1 + 2) is read from the physical DAG's scalar_roots by query position; other queries take the next entry of roots. In both views its root is a Scalar node showing the expression, over the operator nodes it reads.
Load an exported file / standalone HTML	drag-and-drop and render.py take one stage document, embedded unchanged and read by stage-workload.js

Not covered, dropped (accepted by the user):

• --planner-cost-json / --default-cost / --topk-margin-json. These ranked the legacy search with a deployment physical-evidence document. Stage 3 prices candidates with its own built-in model. Evidence-driven Stage 3 cost is being redone in Calibrate plan selection's costs from measured sketch costs #679/Stage 3: evidence-driven cost (storage I/O, hand-off bytes) #615. The library types (PhysicalEvidenceSnapshot, PhysicalPlanCostModel) stay.
• The pre-ASAP replacement notes: the badges dag_export put on Stage 0 nodes from asap_aware_mapping::explain_replacements (which ASAP replacement could apply there). The library function stays; the viewer does not show them yet.
• --progress: dag_export's per-phase progress and timing lines on stderr. stage_pipeline prints nothing while it runs.
• The legacy-only replacement "translation" cards and baseline/benefit/edge cost annotations. Their stage-document equivalents are shown instead: per-node Stage 3 cost, and the candidate table with costs and rejection reasons.
• The old export format is no longer supported (user decision). The viewer and render.py read only asap-stage-pipeline/v1 documents. Removed: the WorkloadDAG pre/post reader, multi-file merge, render.py's IR labeling (prepare_workload, _semantic_label), the cost-annotation code and CSS, and post_asap_fixture.json. test_render.py now covers the stage-document path.

Deleted: crates/devtools/src/bin/dag_export.rs, crates/types/src/dag_export.rs, crates/types/src/cost.rs.

Kept: CostUnit. It is the only asap_types::cost item used elsewhere, so it moves into asap_aware_mapping::cost_model, which already re-exported it. Its serde stays the same.

How

• stage_pipeline: --sql / --table (repeatable). Adds planning_stages::lower_sql_roots, and workload.queries[].language now records the real language.
• stage_pipeline: a workload with more than --max-combinations (default 10^4) whole-workload candidates is refused before Stage 1 enumerates any. It exits with status 3, the count and the limit, and writes no document. It is never truncated. Pass 1 runs once: the library splits run_stage1_to_stage3 at Pass 1 (run_stage1_to_stage3_from_pass1, PerComputationCandidates::combinations), and planning_stages checks the limit on that inventory (PlanningStagesError). The library still enumerates and ranks every candidate; only --max-displayed caps the display. A Pass 1 failure still writes the document (Stage 0 and the error). A --table time_index that is not a column index ("0", -1, out of range) is an error. A --table declared twice is rejected, and a duplicate query name says when it is a query's default name.
• stage_pipeline / planning_stages: mixed-language ad-hoc runs and --name. run_planning_stages_over(roots, targets, data) is the language-agnostic core; run_planning_stages(workload, roots) calls it.
• server.py: /api/plan plans one request at a time (a lock around stage_pipeline). It validates the editor payload: field types (the request is an object; query name, language and text; table column entries and time_index; selected schemas are declared table names; epsilon and interval_ms are numbers, not booleans), unique table names, and unique query names (the error says which query already uses the name). SQL tables are passed only when there is a SQL query. It runs stage_pipeline (honoring CARGO_TARGET_DIR) and returns the asap-stage-pipeline/v1 document. The combination-limit refusal comes back as 422 with stage_pipeline's message, which the editor shows as is. /api/example serves dag.example.json. The editor's default rows are all checked again.
• stage-workload.js (new, DOM-free) is loaded before viewer.js. viewer.js reads stage documents through it.
• generate-sample.sh writes dag.example.json with stage_pipeline: the editor's default mixed, named workload (3 SQL + 2 PromQL), with --max-displayed 4. That flag is display-only, so Stage 3 still selects among all 144 candidates.
• Tests updated for the deletion: the explanation node_hash test now compares against structural_hash directly. exact_composition keeps its physical-export timing assertions and drops the WorkloadDAG ones.
• Absorbs chore(devtools): delete the dag_export bin #616 (delete the dag_export bin), chore(types): delete the dag_export module #617 (delete the dag_export module) and the server/generate-sample/editor parts of feat(dag-viewer): redesign the viewer around the stage pipeline #574.

Before

The editor and server.py ran dag_export --post-asap (the legacy search). dag.example.json was a dag_export WorkloadDAG.

After

The editor and server.py run stage_pipeline (Stage 0 → Stage 3). The viewer shows Stage 0 as pre-ASAP and the Stage 3 selected physical plan as post-ASAP, with per-node Stage 3 costs. dag_export, asap_types::dag_export and asap_types::cost are gone.

Verification

• cargo fmt --all -- --check: OK
• cargo clippy --workspace --all-targets -- -D warnings: OK
• cargo nextest run --workspace: 1556 passed, 50 skipped (including the combination limit at 3^9 = 19683 > 10^4, duplicate tables, default-name messages, time_index checks, the Pass 1 failure document, and that examples/scalar-queries.json is current)
• python3 test_render.py (from tools/dag-viewer/): 27 OK. These are stage-document only: render, stageWorkload (including scalar queries first, in the middle and last, and a Pass 1 failure document), stageCandidateRows (with composition-error rows), stageCandidateHeader (with the Stage 1 and selection errors), rejection of other JSON, and selection logic, run with py_mini_racer.
• python3 -m unittest test_server: 11 OK, including mixed-language order with names, request type checks (with columns and time_index), duplicate table and query names, one planning request at a time, the 422 combination-limit refusal, and /api/plan returning a stage_pipeline document whose query keeps its editor name
• bash tools/verify_metricsql_external_consumer.sh: OK
• The Example 1 fixture regenerates byte-identical
• git grep -w dag_export, asap_types::dag_export and asap_types::cost find matches only in Markdown docs (below)

Limitations

• Stale docs, left for Update the docs after the G1–G11 consolidation groups land #677 (no Markdown changes here): tools/dag-viewer/README.md, docs/user_guide_docs/run-a-query.md, docs/develop_docs/{library-api, physical-handoff-costs, storage-operation-costs, end-to-end-accuracy-guarantees, asap-aware-mapping-architecture}.md, docs/design_docs/decisions/{concat-unique-keys, cse-cost-model}.md, old_docs/README.md.
• The rest of the viewer redesign (feat(dag-viewer): redesign the viewer around the stage pipeline #574's stage lanes, candidate ranking, examples manifest) is left for G11. Candidates other than the selected one are not shown in the viewer yet.
• Planner-generated reasons (Stage 3 rejections, Stage 1 errors) still name queries by position (q5), not by the editor's names.
• There are no built-in SQL tables; the editor declares them. Fuller SQL editor support (feat(dag-viewer): plan SQL queries over declared tables in the editor #618) is in G11.

Human review — do not complete with an agent

• The MVP boundary is correct.
• New conceptual layers or public interfaces are necessary.
• The before/after description matches the intended product behavior.
• Human reviewer:
• Decision and rationale:

🤖 Generated with Claude Code