Skip to content

eval: e2e block for multi-grouping and all tables - #209

Merged
milindsrivastava1997 merged 4 commits into
mainfrom
feat/e2e-multigroup-export
Oct 14, 2026
Merged

milindsrivastava1997 merged 4 commits into
mainfrom
feat/e2e-multigroup-export

Conversation

@zzylol

@zzylol zzylol commented Oct 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Every synthetic table now gets the e2e block that #208 added for classic tables, so ASAPQuery can run the multigroup and all tables end to end. Follow-up to #204, which left them out.

The multi-grouping templates (11–17) have no PromQL form: their distinct counts are over a value field (user_id, src_ip), and template 16's p99 pools every sample of a group over the window. ASAPQuery renders SQL for them from the structured fields below; classic templates keep their PromQL.

  • metrics: each schema metric is a records table: SCHEMAS' labels (with child_of and skew), rate as records_per_sec, and its values (with scaled_by, anomaly, burst). Under metric data_1 the table is data_1_http.
  • queries: one entry per distinct schema query, {table, value, capability, grouping, quantile (quantiles only), window_secs, interval_secs, rqes}. Entries identical but for their RQEs (templates 14, 15 and 17 share groupings) are merged into one entry that reads all of them, so each query string maps to its RQEs.
  • Classic entries are unchanged. Their (promql, interval) deduplication drops no RQE ids: the dropped duplicates read the same RQEs as the kept entry.

No plans are regenerated. Nothing in the runner or scripts excluded multigroup or all from the plan export, so #203 already committed their roll-up plans (each rolled-up RQE carries deployment_grouping).

Example (multigroup-shared1, template 16):

{"capability": "quantile", "grouping": ["region", "service", "endpoint"], "interval_secs": 60, "quantile": 0.99,
 "rqes": ["synthetic/templates=multigroup/shared=1/t16_p99_latency_by_region,service,endpoint/values/15m"],
 "table": "http", "value": "latency", "window_secs": 900}

Testing

  • With e2e removed, the 8 regenerated multigroup and all tables are byte-identical to main's. The 4 classic tables regenerate byte-identical including e2e. plan.tsv is unchanged.
  • Each schema RQE id appears in exactly one entry. Schema entries per table: multigroup-shared1 52, all-shared1 52 (+54 classic).
  • Rerunning autosketch_vs_asap on the regenerated multigroup-shared1 and all-shared1 tables reproduces all 36 committed plans.
  • New tests: template 16 gives 7 groupings × 2 windows with the exact fields; with metrics=2 each query's table names one of data_0_http, data_1_flows, …; no two entries match apart from rqes. The Python suite passes (28 tests); cargo test --release -p rqe-optimizer passes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01AnJ7LocWxguh3q4QWkCYae

zzylol and others added 3 commits October 13, 2026 01:11
export_autosketch_eval_table.py --synthetic now writes workloads[0].e2e to
every table. Schema templates have no PromQL (SQL is rendered downstream),
so each schema RQE gets one structured query entry: table, value,
capability, grouping, quantile (p99: 0.99), window_secs, interval_secs and
its rqes. Each schema metric is a `records` table (`http`, `flows`;
`data_i_http` under metric `data_i`) with SCHEMAS' labels, rate as
records_per_sec, and values (scaled_by, anomaly, burst as in SCHEMAS).
Classic entries are unchanged.

The multigroup and all tables in autosketch-vs-asap-hydra/inputs/tables are
regenerated; minus e2e they equal the committed ones byte for byte, and the
classic tables are unchanged. Their roll-up plans were already exported by
#203 (plans/*.plans.json.gz); a rerun of multigroup-shared1 and all-shared1
reproduces all 36 plans of each exactly, so the plans stand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnJ7LocWxguh3q4QWkCYae
Schema query entries identical apart from `rqes` (templates 14, 15 and 17
share some groupings) are merged into one entry whose `rqes` lists every
RQE it reads, sorted, so each distinct SQL query appears once. Classic
entries are unchanged. The multigroup and all tables are regenerated;
minus e2e they still equal main's byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnJ7LocWxguh3q4QWkCYae
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnJ7LocWxguh3q4QWkCYae
@milindsrivastava1997

milindsrivastava1997 commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

@zzylol For ProjectASAP/ASAPQuery#856 (--planner saved for SQL on the multigroup tables), could each structured entry also carry its ASAPQuery SQL string, the way classic entries carry PromQL? For example:

{"sql": "SELECT service, QUANTILE(0.99, latency) FROM http WHERE ts BETWEEN DATEADD(s, -300, NOW()) AND NOW() GROUP BY service", ...}

The planner would use it as written. ASAPQuery#844 would replace NOW() with the floored end time instead of rendering its own asapquery.sql. That keeps one ASAPQuery SQL renderer instead of two (#844's Python and a new one in the planner).

zzylol added a commit that referenced this pull request Oct 13, 2026
Table conflicts resolved by regenerating every table with the merged
exporter and each table's original --revision (classic 44ac160, all and
multigroup beacf68). Every structured e2e entry now has `sql`; the
multigroup tables equal #209's byte for byte. Minus `e2e`, every table
equals #210's previous head, so the plans and results stand. Templates 7
and 9 stay dropped and template 2's PromQL stays
topk(32, sum_over_time(data[w])).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AnJ7LocWxguh3q4QWkCYae
@milindsrivastava1997
milindsrivastava1997 merged commit 2e31da3 into main Oct 14, 2026
2 checks passed
@milindsrivastava1997
milindsrivastava1997 deleted the feat/e2e-multigroup-export branch October 14, 2026 01:34
@milindsrivastava1997
milindsrivastava1997 restored the feat/e2e-multigroup-export branch October 14, 2026 01:42
@milindsrivastava1997
milindsrivastava1997 deleted the feat/e2e-multigroup-export branch October 14, 2026 01:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants