Skip to content

A Sum over a grouped PromQL sum fails to execute: the update reads a sample-value column the grouped output doesn't have #720

Open

Description

@zzylol

Problem, by example

sum(sum by (job) (sum_over_time(m[5m])))

Pass 1 builds the plain candidate as three exact Sum accumulators: per series, then by job, then global. That candidate fails to execute:

summary update column missing or ambiguous

The outer global Sum's update reads the sample-value column from its input, but its input is the grouped sum's output [job, sum], which has no sample-value column. Expected: the outer Sum reads the grouped sum's measure column (sum) and returns the total.

Found while adding execution-equality tests for the reuse rules (PR #716). The nested-sum merge candidate from #716 doesn't hit this, because it reads the raw input rows directly.

Why it happens

When Pass 1 builds a SummaryAgg over the output of a grouped PromQL aggregate, it uses the sample-value lookup meant for raw series rows. It should resolve the update column against the child's actual output schema (the grouped aggregate's measure column).

Proposal

When the child is an aggregate, resolve the Sum (and the other accumulators') update column to the child's measure column, e.g. by using the schema the child's output_schema reports. Add a failing integration test first that executes every candidate of the query above and compares the rows.

Acceptance

All candidates of sum(sum by (job) (sum_over_time(m[5m]))) execute and return the same rows.

Related: #680, #716.

Activity

  1. added
    bugSomething isn't working
    bindingLogical-to-physical binding and implementation selection
    promqlPromQL front-end lowering (L1→L2)
    on Oct 15, 2026
  2. zzylol commented on Oct 22, 2026

    @zzylol
    ContributorAuthor

    Additional reproduction from the observability workload probe on c3be284c97c1f7ecdc490d78ddecf3e5f78c2007 (2026-10-22): the same update-column issue also blocks an outer Max.

    cargo run -p asap-devtools --bin stage_pipeline -- \
      --promql 'max(sum by (job) (rate(http_requests_total[5m])))' \
      --enforce-accuracy-guarantees --out grouped-max.json

    Rerun against the built binary: selected: null, cause executor_cannot_compile, reason:

    query run: invalid DAG: node 6: invalid DAG: summary update column missing or ambiguous
    

    The real motivating query is o11y-bench's worst-backend 5xx share:

    max((sum by (job) (rate(http_requests_total{status=~"5..",job=~"user-service|order-service|payment-service"}[6h]))) / (sum by (job) (rate(http_requests_total{job=~"user-service|order-service|payment-service"}[6h]))))
    

    For that query, both default and exhaustive search select no plan; all 9 exhaustive physical candidates fail executor compilation (node 13). Sources are exact counters, 1,000 series, 15s scrape interval, with a predictable query every 30s and both exact-accuracy fields zero.

    Controls: removing the outer max succeeds; max(rate(http_requests_total[5m])) succeeds. The grouped sum outputs [job, sum], while the outer summary uses SampleValue, resolved as the name value. This extends the existing regression scope from Sum-over-Sum to Max-over-Sum and Max-over-a-ratio-of-grouped-Sums. No numerical execution was performed in this probe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bindingLogical-to-physical binding and implementation selectionbugSomething isn't workingpromqlPromQL front-end lowering (L1→L2)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions