Repository navigation
Conversation
zzylol
force-pushed
the
stack/univmon-l2-accuracy
branch
from
October 5, 2026 03:07
18cb590 to
ba716f2
Compare
zzylol
force-pushed
the
stack/satisfies-error-metric
branch
from
October 5, 2026 03:07
ba0c9e2 to
64ebe46
Compare
This was referenced Oct 5, 2026
zzylol
force-pushed
the
stack/univmon-l2-accuracy
branch
from
October 5, 2026 06:21
ba716f2 to
e34c85c
Compare
zzylol
force-pushed
the
stack/satisfies-error-metric
branch
from
October 5, 2026 06:21
64ebe46 to
4f65ff1
Compare
This was referenced Oct 5, 2026
Closed
zzylol
force-pushed
the
stack/satisfies-error-metric
branch
2 times, most recently
from
October 10, 2026 14:32
e30d21a to
ee6df66
Compare
Stage 3 compared only a summary guarantee's bound and failure probability with the target, so a bound in one metric (Count-Min's L1 frequency error, say) could satisfy a target on another statistic (a distinct count). `AccuracyModel::answers` maps each statistic to the metrics its epsilon is stated in, and `build_violation` checks it before `satisfies`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/satisfies-error-metric
branch
from
October 11, 2026 17:11
ee6df66 to
4423be6
Compare
This was referenced Oct 13, 2026
zzylol
added a commit
that referenced
this pull request
Oct 15, 2026
DefaultAccuracyModel::satisfies now compares a guarantee with a target only when both are stated in the same metric. An L2-scaled bound also meets an L1-scaled target of the same stream, since ‖f‖₂ ≤ ‖f‖₁. A bound in any other metric does not satisfy, however small, and an exact guarantee meets every target. This takes over the separate answers check proposed in #627. A target without a δ checks only the error. A per-value guarantee is still compared with a whole-result target directly until the union bound in #669b; a test pins this behavior. Plan selection's rejection reason now names the guarantee's metric. Example 2's Q3 (an L2 norm, whose root defaults to an absolute error) now names a relative target, the metric UnivMon certifies its L2 in.
zzylol
added a commit
that referenced
this pull request
Oct 15, 2026
DefaultAccuracyModel::satisfies now compares a guarantee with a target only when both are stated in the same metric. An L2-scaled bound also meets an L1-scaled target of the same stream, since ‖f‖₂ ≤ ‖f‖₁. A bound in any other metric does not satisfy, however small, and an exact guarantee meets every target. This takes over the separate answers check proposed in #627. A target without a δ checks only the error. A per-value guarantee is still compared with a whole-result target directly until the union bound in #669b; a test pins this behavior. Plan selection's rejection reason now names the guarantee's metric. Example 2's Q3 (an L2 norm, whose root defaults to an absolute error) now names a relative target, the metric UnivMon certifies its L2 in.
zzylol
added a commit
that referenced
this pull request
Oct 15, 2026
…are satisfies (#690) (#708) feat(types): three error metrics, a shared AccuracySpec and per-root targets (#690) ErrorMetric shrinks to Absolute, Relative and Rank. Absolute carries an optional norm scale (StreamL1 for Count-Min, StreamL2 for Count Sketch); Cardinality becomes Relative and TopKMembership is removed. AccuracySpec<E, P> {metric, error, failure_probability, scope} is the outer shape of both a target and a guarantee. AccuracyTarget is AccuracySpec<f64, Option<f64>> (no δ requirement when None), with exact()/epsilon()/epsilon_delta() constructors. AccuracyGuarantee holds a symbolic AccuracySpec<BoundExpr, ProbabilityExpr> plus its provenance. AccuracyScope is PerValue or WholeResult; guarantees are PerValue and targets default to WholeResult. Workloads and the query-language frontends take a RequestedAccuracy whose metric may be omitted. It still reads the old "Exact"/Epsilon/EpsilonDelta JSON. Plan selection resolves it per query root: counts and top-k default to an L1-scaled absolute error, distinct counts to a relative error, and any other root to an absolute value error. A quantile or histogram_quantile root without a metric is an input error. Example and test workloads with quantile targets now name Rank. A summary is still checked against its intent's target in its own native terms. stage_pipeline gains --metric. The committed planner-layering-example1 viewer fixture is regenerated for the new JSON shape; its selection is unchanged. feat(planner): metric-aware satisfies (#690) DefaultAccuracyModel::satisfies now compares a guarantee with a target only when both are stated in the same metric. An L2-scaled bound also meets an L1-scaled target of the same stream, since ‖f‖₂ ≤ ‖f‖₁. A bound in any other metric does not satisfy, however small, and an exact guarantee meets every target. This takes over the separate answers check proposed in #627. A target without a δ checks only the error. A per-value guarantee is still compared with a whole-result target directly until the union bound in #669b; a test pins this behavior. Plan selection's rejection reason now names the guarantee's metric. Example 2's Q3 (an L2 norm, whose root defaults to an absolute error) now names a relative target, the metric UnivMon certifies its L2 in. feat(types): L2 frequency-moment roots default to a relative error (#690) An L2 norm root without a named metric now resolves to a Relative target, like a distinct count; entropy stays Absolute. UnivMon certifies its L2 in a relative error, so Example 2's Q3 no longer names its metric and still selects the UnivMon L2 candidates. feat(dag-viewer): name the target's error metric in the editor (#690) The editor gains a metric dropdown next to ε (default, Absolute, Relative, Rank). /api/plan passes a non-empty metric to `stage_pipeline --metric` and rejects an unknown metric or a metric without ε. The query header shows the metric a target names. A quantile query planned with a metric selects a plan; without one, the page shows the missing-metric error.
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack: #574 → #620 → #618 → #621 → #627 → #625 → #628 → #632 → #634 → #616 → #617 → #622 → #624 → #629 → #630 → #631 → #633 → #635 → #636 → #637
Problem
DefaultAccuracyModel::satisfiescompares only a guarantee's bound and δ with the target. Stage 3'sbuild_violationtherefore accepts a bound stated in the wrong metric. For example, a Count-Min L1 frequency bound (Frequency) passes a target on a distinct count (Cardinality) whenever its number is small enough.Changes
AccuracyModel::answers(statistic, guarantee): a new trait method with a default implementation. It is a separate method becausesatisfies(guarantee, target)never sees the statistic. Its callers in Pass 1 check composed root guarantees, where noSketchStatisticexists.An exact guarantee answers every statistic.
Otherwise the guarantee's metric must be one the statistic's ε is stated in:
Rank,RelativeValueCardinality,RelativeValueRelativeValueFrequency,L2FrequencyFrequency,L2Frequency,TopKMembership(the score bound Pass 1 already checks top-k targets against)A deployment with a registered cross-metric conversion can override
answers.EstimatorAccuracydelegates it to its base model.build_violationchecksanswersbeforesatisfies. On a mismatch it rejects with "… guarantees a bound, which does not bound ".answers_requires_the_statistics_metric.a_bound_in_another_metric_misses_the_target: a CountSketch+heap build passes for its TopK readout, but is rejected for Cardinality even at ε = δ = 0.5.No example's selection changes. Examples planner-layering-1, 2, 3a, 3b, 4a and 4b select the same plans as before, and none of their rejections has the new reason.
Test plan
cargo fmt --all --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspace: 1679 passed, 0 failed, 21 ignored (re-run after the rebase on main d4869a7)Stacked on #621.