Skip to content

feat(ddsketch): quantile_interpolated, the PromQL / percentile_cont convention - #248

Merged
GordonYuanyc merged 2 commits into
mainfrom
feat/ddsketch-quantile-interpolated
Oct 27, 2026
Merged

GordonYuanyc merged 2 commits into
mainfrom
feat/ddsketch-quantile-interpolated

Conversation

@zzylol

@zzylol zzylol commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Why

ASAPPlanner answers PromQL quantile_over_time and SQL percentile_cont from a DDSketch. Both interpolate linearly between the two order statistics adjacent to rank q · (n − 1). The native DdSketch only has get_value_at_quantile, which uses the nearest-rank convention (sorted[ceil(q·n) − 1]). On [20, 40] at q = 0.9, nearest rank gives 40; PromQL gives 38. The legacy portable DdSketch had an interpolated readout (#140, signed stores from #141), but #241 removed that type.

What

  • New: DdSketch::quantile_interpolated(q) -> Result<Option<f64>, String>.
    • It interpolates linearly between the order statistics at ranks floor(q(n−1)) and ceil(q(n−1)) (0-based).
    • It reads the negative store, the zero bucket and the positive store, in value order.
  • Edge cases:
    • An empty sketch returns Ok(None).
    • q outside [0, 1], NaN included, returns Err. Mapping it to ±Inf, as PromQL does, is the caller's job.
    • The sketch never holds NaN: try_add refuses non-finite values.
    • Ranks are computed in f64, which is exact up to 2^53 samples.
  • Accuracy:
    • Rank 0 and rank n − 1 are the exactly retained minimum and maximum. Other ranks read their bucket's representative, clamped to [min, max].
    • When both neighbours are indexable and share a sign, the result is within alpha relative of the exact interpolated value.
    • Across zero the bound is absolute, alpha · ((1 − f)|lower| + f|upper|) for fractional rank f. It need not be small relative to a result near zero.
    • Tiny magnitudes counted in the zero bucket read as 0.
  • get_value_at_quantile is unchanged.

How

value_at_rank(r) walks the negative store from the most negative bucket, then the zero count, then the positive store, and returns the bucket that holds 0-based rank r. It is the same walk get_value_at_quantile does. quantile_interpolated reads the two neighbouring ranks and blends them by the fractional rank. When the rank is an integer it reads only one.

Before

let mut s = DdSketch::new(0.01);
s.add(&20.0);
s.add(&40.0);
s.get_value_at_quantile(0.9); // Some(40.0): nearest rank, no interpolated readout

After

let mut s = DdSketch::new(0.01);
s.add(&20.0);
s.add(&40.0);
let p90 = s.quantile_interpolated(0.9).unwrap().unwrap(); // 38.0
assert!((p90 - 38.0).abs() <= 0.01 * 38.0);
assert!(s.quantile_interpolated(1.5).is_err());
assert_eq!(DdSketch::new(0.01).quantile_interpolated(0.5).unwrap(), None);

Verification

Tests were written first and failed to compile before the method existed:

  • quantile_interpolated_two_values: [20, 40] at q = 0, 0.5, 0.9, 1 gives 20, 30, 38, 40 within alpha; nearest rank still gives 40 at 0.9.
  • quantile_interpolated_one_value_and_empty.
  • quantile_interpolated_refuses_q_outside_the_unit_interval: −0.1, 1.1, NaN and ±inf.
  • quantile_interpolated_signed_values: [−40, −20, −5, 0, 10, 30, 70] at 101 values of q, held to the absolute bound everywhere and to alpha relative where the neighbours share a sign.
  • quantile_interpolated_matches_the_exact_value_on_a_large_stream: 20,000 seeded values ±exp(U(−3, 6)), 20% negative, at 201 values of q, against the exact interpolated quantile.

Checks: cargo fmt --all -- --check; clippy -D warnings with --all-features, default features, octo-runtime and experimental; cargo test --all-features --locked (2,238 passed) and cargo test --locked (1,982 passed); RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features --locked.

…onvention

DdSketch::quantile_interpolated(q) interpolates linearly between the order
statistics at ranks floor(q (n - 1)) and ceil(q (n - 1)), the convention of
PromQL quantile_over_time and SQL percentile_cont; get_value_at_quantile
keeps its nearest-rank convention. The legacy portable DdSketch had it
(#140) before #241 removed that type.

- Ranks 0 and n - 1 read the exact min and max; other ranks read their
  bucket's representative from the negative store, the zero bucket or the
  positive store, clamped to [min, max].
- Ok(None) on an empty sketch; Err for q outside [0, 1], NaN included.
- Within alpha relative when both neighbours are indexable and share a
  sign; across zero the bound is alpha ((1 - f) |lower| + f |upper|).

Tests, written first: [20, 40] at q = 0.9 gives 38, one value, empty and
out-of-range q, signed values across both stores and the zero bucket, and
a 20,000-value signed stream against the exact interpolated quantile.
@zzylol zzylol added the enhancement New feature or request label Oct 21, 2026
@GordonYuanyc
GordonYuanyc merged commit e192e63 into main Oct 27, 2026
5 checks passed
@GordonYuanyc
GordonYuanyc deleted the feat/ddsketch-quantile-interpolated branch October 27, 2026 20:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants