Repository navigation
feat(ddsketch): quantile_interpolated, the PromQL / percentile_cont convention - #248
Merged
Merged
Conversation
…onvention DdSketch::quantile_interpolated(q) interpolates linearly between the order statistics at ranks floor(q (n - 1)) and ceil(q (n - 1)), the convention of PromQL quantile_over_time and SQL percentile_cont; get_value_at_quantile keeps its nearest-rank convention. The legacy portable DdSketch had it (#140) before #241 removed that type. - Ranks 0 and n - 1 read the exact min and max; other ranks read their bucket's representative from the negative store, the zero bucket or the positive store, clamped to [min, max]. - Ok(None) on an empty sketch; Err for q outside [0, 1], NaN included. - Within alpha relative when both neighbours are indexable and share a sign; across zero the bound is alpha ((1 - f) |lower| + f |upper|). Tests, written first: [20, 40] at q = 0.9 gives 38, one value, empty and out-of-range q, signed values across both stores and the zero bucket, and a 20,000-value signed stream against the exact interpolated quantile.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ASAPPlanner answers PromQL
quantile_over_timeand SQLpercentile_contfrom a DDSketch. Both interpolate linearly between the two order statistics adjacent to rankq · (n − 1). The nativeDdSketchonly hasget_value_at_quantile, which uses the nearest-rank convention (sorted[ceil(q·n) − 1]). On[20, 40]atq = 0.9, nearest rank gives 40; PromQL gives 38. The legacy portableDdSketchhad an interpolated readout (#140, signed stores from #141), but #241 removed that type.What
DdSketch::quantile_interpolated(q) -> Result<Option<f64>, String>.floor(q(n−1))andceil(q(n−1))(0-based).Ok(None).qoutside[0, 1], NaN included, returnsErr. Mapping it to±Inf, as PromQL does, is the caller's job.try_addrefuses non-finite values.f64, which is exact up to2^53samples.n − 1are the exactly retained minimum and maximum. Other ranks read their bucket's representative, clamped to[min, max].alpharelative of the exact interpolated value.alpha · ((1 − f)|lower| + f|upper|)for fractional rankf. It need not be small relative to a result near zero.get_value_at_quantileis unchanged.How
value_at_rank(r)walks the negative store from the most negative bucket, then the zero count, then the positive store, and returns the bucket that holds 0-based rankr. It is the same walkget_value_at_quantiledoes.quantile_interpolatedreads the two neighbouring ranks and blends them by the fractional rank. When the rank is an integer it reads only one.Before
After
Verification
Tests were written first and failed to compile before the method existed:
quantile_interpolated_two_values:[20, 40]atq = 0, 0.5, 0.9, 1gives20, 30, 38, 40withinalpha; nearest rank still gives 40 at 0.9.quantile_interpolated_one_value_and_empty.quantile_interpolated_refuses_q_outside_the_unit_interval:−0.1,1.1, NaN and±inf.quantile_interpolated_signed_values:[−40, −20, −5, 0, 10, 30, 70]at 101 values ofq, held to the absolute bound everywhere and toalpharelative where the neighbours share a sign.quantile_interpolated_matches_the_exact_value_on_a_large_stream: 20,000 seeded values±exp(U(−3, 6)), 20% negative, at 201 values ofq, against the exact interpolated quantile.Checks:
cargo fmt --all -- --check; clippy-D warningswith--all-features, default features,octo-runtimeandexperimental;cargo test --all-features --locked(2,238 passed) andcargo test --locked(1,982 passed);RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features --locked.