Repository navigation
Conversation
acceptance.py runs each use case (latency, cardinality, drift, top-k) on the real collector containers with time scaled 6x and checks sketch answers against the exact ground truth through Prometheus and Loki: every stage up, full rate delivered, identical points per window, bounds containing the exact answer, classes, drift and novelty detected with no false positives, events in Loki. Exits non-zero on any failure. experiments.py measures branch CPU with and without each function over offered rates (inline cost) and error vs sketch size (KLL k, Count-Min width). Answer paths report asap.answer.delay, seconds from a window's end to its answers.
zzylol
force-pushed
the
feat/e2e-acceptance
branch
from
October 13, 2026 00:20
9a62ef4 to
ff1cc27
Compare
zzylol
force-pushed
the
feat/merge-bench
branch
from
October 13, 2026 00:20
ebec58d to
3191eeb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 8 of the RFC 0001 stack: the end-to-end acceptance, plus the tooling for experiments 1 and 2. The measured results come in the last PR of the stack.
deploy/telemetry-functions/acceptance.pyRuns all four use cases on the real collector containers (generators, branches, merge, detector, exact baseline, Prometheus, Loki). It checks the sketch answers against the exact ground truth through Prometheus and Loki, prints PASS or FAIL per check, and exits non-zero on any failure.
To keep it to a few minutes per use case, time is scaled down 6×: 10-second windows, a 2-second grace (raised back to the deployed 5 s in #31, after the first real CI run showed sketches arriving later than 2 s on a shared 4-CPU runner), changes at second 100, and growth 6× faster. Rules counted in windows (baselines, thresholds) are as deployed. Comparisons are pinned to one window answered by both methods, with PromQL's
@modifier.Result on a 56-core x86 server: 25/25 checks passed.
/new/hot-routein both top-10s; recall@10 = 1.00; novelty event in LokiInformational, not a pass condition: with 10-second windows, the p99 relative error for routes with ≥ 20,000 points reached 12.8%. That is KLL's rank error in a log-normal tail (see #25).
experiments.pyandreport_experiments.pyinline-cost: branch CPU with and without each function, over offered rates. In this run the exact stage only counts points, so it cannot throttle the branch.accuracy-size: error against exact as KLLkand Count-Min width change, with the bytes merge receives per window.asap_answer_delay: seconds from a window's end to its answers.report_experiments.pyturns the CSVs into a Markdown report with tables and a figure.Also includes
PIPELINESand the published Loki port in compose,run.sh --build-only, and a README covering every use case.cargo fmt --check,cargo clippy ... -D warningsandcargo test --features otap-engine,stream-benchmarkpass at this PR's head.