Skip to content

Add telemetry-function experiment results (inline cost, exact baseline, accuracy vs size) - #33

Open
zzylol wants to merge 1 commit into
docs/governance-and-rfcfrom
feat/experiment-results
Open

zzylol wants to merge 1 commit into
docs/governance-and-rfcfrom
feat/experiment-results

Conversation

@zzylol

@zzylol zzylol commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Last part (11) of the RFC 0001 stack: the measured results of experiments 1 (inline cost) and 2 (accuracy versus size), generated by report_experiments.py from experiments.py output (#30). Every stage is a pinned asap-collector container on 56-core x86 servers, with 10-second windows. Full tables and CSVs are in docs/results/experiments.md.

Inline cost: branch CPU with and without each function

Branch CPU with and without each function

Function Added branch CPU per point Branch CPU at 200,000 points/s per source
none (forwarding only) (about 0.3 µs in total) 0.06 cores
KLL latency quantiles 2.7–4.1 µs 0.60 cores
Count-Min + heap, top-k 3.7–5.3 µs 0.81 cores
HLL over 4 attribute keys 9–11 µs (about 2.3 µs per key) one core reached at about 100,000 points/s
  • The global stages (merge, detector) receive a few KB to 180 KB per window and use under 0.01 cores.
  • The exact baseline, which receives every raw point, needs a full core at about 95,000 points/s for cardinality and about 400,000 points/s for latency quantiles.
  • Answers arrive 2–3 s after a window ends (these runs used a 2-second merge grace).
  • These per-point costs put a branch below the ~1M points/s per core reference. The next step is a profile of the per-point path; decoding OTAP into observations and the per-attribute split are the likely costs, not yet measured.

Accuracy versus size (one window per setting)

Setting Error Merge bytes per window
KLL k = 50 / 100 / 200 / 400 p99 median error 13.3% / 7.0% / 7.5% / 2.0% (routes ≥ 20,000 points) 39 / 58 / 96 / 181 KB
Count-Min 256 / 512 / 1,024 / 2,048 columns, 100,000 keys recall@10 1.0 at every width; top-10 count error 6.0% / 2.8% / 1.3% / 0.46% 7 / 11 / 12 / 19 KB

Single windows, so neighbouring settings can cross (KLL k = 100 vs 200).

Inline cost (branch CPU with and without each function, 20k-200k points/s
per source), the exact baseline's own cost, and accuracy versus sketch size
(KLL k, Count-Min width over 100,000 keys), measured on 56-core x86 servers
with every stage a pinned asap-collector container. Generated by
report_experiments.py from experiments.py output.
@zzylol
zzylol force-pushed the docs/governance-and-rfc branch from 0965b48 to 17186b1 Compare October 12, 2026 22:09
@zzylol
zzylol force-pushed the feat/experiment-results branch from 67930ea to 4cf35de Compare October 12, 2026 22:09
@zzylol
zzylol force-pushed the docs/governance-and-rfc branch from 17186b1 to f263a6c Compare October 13, 2026 00:20
@zzylol
zzylol force-pushed the feat/experiment-results branch from 4cf35de to be08a0a Compare October 13, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant