Skip to content

Classify attribute cardinality: identifier-like, growing, stable - #26

Open
zzylol wants to merge 2 commits into
feat/collector-containersfrom
feat/cardinality
Open

zzylol wants to merge 2 commits into
feat/collector-containersfrom
feat/cardinality

Conversation

@zzylol

@zzylol zzylol commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Part 4 of the RFC 0001 stack: use case 1, cardinality and uniqueness.

What it adds

  • per_attribute in asap_sketches: each point becomes one series per attribute key (attribute.key=<key>, "*" = every key) whose values are sketched. An HLL then counts distinct values per key.
  • A classifier shared by the detector and exact paths. Each key is compared with its own last 10 windows: identifier-like when distinct / points ≥ 0.9, growing when distinct is more than 20% above the baseline mean, otherwise stable. The class is a gauge (asap_cardinality_state), and class changes are OTLP log events sent to Loki.
  • The cardinality scenario: user.id unique, k8s.pod.name growing 5% per minute, http.route and http.method fixed. Plus a dashboard.
  • Fix: processors ack their consumed inputs. asap_sketches, window_merge and the answer processors keep their input in state but never acked it, so the official fanout held each message until timeout. That cut a branch to about a quarter of the offered rate: 640,000 instead of 2.4 million points per window. They now ack. Every window delivers the full rate.

Result (56-core x86 server, 1-minute windows, 2 × 20,000 points/s)

Cardinality dashboard

Key Exact Sketch Class (sketch / exact)
user.id 640,000 644,851 (+0.76%) identifier-like / identifier-like
k8s.pod.name 51 51 growing / growing (after baseline)
http.route 20 20 stable / stable
http.method 5 5 stable / stable

(The user.id row was measured before the ack fix, when the window held 640,000 points.)

cargo fmt --check, cargo clippy --all-targets --features otap-engine,stream-benchmark -- -D warnings and cargo test --features otap-engine,stream-benchmark pass at this PR's head.

asap_sketches gains per_attribute: each point becomes one series per
attribute key (attribute.key=<key>) whose values are sketched, so an HLL
counts distinct values per key. The detector and exact_answers classify each
key against its own recent windows (distinct/points >= 0.9 identifier-like;
> 20% above the baseline mean growing) and emit the class as a gauge and
class changes as OTLP log events (to Loki). Adds the cardinality scenario
and dashboard; stage configs move to one directory per use case.
asap_sketches, window_merge and the answer processors keep their input in
state instead of forwarding it, but never acked it. Upstream nodes that wait
for acks (the official fanout does by default) then held each message until
timeout, cutting a branch's throughput to about a quarter. Each processor now
acks the input once it has taken it.
@zzylol
zzylol force-pushed the feat/collector-containers branch from 80a9024 to 0f795de Compare October 13, 2026 00:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant