Repository navigation
perf(ddsketch): recycle sketch state across windows via a pool - #418
Merged
Merged
Conversation
In window mode the processor swaps its per-metric Precompute map every
flush, so all ~30K per-series DDSketches are dropped and rebuilt each
60s window. The sketch store (bucket Vector1D) was therefore allocated +
grown from empty every window.
Add a cross-window sketch pool, owned by the (long-lived) processor so
it survives the per-window Precompute swap:
- precompute: a new optional SketchSink (SetSketchSink) is invoked by
finishRotate once a series' envelope is serialized — the symmetric
Put side of a host pool. nil sink keeps the prior drop/GC behavior.
- DDSketchWrapper.Reset() now does an in-place Clear (keeps the store's
backing-array capacity) instead of reallocating via NewDDSketch.
- ddsketchprocessor owns a sync.Pool; getOrCreate's factory draws from
it (Get) and the sink Resets + returns sketches (Put). Every sketch
in one processor shares the configured relative_accuracy, so a single
type+shape pool is correctness-safe. proc==nil (test batch path)
falls back to fresh sketches.
Steady-state effect (cold-off 1-sketch agent, 30K series, 100 Hz; heap
alloc diffed across two 60s windows): DDSketch store allocation
(Buckets.ensure / FilledVector1D / GrowZeroed) drops to ~0 — the pool
fully recycles backing arrays across windows. The one-time window-1 cold
allocation is the only store alloc; remaining steady-state alloc is all
framework OTLP decode + contrib hashkey, untouched by this change. CPU is
neutral (the store churn was ~2% of alloc, never CPU-bound) — the win is
GC pressure / allocation, not cycles.
Depends on ProjectASAP/sketchlib-go#55 (DDSketch.Clear + capacity reuse);
builds locally via the `=> ../../sketchlib-go` replace.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
In window mode the processor swaps its per-metric
Precomputemap every flush, so all ~30K per-series DDSketches are dropped and rebuilt each 60s window — the bucket store was allocated + grown from empty every window.Adds a cross-window sketch pool owned by the long-lived processor (survives the per-window Precompute swap):
SketchSink(SetSketchSink), invoked byfinishRotateonce a series' envelope is serialized — the Put side of a host pool. nil sink = prior drop/GC behavior.Clear(keeps store capacity) instead of reallocating viaNewDDSketch.sync.Pool;getOrCreate's factoryGets from it, the sinkResets +Puts. Onerelative_accuracyper processor ⇒ a single type+shape pool is correctness-safe.proc==nil(test batch path) falls back to fresh sketches.Why
Issue #46 agent perf. Completes the "pool the sketch since its shape is config-fixed" idea — the storage primitive is in sketchlib-go #55, this wires it through.
Steady-state effect (cold-off 1-sketch agent, 30K series, 100 Hz)
Heap
alloc_spacediffed across two 60s windows (pprof -base): DDSketch store allocation (Buckets.ensure/FilledVector1D/GrowZeroed) drops to ~0 — the pool fully recycles backing arrays across windows. The cumulative-from-start profile only showed ~1% because window-1 cold dominates; measured across steady-state windows the per-window store alloc is gone. Remaining steady-state alloc is entirely framework OTLP decode + contrib hashkey.CPU is neutral — the store churn was ~2% of alloc and never CPU-bound; the win is allocation / GC pressure, not cycles. (Part A — series-key pooling, #417 — is the CPU win.)
Correctness
quantile_over_time(0.99, http_requests_total_latency_ms[5m])⇒ p99 ≈ 90.9 ms, p50 ≈ 18.7 ms (plausible, over recycled sketches).Clear()+refill match a fresh sketch exactly; no stale-zero serialization.Depends on
ProjectASAP/sketchlib-go#55 (DDSketch.Clear + capacity reuse). Builds locally via the `=> ../../sketchlib-go` replace; merge #55 first.
🤖 Generated with Claude Code