Repository navigation
feat: SDK pipeline configuration files (sketch type, transmit_sketch, instruments, ASAPQuery integration) - #55
Merged
Merged
Conversation
Introduce YAML config files for the opentelemetry-app SDK pipeline, covering all supported sketch types (DDSketch, KLL, HLL, CountSketch, CountMinSketch) and a raw-gauge baseline. Each config exposes: - sketch.type and sketch.transmit_sketch (SDK-side vs collector-side aggregation) - per-sketch tuning parameters (k, relative_accuracy, rows/cols, etc.) - instruments block: which metrics to aggregate, group_by label keys, series_per_sketch - asap_query block: mirrors ControllerConfig from asap-planner-rs (PR #174) with metrics, query_groups (accuracy_sla / latency_sla), and sketch_parameters - exporter, reader, and load generation knobs (replaces scattered CLI flags) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Document the new config.yaml / config-<sketch>.yaml files: - transmit_sketch semantics (SDK-side vs collector-side aggregation) - full config schema with all sections (exporter, reader, sketch, instruments, asap_query, load) - per-file table mapping sketch type to transmit_sketch value - asap_query block note linking to asap-planner-rs ControllerConfig schema Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire the YAML pipeline config files into the SDK and app layer. opentelemetry-go-patch/sdk/metric/pipelineconfig.go (new): - PipelineConfig struct hierarchy matching the YAML schema - LoadPipelineConfig(path) — reads and unmarshals the YAML file - ToAggregation() — maps sketch.type + params to AggregationXxx - ToViews() — builds Views from instruments[] when transmit_sketch:true; returns nil for baseline / transmit_sketch:false - ReaderInterval() — returns reader.interval with a 1s default opentelemetry-go-patch/sdk/metric/go.mod: - Promote go.yaml.in/yaml/v2 from indirect to direct dependency opentelemetry-app/cmd/fakemetricload/main.go: opentelemetry-app/cmd/e2esdkbench/main.go: - Add --config flag; LoadPipelineConfig() propagates YAML values into existing flag variables (fully backward compatible — CLI-only usage unchanged) - Provider construction uses pipelineCfg.ToViews() when config is loaded, so transmit_sketch is config-file driven opentelemetry-app/.gitignore: - Anchor binary patterns (/fakemetricload, /e2esdkbench) so they no longer accidentally ignore the cmd/ source directories Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
20 tests covering all three public methods and LoadPipelineConfig: LoadPipelineConfig: parses DDSketch, KLL, HLL, CountSketch, CountMinSketch, baseline, and ASAPQuery blocks; errors on missing file and invalid YAML. ToAggregation: table-driven across all 7 sketch types (including baseline and unknown → nil). ToViews: transmit_sketch:true returns one View per instrument; transmit_sketch:false returns nil; baseline returns nil regardless of transmit_sketch; no instruments returns empty slice. ReaderInterval: configured value, unset default (1s), zero default (1s). All 20 tests pass. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
Author
|
closes #56 |
fakemetricload: add baseline mode
- Valid sketch types now: ddsketch|kll|countsketch|countminsketch|hll|baseline
- baseline sets delivery=sdkGauge (raw LastValue gauge, no sketch view)
e2esdkbench: add countsketch and countminsketch modes
- Valid sketch types now: ddsketch|kll|hll|countsketch|countminsketch|baseline
- countsketch uses AggregationCountSketch{Epsilon, Delta}
- countminsketch uses AggregationCountMinSketch{Rows, Cols}
- Flags for both (--countsketch-epsilon/delta, --countmin-rows/cols) were
already present; only the validation switch and aggregation cases were missing
Both binaries now support the full set of modes symmetrically.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SieDeta
pushed a commit
that referenced
this pull request
Apr 17, 2026
feat: SDK pipeline configuration files (sketch type, transmit_sketch, instruments, ASAPQuery integration)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
opentelemetry-app/for the full SDK→collector pipeline, one per sketch type plus a raw-gauge baselinesketch.transmit_sketchas a first-class config knob:true= SDK pre-aggregates into a sketch payload;false= SDK emits raw gauges and the collector-side processor aggregatesasap_queryblock that mirrors theControllerConfigschema from asap-planner-rs PR #174 —metrics,query_groups(withaccuracy_sla/latency_sla), andsketch_parameters— so the same config can be consumed by asap-planner-rs directlyexporter,reader,sketch,instruments,loadFiles
transmit_sketchconfig.yamltrueconfig-ddsketch.yamltrueconfig-kll.yamltrueconfig-hll.yamltrueconfig-countminsketch.yamltrueconfig-countsketch.yamltrueconfig-baseline.yamlfalseSchema highlights
sketch.transmit_sketch— controls SDK-side vs collector-side aggregation pathinstruments[].group_by— attribute keys that form the sketch dimension (maps to--series-per-sketchand attribute labels in the existing apps)instruments[].series_per_sketch— 1=per-series, 0=all-in-one, N=matrix groupingasap_query.sketch_parameters— uses the same key names asSketchParameterOverridesin asap-planner-rs (DatasketchesKLL,CountMinSketch, etc.)exporter.max_send_msg_size_mb: 64included by default (required for HLL/CountSketch at scale)Test plan
fakemetricload/e2esdkbenchtransmit_sketch: trueconfigs produce sketch-typed OTLP data points at the collectorconfig-baseline.yaml(transmit_sketch: false) produces raw gauge data pointsasap_queryblock round-trips into asap-planner-rsControllerConfig🤖 Generated with Claude Code