Describe your environment
OS: macOS (arm64)
Python version: 3.12
SDK version: main at 18d7e24 (opentelemetry-sdk installed from source)
API version: main at 18d7e24
What happened?
With DELTA temporality, the exponential histogram aggregation lowers its scale when an interval's values span a wide range, but never raises it again. Collection resets the buckets but not self._mapping (_ExponentialBucketHistogramAggregation.collect, the DELTA→DELTA branch in opentelemetry-sdk/src/opentelemetry/sdk/metrics/_internal/aggregation.py). So every later delta point keeps the lowest scale any earlier interval needed, even when it holds a single value.
The SDK spec says: "When the histogram contains not more than one value in either of the positive or negative ranges, the implementation SHOULD use the maximum scale." A delta point covers only its own interval.
Steps to Reproduce
from opentelemetry.sdk.metrics import Histogram, MeterProvider
from opentelemetry.sdk.metrics.export import AggregationTemporality, InMemoryMetricReader
from opentelemetry.sdk.metrics.view import ExponentialBucketHistogramAggregation, View
reader = InMemoryMetricReader(preferred_temporality={Histogram: AggregationTemporality.DELTA})
provider = MeterProvider(
metric_readers=[reader],
views=[View(instrument_name="latency", aggregation=ExponentialBucketHistogramAggregation())],
)
hist = provider.get_meter("demo").create_histogram("latency")
def point():
return reader.get_metrics_data().resource_metrics[0].scope_metrics[0].metrics[0].data.data_points[0]
hist.record(1e-6)
hist.record(1e6)
print(point().scale) # interval 1: 2
for _ in range(3):
hist.record(5.0)
print(point().scale) # each later interval holds one value: 2, 2, 2
Expected Result
Each interval after the first holds a single value, so it's exported at the maximum scale (20 by default), with positive.offset = 2434718.
Actual Result
Every later interval is exported at scale 2, with positive.offset = 9. That bucket is (4.757, 5.657], a relative error of about 8.6% instead of about 3.3e-7 at scale 20. It never recovers.
In a worse case, one interval that records 1e-300 and 1e300 pins later intervals at scale -4. After that, four measurements of 10, 12, 15 and 20 ms all land in the single bucket (2^-16, 1].
Additional context
Adding self._mapping = self._new_mapping(self._max_scale) in that DELTA→DELTA branch, next to the bucket reset, restores the maximum scale for every interval. With it, the reproduction above prints 20 for each later interval, and the outlier case goes back to scale 7 with the four values in separate buckets. The 335 tests in opentelemetry-sdk/tests/metrics pass with and without the change, so none of them currently covers this.
For comparison, jmacd noted during review of #2964 that when the structure goes from 0 measurements to 1, "the scale is supposed to reset to the maximum scale again".
Found while auditing the metrics SDK with Claude Code. The outputs above come from running the script against main at 18d7e24.
Would you like to implement a fix?
Not at the moment. The one-line change above is what I tested, and anyone is welcome to pick it up.
Describe your environment
OS: macOS (arm64)
Python version: 3.12
SDK version:
mainat 18d7e24 (opentelemetry-sdkinstalled from source)API version:
mainat 18d7e24What happened?
With DELTA temporality, the exponential histogram aggregation lowers its scale when an interval's values span a wide range, but never raises it again. Collection resets the buckets but not
self._mapping(_ExponentialBucketHistogramAggregation.collect, the DELTA→DELTA branch inopentelemetry-sdk/src/opentelemetry/sdk/metrics/_internal/aggregation.py). So every later delta point keeps the lowest scale any earlier interval needed, even when it holds a single value.The SDK spec says: "When the histogram contains not more than one value in either of the positive or negative ranges, the implementation SHOULD use the maximum scale." A delta point covers only its own interval.
Steps to Reproduce
Expected Result
Each interval after the first holds a single value, so it's exported at the maximum scale (20 by default), with
positive.offset= 2434718.Actual Result
Every later interval is exported at scale 2, with
positive.offset= 9. That bucket is (4.757, 5.657], a relative error of about 8.6% instead of about 3.3e-7 at scale 20. It never recovers.In a worse case, one interval that records 1e-300 and 1e300 pins later intervals at scale -4. After that, four measurements of 10, 12, 15 and 20 ms all land in the single bucket (2^-16, 1].
Additional context
Adding
self._mapping = self._new_mapping(self._max_scale)in that DELTA→DELTA branch, next to the bucket reset, restores the maximum scale for every interval. With it, the reproduction above prints 20 for each later interval, and the outlier case goes back to scale 7 with the four values in separate buckets. The 335 tests inopentelemetry-sdk/tests/metricspass with and without the change, so none of them currently covers this.For comparison, jmacd noted during review of #2964 that when the structure goes from 0 measurements to 1, "the scale is supposed to reset to the maximum scale again".
Found while auditing the metrics SDK with Claude Code. The outputs above come from running the script against
mainat 18d7e24.Would you like to implement a fix?
Not at the moment. The one-line change above is what I tested, and anyone is welcome to pick it up.