@effect/opentelemetry: MetricProducer hardcodes CUMULATIVE temporality, ignoring OTLPMetricExporter temporalityPreference

[alesk]

Summary

@effect/opentelemetry's MetricProducer does not query the registered MetricReader's selectAggregationTemporality(instrumentType) — it always hardcodes AggregationTemporality.CUMULATIVE on every Sum data point (Counter and UpDownCounter). This means any non-default temporality preference (e.g. temporalityPreference: DELTA on OTLPMetricExporter) is silently ineffective: temporality is baked into the data point by the producer before the exporter sees it.

Per the OTel spec, producers should query the reader for the preferred temporality per instrument type rather than choosing unconditionally.

Note: an earlier version of this issue claimed this causes Datadog to display inflated rates. That turned out to be a different bug on our side (we were missing { incremental: true } on Metric.counter, so the metric was exported as a non-monotonic Sum and DD treated it as a gauge). With incremental: true, DD ingests the cumulative monotonic Sum correctly via internal cumulative→delta conversion. The temporality-hardcode issue described here is real and reproducible, but its impact is narrower than I originally framed it. Apologies for the noise.

Reproduction

import { Effect, Metric } from "effect";
import { Metrics as EffectOtel, Resource } from "@effect/opentelemetry";
import {
    AggregationTemporality,
    InMemoryMetricExporter,
    PeriodicExportingMetricReader,
} from "@opentelemetry/sdk-metrics";

const exporter = new InMemoryMetricExporter(AggregationTemporality.DELTA);
const reader = new PeriodicExportingMetricReader({ exporter, exportIntervalMillis: 60_000_000 });

await Effect.runPromise(
    Effect.scoped(
        Effect.gen(function* () {
            const counter = Metric.counter("test.counter", { incremental: true });
            yield* Effect.succeed(5).pipe(Metric.increment(counter));
            yield* Effect.succeed(7).pipe(Metric.increment(counter));
            yield* Effect.promise(() => reader.forceFlush());
        }).pipe(
            Effect.provide(EffectOtel.layer(() => reader)),
            Effect.provide(Resource.layer({ serviceName: "test" }))
        )
    )
);

const data = exporter.getMetrics();
const counter = data[0].scopeMetrics[0].metrics[0];
console.log(counter.aggregationTemporality);   // 1 (CUMULATIVE) — expected 0 (DELTA)
console.log(counter.isMonotonic);              // true (good)
console.log(counter.dataPoints[0].value);      // 12 (good)

The InMemoryMetricExporter is constructed with AggregationTemporality.DELTA, but the produced data point arrives as CUMULATIVE. Same result with a real OTLPMetricExporter({ temporalityPreference: DELTA }).

Root cause

packages/opentelemetry/src/internal/metrics.ts (line numbers from v0.61.0):

if (MetricState.isCounterState(metricState)) {
    return {
        descriptor: makeDescriptor(metricPair),
        dataPointType: SdkMetrics.DataPointType.SUM,
        isMonotonic: descriptor.type === SdkMetrics.InstrumentType.COUNTER,
        aggregationTemporality: SdkMetrics.AggregationTemporality.CUMULATIVE,  // <-- hardcoded
        dataPoints: [{ /* ... */ }],
    };
}

The same hardcoded value appears for every Sum-typed path in the file (Counter, UpDownCounter, FrequencyMetric, etc.). No producer-side or layer-level option to override it.

Expected behavior

The producer should look up the reader's preferred temporality for the instrument type and stamp that on each data point — e.g. by calling reader.selectAggregationTemporality(instrumentType) and using the returned value. This is the spec-compliant approach.

Impact

Narrow but real. Affects pipelines where the configured backend requires (or strongly prefers) DELTA temporality and where the user expects temporalityPreference on the exporter to control that:

• Some OTel collector configurations downstream of @effect/opentelemetry that are themselves set to DELTA-only routing.
• Backends that do not handle cumulative monotonic Sums on ingest (most major vendors do, so this is genuinely niche).

It does not noticeably affect default OTLP-to-Datadog or Prometheus-pull setups, which handle cumulative monotonic Sums correctly.

Workaround

For users who genuinely need DELTA-temporality export and can't change the backend:

• Bypass @effect/opentelemetry for the affected instruments — e.g. emit via a different SDK (dd-trace's dogstatsd, raw OTel API) and keep @effect/opentelemetry for everything else.
• There's no in-library escape hatch today.

Suggested fix

Either:

1. Drive temporality from the registered MetricReader — query reader.selectAggregationTemporality(instrumentType) per produced data point and stamp accordingly. Spec-compliant; preserves all existing default behavior for users on CUMULATIVE-preferring readers.
2. Accept a temporality option on EffectOtel.layer / makeProducer — escape hatch: { counterTemporality: DELTA, upDownCounterTemporality: ... }. Simpler, less correct.

Happy to PR (1) if there's interest.

Environment

• @effect/opentelemetry 0.61.0
• @opentelemetry/sdk-metrics 2.2.0
• @opentelemetry/exporter-metrics-otlp-grpc 0.207.0

[mohamedramadan14]

Opened a fix in #6263.

Root cause confirmed at packages/opentelemetry/src/internal/metrics.ts — MetricProducerImpl.collect() hardcoded AggregationTemporality.CUMULATIVE on every Sum-typed data point (Counter, UpDownCounter, Frequency, Summary count/sum) and on Gauge/Histogram. It never consulted the registered MetricReader, so any temporalityPreference configured on the exporter (e.g. OTLPMetricExporter({ temporalityPreference: DELTA }) or any custom aggregationTemporalitySelector) was silently overridden.

Why suggested fix #1 needs a small adaptation: the OTel MetricProducer interface only receives { timeoutMillis } from the reader — not a reader reference — so the producer can't query selectAggregationTemporality directly during collect(). The reader is given to the producer one-way via reader.setMetricProducer(producer). The fix is for the producer to remember the reader at registration time and use it during collect().

What the fix does:

• MetricProducerImpl now holds a list of registered MetricReaders, populated in registerProducer.
• collect() calls reader.selectAggregationTemporality(descriptor.type) per produced data point and stamps that on the result. The 5 hardcoded sites (Counter, Gauge, Histogram, Frequency, Summary count/sum) all go through one helper.
• No reader registered → unchanged (CUMULATIVE, matching the OTel default).
• Multiple readers, all agree → use that preference.
• Multiple readers, disagree → use the first reader's preference and emit a single diag.warn(...) (non-fatal, non-breaking).

Tests added (all 60 in @effect/opentelemetry pass): a counter DELTA repro that mirrors the issue, a gauge DELTA case, and a no-reader CUMULATIVE fallback.

Out of scope (separate code path, calling out as a follow-up): packages/opentelemetry/src/OtlpMetrics.ts (the non-SDK direct-OTLP snapshot path) has the same hardcoded CUMULATIVE, but it doesn't use a MetricReader. A follow-up could add a temporality option to OtlpMetrics.make since there's no reader to query there.

PR: #6263