Skip to content

metrics: emit goodput track as a point attribute so the bandit evaluator can slice it#675

Merged
reflog merged 1 commit into
mainfrom
reflog/goodput-track-point-attr
Jun 30, 2026
Merged

metrics: emit goodput track as a point attribute so the bandit evaluator can slice it#675
reflog merged 1 commit into
mainfrom
reflog/goodput-track-point-attr

Conversation

@reflog

@reflog reflog commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

What

proxy.session.goodput now carries track as a point (datapoint) attribute keyed track, in addition to the existing proxy.track resource attribute.

Why

The goodput histogram (#674) tagged track only via the OTEL resource (semconv.ProxyTrackKey = "proxy.track"). SigNoz's metrics pipeline doesn't expose resource attributes as queryable metric labels, and the bandit experiment evaluator (lantern-cloud cmd/api/experiment/signoz.go) groups and filters goodput by the point attribute literally named track — the way lantern-box emits it.

Result: the evaluator's track IN [challenger, control] query matched sing-box/lantern-box tracks but never http-proxy tracks. Verified on prod:

  • track = 'hysteria2-oci-free-vps' (sing-box) → goodput data
  • track = 'blastoise-vps' (busy http-proxy incumbent, 21 routes, 24h) → 0; all http-proxy goodput sat in an untagged track="" bucket.

So every http-proxy challenger and http-proxy control arm read 0 goodput, and no http-proxy-backed bandit experiment could ever reach a verdict (it would hold until the 14-day max-gathering cutoff and retire inconclusive). This blocks the eng#3651 market-pivot experiments, whose challengers are http-proxy (tls_*) tracks.

Change

  • instrument.SessionGoodput emits attribute.String("track", ins.track) as a point attribute alongside geo.country.iso_code and network.io.direction.
  • track is threaded into defaultInstrument via NewDefault from the proxy's existing -track flag (p.Track).
  • The resource still carries proxy.track for all other signals — unchanged.
  • cloud.region is intentionally left a resource attribute: the evaluator's strata are (track, country) and a challenger is pinned to one DC, so region as a point attr would add cardinality with no decision value.

Tests

go build ./instrument/... + go vet clean; go test ./instrument/ -run TestSessionGoodput passes. Extended TestSessionGoodput to assert the track point attribute is present and equals the configured track. (Repo-wide go build ./... fails only on the pre-existing go-libutp CGO/compiler incompatibility, unrelated to this change.)

Rollout note

After merge: rebuild the http-proxy image and bump bandit_vps_http_proxy_default_short_tag in lantern-cloud so new VPS provisions emit track-tagged goodput. Existing experiment challengers were provisioned on the prior image; they'll pick up the new image on VPS autoreplace/recreate (or just let the current 4 retire and let post-rollout experiments be the measurable ones).

Summary by CodeRabbit

  • New Features

    • Added tracking information to session goodput metrics, improving visibility into proxy activity reporting.
    • Metrics now include the proxy’s track label alongside existing location details.
  • Tests

    • Updated metric tests to verify the new track label is included in emitted goodput data.

…tor can slice it

proxy.session.goodput tagged track only via the OTEL resource (semconv.ProxyTrackKey
= "proxy.track"). SigNoz's metrics pipeline doesn't expose resource attributes as
queryable labels, so the bandit experiment evaluator — which groups/filters goodput
by the point attribute "track" (matching lantern-box) — saw zero samples for every
http-proxy track: a busy http-proxy incumbent returned 0 for track='<name>' while a
sing-box track returned data. Every http-proxy challenger/control arm therefore read
0 goodput, so no http-proxy experiment could ever reach a verdict.

Emit track as a point attribute keyed "track" on the goodput histogram (threaded
through NewDefault from the proxy's -track flag). The resource keeps proxy.track for
everything else. cloud.region is intentionally left a resource attr: the evaluator's
strata are (track, country) and a challenger is pinned to one DC, so region would add
cardinality without decision value.
@reflog reflog merged commit 35a971c into main Jun 30, 2026
1 of 2 checks passed
@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: fff4fb2c-c856-40b4-b942-ce0de4861f25

📥 Commits

Reviewing files that changed from the base of the PR and between b0d6915 and ccf4407.

📒 Files selected for processing (4)
  • http_proxy.go
  • instrument/goodput_test.go
  • instrument/instrument.go
  • instrument/otelinstrument/otelinstrument.go

📝 Walkthrough

Walkthrough

NewDefault gains a track string parameter stored on defaultInstrument. SessionGoodput records this value as a track point attribute on the proxy.session.goodput histogram. ListenAndServe passes p.Track to NewDefault, and tests are updated to supply and assert the new attribute.

Changes

Track attribute in goodput instrumentation

Layer / File(s) Summary
defaultInstrument track field and SessionGoodput attribute
instrument/instrument.go, instrument/otelinstrument/otelinstrument.go
defaultInstrument gains a track field; NewDefault signature adds a track parameter and initializes the field; SessionGoodput includes track as a point attribute when recording the histogram. The otelinstrument comment clarifying which dimensions are point vs. resource attributes is updated accordingly.
Proxy wiring and test assertions
http_proxy.go, instrument/goodput_test.go
ListenAndServe passes p.Track to NewDefault. Tests supply "test-track" to NewDefault and assert the histogram sample carries a track attribute equal to "test-track".

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

  • getlantern/http-proxy#674: Directly related — introduces the proxy.session.goodput histogram that this PR extends with the track point attribute.

Poem

🐇 A track was lost, now found and stored,
Stitched into metrics, carefully poured.
The goodput histogram hops along,
With track as its label, precise and strong.
No attribute left behind! 🎉

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch reflog/goodput-track-point-attr

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant