Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 40 additions & 20 deletions quest/m1/stats-prefix-tracks.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,17 @@ Decided 2026-10-05 and 2026-10-07:
Nothing is produced for a prefix no one holds. The relay's `--stats-depth`
only places the group broadcasts; requests are not capped by it. Track names
must stay injective: tiers and paths both contain `/`.
- Flow counters (bytes, frames, groups, fetches) count from zero when the
prefix is first held and fold in members that end while it is held. Their
absolute values are not authoritative, so readers take differences within
one subscription; the group totals carry the real sums.
- Started/ended pairs (announces, broadcasts, subscriptions) are seeded at
registration: `started` with each current member's `started - ended`,
`ended` at zero, so every `active` stays exact for a prefix held
mid-stream. Test a prefix held while 300 subscriptions are live, then watch
them leave.
- Counters follow presence (revised 2026-10-16, #5719): each drain sums the
live members' readouts plus the closing readouts of members that ended
while the prefix was held. Live members carry their history from before
the hold, so absolute values are not authoritative and readers take
differences within one subscription; the group totals carry the real sums.
Every `active` (started less ended) is exact for a prefix held mid-stream.
Test a prefix held while 300 subscriptions are live, then watch them leave.
- A request for a prefix with no members is held open at zero and reclaimed
when its last consumer leaves.
- A child leaves the one-level map after the frame carrying its closing
readout; its tail stays in the parent's rollup. Decide while implementing
whether a map with hundreds of children needs a cap or paging.
readout; its tail stays in the parent's rollup.
- **Self counters.** A prefix track also carries cumulative counters for
members at exactly the prefix, so a reader can show a broadcast that also has
nested broadcasts. A reader can't derive them as the rollup less the current
Expand All @@ -51,24 +48,47 @@ Decided 2026-10-05 and 2026-10-07:
upstream (node, epoch) hold and treats a decrease as that upstream's reset,
so a merged prefix track never regresses within one downstream
subscription. Test a join, a leave, and a restart under a held prefix.
- **Hot path.** Keep it lock-free: a held prefix registers atomic
accumulators, each entry caches its held ancestors (and the matching child
slot in each), and an update bumps them in a loop, about twice the held
depth per update. Registration and release reach existing entries without a
lock or a walk of every entry, for example an `ArcSwap` ancestor list or a
generation counter rechecked on bump. Benchmark across held-prefix count and
depth.
- **Drain-time sums** (revised 2026-10-16, #5719). Sum held prefixes in
`moq-stats` per drain with `Cumulative`, like presence: walk each report
row's ancestors and look each up in the held set, so the cost scales with
rows times path depth, not held prefixes, and nothing touches the per-frame
path. No `moq-net` API change: registry-level prefix accumulators would
push request lifecycle into the collection layer and put shared atomics on
every frame bump under a held prefix. Extend `producer_tick` to sweep rows,
held prefixes, and depth.
- Held prefixes have no cap: a hold is reclaimed when its last reader leaves,
like presence tracks, whose cap was dropped for that reason (decided
2026-10-12 in #5084).
- Decide whether prefix tracks keep `.json.z` siblings. Update the
aggregator, `doc/concept/stats.md`, and `doc/bin/inspect.md`.
- Update the aggregator, `doc/concept/stats.md`, and `doc/bin/inspect.md`.

Decided 2026-10-12 (#5084): the map tracks are kept, not retired. Prefix
tracks are added beside them so existing readers keep working, the same
backwards-compatible shape stats-split took when it kept `sessions.json`.
This supersedes the 2026-10-05 plan to retire the maps in the same release.

Decided 2026-10-16 (#5719):

- **Name.** `<path>/traffic.json`, where `<path>` is the full broadcast
prefix, as presence takes the full root, and a prefix outside the group is
refused. The last segment picks the kind and everything before it is the
key, like `<root>/presence.json`, so the name is injective without a
namespace segment. Tiers ride the payload, never the name, so one
subscription returns the whole prefix and no escaping rule is needed.
- **Payload.** Keyed by tier like `totals.json`. Each tier entry carries the
rollup's `publisher` and `subscriber` at its top level, so a totals reader
reads the rollup unchanged, plus `self: {publisher, subscriber}` and
`children: {"<segment>": {publisher, subscriber}}`, keyed by the child's
next path segment.
- **Encodings.** Plain and `.json.z` siblings, like every other stats track.
- **No cap or paging on children.** A cap would make the rollup disagree with
its visible children, and paging adds ordering and consistency rules no
measurement asks for yet. A child entry carries both roles per tier, so the
frame reaches the 32 MB `MAX_CACHE_BYTES` limit sooner than `publisher.json`
does (roughly ten thousand children across four tiers), far past the
hundreds a channel holds. An oversized frame must fail loud: end the prefix
track with `FrameTooLarge` instead of the existing tracks' debug log, which
leaves readers on a stale snapshot. Test a prefix whose frame overflows.

Public API: `moq-stats` producer and consumer types. Wire: stats track names
and payloads.

Expand Down
Loading