diff --git a/quest/m1/stats-prefix-tracks.md b/quest/m1/stats-prefix-tracks.md index 42dacdccd2f..590a80a3b13 100644 --- a/quest/m1/stats-prefix-tracks.md +++ b/quest/m1/stats-prefix-tracks.md @@ -26,20 +26,17 @@ Decided 2026-10-05 and 2026-10-07: Nothing is produced for a prefix no one holds. The relay's `--stats-depth` only places the group broadcasts; requests are not capped by it. Track names must stay injective: tiers and paths both contain `/`. -- Flow counters (bytes, frames, groups, fetches) count from zero when the - prefix is first held and fold in members that end while it is held. Their - absolute values are not authoritative, so readers take differences within - one subscription; the group totals carry the real sums. -- Started/ended pairs (announces, broadcasts, subscriptions) are seeded at - registration: `started` with each current member's `started - ended`, - `ended` at zero, so every `active` stays exact for a prefix held - mid-stream. Test a prefix held while 300 subscriptions are live, then watch - them leave. +- Counters follow presence (revised 2026-10-16, #5719): each drain sums the + live members' readouts plus the closing readouts of members that ended + while the prefix was held. Live members carry their history from before + the hold, so absolute values are not authoritative and readers take + differences within one subscription; the group totals carry the real sums. + Every `active` (started less ended) is exact for a prefix held mid-stream. + Test a prefix held while 300 subscriptions are live, then watch them leave. - A request for a prefix with no members is held open at zero and reclaimed when its last consumer leaves. - A child leaves the one-level map after the frame carrying its closing - readout; its tail stays in the parent's rollup. Decide while implementing - whether a map with hundreds of children needs a cap or paging. + readout; its tail stays in the parent's rollup. - **Self counters.** A prefix track also carries cumulative counters for members at exactly the prefix, so a reader can show a broadcast that also has nested broadcasts. A reader can't derive them as the rollup less the current @@ -51,24 +48,47 @@ Decided 2026-10-05 and 2026-10-07: upstream (node, epoch) hold and treats a decrease as that upstream's reset, so a merged prefix track never regresses within one downstream subscription. Test a join, a leave, and a restart under a held prefix. -- **Hot path.** Keep it lock-free: a held prefix registers atomic - accumulators, each entry caches its held ancestors (and the matching child - slot in each), and an update bumps them in a loop, about twice the held - depth per update. Registration and release reach existing entries without a - lock or a walk of every entry, for example an `ArcSwap` ancestor list or a - generation counter rechecked on bump. Benchmark across held-prefix count and - depth. +- **Drain-time sums** (revised 2026-10-16, #5719). Sum held prefixes in + `moq-stats` per drain with `Cumulative`, like presence: walk each report + row's ancestors and look each up in the held set, so the cost scales with + rows times path depth, not held prefixes, and nothing touches the per-frame + path. No `moq-net` API change: registry-level prefix accumulators would + push request lifecycle into the collection layer and put shared atomics on + every frame bump under a held prefix. Extend `producer_tick` to sweep rows, + held prefixes, and depth. - Held prefixes have no cap: a hold is reclaimed when its last reader leaves, like presence tracks, whose cap was dropped for that reason (decided 2026-10-12 in #5084). -- Decide whether prefix tracks keep `.json.z` siblings. Update the - aggregator, `doc/concept/stats.md`, and `doc/bin/inspect.md`. +- Update the aggregator, `doc/concept/stats.md`, and `doc/bin/inspect.md`. Decided 2026-10-12 (#5084): the map tracks are kept, not retired. Prefix tracks are added beside them so existing readers keep working, the same backwards-compatible shape stats-split took when it kept `sessions.json`. This supersedes the 2026-10-05 plan to retire the maps in the same release. +Decided 2026-10-16 (#5719): + +- **Name.** `/traffic.json`, where `` is the full broadcast + prefix, as presence takes the full root, and a prefix outside the group is + refused. The last segment picks the kind and everything before it is the + key, like `/presence.json`, so the name is injective without a + namespace segment. Tiers ride the payload, never the name, so one + subscription returns the whole prefix and no escaping rule is needed. +- **Payload.** Keyed by tier like `totals.json`. Each tier entry carries the + rollup's `publisher` and `subscriber` at its top level, so a totals reader + reads the rollup unchanged, plus `self: {publisher, subscriber}` and + `children: {"": {publisher, subscriber}}`, keyed by the child's + next path segment. +- **Encodings.** Plain and `.json.z` siblings, like every other stats track. +- **No cap or paging on children.** A cap would make the rollup disagree with + its visible children, and paging adds ordering and consistency rules no + measurement asks for yet. A child entry carries both roles per tier, so the + frame reaches the 32 MB `MAX_CACHE_BYTES` limit sooner than `publisher.json` + does (roughly ten thousand children across four tiers), far past the + hundreds a channel holds. An oversized frame must fail loud: end the prefix + track with `FrameTooLarge` instead of the existing tracks' debug log, which + leaves readers on a stale snapshot. Test a prefix whose frame overflows. + Public API: `moq-stats` producer and consumer types. Wire: stats track names and payloads.