diff --git a/quest/m1/README.md b/quest/m1/README.md index d1018cae575..e1ab2334f4c 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -61,8 +61,7 @@ session outcomes, and embedder refusals inside session outcomes. - [Per-frame arrivals in @moq/watch](/quest/m1/watch-arrivals.md) - the audio and video decoders expose a window of recent frame arrivals, late and skipped marks included, as a signal - [Bump web-transport-iroh for the capsule close](/quest/m1/iroh-capsule-bump.md) - condition: moq-dev/web-transport#419 ships in a release, then moq's iroh HTTP/3 client reports a peer's close capsule - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - every stream group in a lite subscription arrives or is dropped by name, and lite-07 drops its stream count for it -- [lite-07 Live flag](/quest/m1/lite-live.md) - a separate `Live` field on lite-07 SUBSCRIBE, so merged floors never starve a subscriber; late lower groups build on it -- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges in either order and replaces FETCH; relays fill misses by range, including over moq-transport +- [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Reproduce cross-node announce flapping](/quest/m1/cross-node-flapping.md) - a clean close reaches a client on another node as one retraction, checked by a 3+ relay drill on main - [Untimed lite-07](/quest/m1/lite-untimed.md) - lite-07 carries an untimed track in both languages; lite-05/06 write send time - [A watch and publish release ships assets()](/quest/m1/assets-release.md) - the release that lets the sites host the worklets diff --git a/quest/m1/cache-max-age.md b/quest/m1/cache-max-age.md index f2a88cc5a44..5e2a33802e3 100644 --- a/quest/m1/cache-max-age.md +++ b/quest/m1/cache-max-age.md @@ -126,7 +126,7 @@ encoding change; Max Age semantics in the lite draft change. ## Related -- [lite-07 Live flag](/quest/m1/lite-live.md) - its untimed `Live` start (the latest group) follows this rule instead: replay what is not stale +- [Model ranges](/quest/m1/subscribe-ranges/model.md) - an untimed `Absolute(0)..` range follows this rule: replay what is not stale - [JS track handover](/quest/m1/js-group-handover.md) - mirrors the failover rule in JS - [Generated @moq/net](/quest/m1/rs2ts/README.md) - retires js/net's hand-written model - [FETCH max-delay](/quest/m1/fetch-max-delay.md) - applies the shared rule to historical requests and reconciles native IETF retention clocks diff --git a/quest/m1/lite-live.md b/quest/m1/lite-live.md deleted file mode 100644 index deadbc9ebeb..00000000000 --- a/quest/m1/lite-live.md +++ /dev/null @@ -1,93 +0,0 @@ -# [M] lite-07 Live flag - -## Goal - -Subscribers with different group and frame floors that share one upstream -subscription, in-process or through a relay, each receive everything their own -floor and `Subscriber Max Age` allow. No subscriber's floor starves another: -a resumed subscriber whose floor sits above the live edge never hides the -latest group from a subscriber that wants it. - -Today a floor and "no floor" are one field. On lite-06 `Group Start` 0 means no -floor, and the model keeps a separate `None` from pre-06's "absent means the -latest group". Merging an explicit floor with `None` either drops the late -group the floor asked for (main) or starves the floorless subscriber until the -floor's group exists, which on a quiet catalog track is never (#5000, the -failure #4940 fixed from the other side). - -## Plan - -Decided (maintainer, 2026-10-07): - -- lite-07 SUBSCRIBE and SUBSCRIBE_UPDATE carry a separate `Live` boolean. - `Group Start` and `Frame Start` become a plain absolute floor, with no - "0 means none" sentinel. -- A live-only subscriber has no floor, the neutral value for the minimum, so - lite-07 makes the floor optional. A subscription with neither a floor nor - `Live` is refused. The draft names how an absent floor is encoded (not a - value-plus-one sentinel) and how SUBSCRIBE_UPDATE clears a floor or `Live`. -- `Live: true` lowers the floor to frame 0 of the oldest group that - `Subscriber Max Age` has not expired, when that position sorts below the - floor. That is the start lite-06 already resolves for `Group Start` 0, so a - buffering subscriber keeps its backlog, and a max age of 0 still means the - latest group. On an untimed track the wall clock alone judges staleness - ([One max_age meaning](/quest/m1/cache-max-age.md)), so `Live` starts at - the oldest group that isn't stale there too; until that lands it means the - latest group (as `model/subscription.rs` documents today). Comparing - positions, not groups, means a merged floor of (3, 5) with group 3 as the - latest still delivers frames 0-4. -- [Model ranges](/quest/m1/subscribe-ranges/model.md) replaces the floor - with ranges after this lands: the floor becomes the lowest range start, and - `Live` stays its own flag. -- The model mirrors the wire: `Subscription { floor: Option<(group, frame)>, - live }` in Rust and `@moq/net`. -- Merging across subscribers (`Subscription::poll_combined`, JS - `combineSubscriptions`): `live` is the OR, and the floor is the minimum - `(group, frame)`. Each subscriber's own cursor still filters what it sees. -- Older wires map at the codec only, with no change to a published version: - lite-06 `Group Start` 0 with `Frame Start` 0 is `live`, any other pair a - floor (`(0, N)` is a catalog resume); pre-06 absent is `live`. - moq-transport has no current-group filter, so `live` is the bridge - `ietf::subscriber::subscribe_join` already sends: `Largest Object` plus the - current group's prefix, from a `Relative(1)` fill on draft-20+ or a relative - joining FETCH on older drafts. `Next Group Start` (draft-20 `Relative(0)`) - keeps its draft meaning, a floor at the group after Largest Object; a bare - `Largest Object` is a floor at the next object; draft-20 `Relative(1)` is - `live`; `Absolute Start` is a floor. -- When merged subscriptions need both `live` and a floor that an older - upstream wire cannot express in one SUBSCRIBE, a moq-transport upstream is - subscribed from `Absolute Start` at group 0, and the relay filters locally - by age and floor. The relay can't learn the oldest unexpired group - upstream: the ordinary join (`subscribe_join`) fetches only the latest - group's prefix. Decided 2026-10-08: no TRACK_STATUS step to find the - largest group first, since it covers only an unbuffered `live` and adds a - round trip. For an empty track the `live` mapping alone covers both, since - moq-transport starts it at {0, 0}. moq-lite has no - TRACK_STATUS, so a lite-06 or lite-03 to 05 upstream is subscribed at floor - 0 and the relay filters locally. On lite-06 that is `Group Start` 0, exactly - the `live` mapping. On lite-03 to 05 the floor must go out as an explicit - group 0 (replay from the beginning): the codec folds a floor of 0 to absent - today (`encode_start_group`), which those versions read as the latest - group, so a merged `{floor: 2, live}` with groups 2-4 retained would lose - 2-3. lite-01 and 02 carry no `Group Start` at all, so a merged floor through - them gets what they send from the latest group; that is a limitation of - those published versions, not something the codec can fix. -- Docs stay inline: the lite draft (field, semantics, lite-07 changelog), - `doc/concept`, and the Rust and JS API docs. - -Test with mock time: the starvation case (a floor-4 subscriber and a `live` -subscriber on a quiet track whose newest group is 3) in-process and through a -relay on lite-06 and lite-07-wip, a buffering `live` subscriber (max age > 0) -merged with a floor above the live edge, a frame floor merged with `live`, a -merged `{floor: 2, live}` through a lite-03 to 05 upstream with groups 2-4 -retained that delivers all three, a buffered `live` merged with a floor-5 -subscriber over a moq-transport upstream with fresh groups 2-4 that still -delivers 2-4, an untimed `live` merged with a floor, and the codec mapping -of each older wire. Also the mixed case #5000 left out: through a lite-05 -relay, a `live` subscriber already reading group 1, then one with a floor of -group 0, and a fresh group 0 reaches only the second. - -## Related - -- [One max_age meaning](/quest/m1/cache-max-age.md) - decides where an untimed `Live` starts -- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - names undelivered groups once publishers report them diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index 0bb1a38219f..d961b9908b5 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -72,7 +72,6 @@ is published; older versions are unchanged. ## Required -- [lite-07 Live flag](/quest/m1/lite-live.md) - SUBSCRIBE carries `Live` apart from its floor - [In-band auth](/quest/m1/auth/README.md) - lite-07 carries the Auth Stream and UNAUTHORIZED (0x3B) - [Routes and announces](/quest/m1/cluster-routing/routes.md) - lite-07 carries the route layer: ROUTE per origin node and path-less ANNOUNCE, with the hop list gone - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - lite-07 restores SUBSCRIBE_DROP in place of `Stream Count` diff --git a/quest/m1/pipeline-requests/subscribe.md b/quest/m1/pipeline-requests/subscribe.md index fadb81be329..a04abc7e045 100644 --- a/quest/m1/pipeline-requests/subscribe.md +++ b/quest/m1/pipeline-requests/subscribe.md @@ -94,6 +94,5 @@ blocks the Related quests below; whichever lands second rebases. ## Related - [Pipelined first FETCH](/quest/m1/pipeline-requests/fetch.md) - the same change for fetch-only readers -- [lite-07 Live flag](/quest/m1/lite-live.md) - reshapes the same SUBSCRIBE fields - [One max_age meaning](/quest/m1/cache-max-age.md) - the staleness rule that makes the wire clamp redundant - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - replaces the SUBSCRIBE floor with ranges diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 603e024a3ea..f93f0a76e85 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -16,7 +16,7 @@ Decided in planning (09-29), after the fetch-span agent (#4558) found that a relay bounds a sparse FETCH only if the model can request ranges: - **Wire (lite-07, unpublished).** SUBSCRIBE carries a list of group ranges - (an open end means live) in place of Group Start/End, plus + (an open end means live; ends may be relative, below) in place of Group Start/End, plus `order: asc | desc` (desc, newest first, is today's rule and the default). Subscriber Max Age (the model's `max_delay`) stays as a cap alongside the ranges, since a group's timestamp isn't always known. Decided 2026-10-10: @@ -43,10 +43,22 @@ relay bounds a sparse FETCH only if the model can request ranges: group's existing prefix but never wait for its future objects. Downstream, an IETF FETCH is served from the model's ranges, capped at the Largest Object. -- The `Live` flag from [lite-07 Live flag](/quest/m1/lite-live.md) - stays a separate field beside the range list (maintainer, 10-07): a range - starting above the live edge must not hide the latest group from a - subscriber merged with it. +- **Relative ends** (maintainer, 2026-10-11, superseding the 10-07 `Live` + flag and closing #5085). Each range end is `Absolute(group, frame)` or + `Relative(n)`, naming the group n before the latest; either end may be + relative, and an omitted end is live. An offset past the track's start + clamps to group 0. Before any group exists, relative ends resolve when the + first one arrives, so `Relative(0)..` starts at it. The cross-language and + bridge tests cover both edges. Ends are inclusive of whole groups: + a relative start resolves to frame 0 of that group, and a relative end to + just after its last frame (the model's exclusive `Position::after_group`), + so `Absolute(0)..Relative(0)` includes the latest group. Live is `Relative(0)..`, so no separate `Live` field exists. + Merging subscribers is the union of their ranges, so a range above the + live edge never hides the latest group from a merged live subscriber, + which is the starvation the flag existed to prevent. Subscriber Max Age + still caps every range, so `Absolute(0)..` means everything not stale. + A relay that cannot resolve a relative end locally forwards it upstream as + relative. - Ranges are frame-precise (`Position`), not whole groups. The IETF joining FETCH for a mid-group SUBSCRIBE's uncached prefix stays, because today's bridge relies on it (maintainer, 09-29). @@ -59,7 +71,6 @@ every sequence delivered or dropped. ## Required -- [lite-07 Live flag](/quest/m1/lite-live.md) - the `Live` field the range list sits beside - [Model ranges](/quest/m1/subscribe-ranges/model.md) - `Subscription` carries ranges and order, and `Dynamic` fills misses by range - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - publishers name every group they won't deliver - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft diff --git a/quest/m1/subscribe-ranges/lite.md b/quest/m1/subscribe-ranges/lite.md index 102dd8709f7..2906015a7a4 100644 --- a/quest/m1/subscribe-ranges/lite.md +++ b/quest/m1/subscribe-ranges/lite.md @@ -9,6 +9,30 @@ lite upstream by subscribing to the missing ranges. ## Plan +Each range end encodes `Absolute(group, frame)`, `Relative(n)`, or open, and +a SUBSCRIBE with no ranges is refused. Older wires map at the codec only, with +no change to a published version: lite-06 `Group Start` 0 with `Frame Start` +0, and pre-06 absent, are `Absolute(0)..` capped by max age (the oldest group +not yet stale); any other lite-06 pair is an absolute start. When merged +ranges can't be expressed on an older upstream wire, subscribe from group 0 +and filter locally; lite-03 to 05 must send an explicit group 0, since the +codec folds 0 to absent (`encode_start_group`), which they read as the latest +group. moq-transport's relative offsets differ by one: draft-20 +`Relative(0)` is the next group and `Relative(1)` the current one, so an +open-ended `Relative(n)..` maps to draft-20 `Relative(n + 1)`, saturating +(an offset past the track's start clamps to group 0 either way). Older +drafts only encode a relative 0, so there it is SUBSCRIBE plus a relative +joining FETCH n groups back, as the live join already does. That filter +has no end, so a range with a bounded end resolves both bounds against the +Largest Object and goes upstream as an absolute range. Live keeps today's +join, `NextObject` plus a `Relative(1)` fill on draft-20+ or a relative +joining FETCH on older drafts (`ietf::subscriber::subscribe_join`), so the +current group's existing prefix still arrives. A relative end resolves +against the latest group the answering hop knows. Behind an IETF upstream +that is the Largest Object's group, so a newest group with no objects yet is +not counted until its first object arrives; the relay reports what it has +rather than waiting for the peer. + Update `drafts/draft-lcurley-moq-lite.md` in the same PR: the SUBSCRIBE and SUBSCRIBE_UPDATE fields, the Group Order section (the order is now a field), the removal of FETCH, and the lite-07 changelog. Fix the stale "offset by 1" @@ -20,6 +44,5 @@ FETCH, answered through the model's ranges. Run `just drafts check` and ## Required -- [lite-07 Live flag](/quest/m1/lite-live.md) - reshapes the same SUBSCRIBE first - [Model ranges](/quest/m1/subscribe-ranges/model.md) - the model this serves - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - lite-07's DROP for holes diff --git a/quest/m1/subscribe-ranges/model.md b/quest/m1/subscribe-ranges/model.md index cecc7ccbf8b..69161df13e1 100644 --- a/quest/m1/subscribe-ranges/model.md +++ b/quest/m1/subscribe-ranges/model.md @@ -13,9 +13,23 @@ requests for misses instead of one request per group. See the [line](/quest/m1/subscribe-ranges/README.md) for the decisions. Guidance: -- Aggregating several subscribers' ranges and orders onto one upstream - request needs a rule. For example, the union of ranges, and desc wins when - orders conflict. Decide it and write it down. +- Ranges have `Absolute` or `Relative` ends (see the line). Aggregating + subscribers onto one upstream request is the union of their ranges; a + relative end resolves against the track's latest group when the publisher + serves it. Decide and document the order rule when orders conflict (for + example, desc wins). +- This absorbs the deleted lite-07 Live flag quest, so the model mirrors the + wire with no `live` field, and its tests move here, on mock time and both + in-process and through a relay: a subscriber from `Absolute(4)` merged with + a `Relative(0)..` subscriber on a quiet track whose newest group is 3 (the + live one still gets group 3); a buffering `Relative(k)..` merged with a + range above the live edge; a frame-precise start merged with live; an + untimed `Absolute(0)..` (it follows + [One max_age meaning](/quest/m1/cache-max-age.md)); and the lite-05 relay + case #5000 left out: a live subscriber already reading group 1, then one + from `Absolute(0)`, and a fresh group 0 reaches only the second. The closed + #5085 branch has reusable pieces (older-wire codec mappings, the JS fix + keeping groups below a floor buffered, mocked-time tests). - Replace `requested_group` and `fetch_group` rather than keep both. Update every caller (relay, moq-archive, hls, the ladder) in the same PR, plus the published surfaces built on them, per the cross-package table: moq-ffi @@ -32,10 +46,6 @@ Guidance: enforcement; reuse its contract rather than restoring a historical exemption. - Benchmark range count and span as separate axes (AGENTS.md fan-out rule). -## Required - -- [lite-07 Live flag](/quest/m1/lite-live.md) - reshapes the same `Subscription` first - ## Related - [FETCH max-delay](/quest/m1/fetch-max-delay.md) - historical request budgets and per-caller enforcement