From f70e7939b650dfceed494c2c635d36157dedf734 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 19:35:47 -0700 Subject: [PATCH 1/5] chore(quest): claim quest/m1/subscribe-ranges/bounded-tail.md Quest-Claim: 785fb253d4b19f2f From d221093adb4484e3e2f8b74d03f5c26cc558193a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 19:59:26 -0700 Subject: [PATCH 2/5] test(net): reproduce bounded group tail loss Preserve failing relay regressions and passing source-end controls while documenting the unresolved stream completion signal. Co-Authored-By: GPT-6 --- quest/m1/subscribe-ranges/bounded-tail.md | 83 ++++++---- rs/moq-net/tests/integration/bounded_tail.rs | 151 +++++++++++++++++++ rs/moq-net/tests/integration/main.rs | 1 + 3 files changed, 209 insertions(+), 26 deletions(-) create mode 100644 rs/moq-net/tests/integration/bounded_tail.rs diff --git a/quest/m1/subscribe-ranges/bounded-tail.md b/quest/m1/subscribe-ranges/bounded-tail.md index edd6a9065f1..939c8858f12 100644 --- a/quest/m1/subscribe-ranges/bounded-tail.md +++ b/quest/m1/subscribe-ranges/bounded-tail.md @@ -10,32 +10,63 @@ plain and compressed tracks on every version that carries frame bounds. ## Plan -Found on [moq-dev/moq#6242](https://github.com/moq-dev/moq/pull/6242) while -answering Codex's -[P1](https://github.com/moq-dev/moq/pull/6242#discussion_r4345457940): through -one relay, a subscriber reads group 5 frames 0..3 and a second then asks for -3..5. The tail never arrives, on a plain track and on a compressed one alike. - -- `GroupRecv` (`rs/moq-net/src/lite/subscriber.rs`) calls `group.finish()` - when a group stream ends cleanly, including one that ended only because the - request's end bound was reached. The truncated copy then looks complete. -- `claim_sequence` and `covering_group` (`rs/moq-net/src/model/track.rs`) - judge a copy only by its first frame, so every later stream for that - sequence is `Duplicate`, whatever frame it was asked from. -- Fix it in the receive path: a stream that stopped at the request's end - bound is not the group's end. Weigh leaving the copy unfinished so a tail - appends from its end (one copy; a compressed group keeps inflating against - its own prefix, which `widen_compressed` relies on when it keeps a request at - N) against letting a bounded copy stop covering a request past its end so - the new stream replaces it (compressed requests would then widen to frame 0 - again). -- A peer must not be able to pass off an early FIN as reaching the bound. - -Test on mock time through a relay: a bounded first subscriber, then a second -asking past its end, on plain and compressed lite-07 tracks, and on -moq-transport draft-20+ if its Absolute end reaches the same path. - -Public API: none expected. Wire: none. +Reproduced on `ee1111a91b` through one relay: subscriber A reads group 5 +frames 0..3, then subscriber B asks for 3..5. Local readers get all requested +frames, but a relayed lite-07 copy loses the tail on both plain and compressed +tracks. Draft-20, 21, and 22 lose it too. The deterministic reproductions and +early/exact-end controls are in `rs/moq-net/tests/integration/bounded_tail.rs`. +The tail tests assert the intended behavior and currently fail; they are not +ignored or changed to require the bug. + +`GroupRecv` (`rs/moq-net/src/lite/subscriber.rs`) always calls `group.finish()` +on FIN. `claim_sequence` and `covering_group` (`model/track.rs`) then treat +the copy as covering every later request from its first frame. IETF +`GroupIngest` similarly collapses a bare FIN and an explicit END_OF_GROUP into +`Ended::Group`, even though its publisher clears `GroupFlags.has_end` when +serving a bounded slice. JS has the same unconditional close in its lite +subscriber. + +Settle the stream-boundary distinction before changing ownership. Lite-07's +GROUP header carries only subscription ID, sequence, and starting frame. It +does not carry the request's end or revision. After an update widens 3 to 5, +the same header, frames 0..3, and FIN can be: + +- A stream admitted under the old end of 3, with more source frames to come. +- A stream admitted under the new end of 5, whose source really ends at 3. + +Latching the receiver's current end when the header arrives cannot distinguish +these cases: control and data streams can arrive out of order. Remembering +every historical end has unbounded cost without a retirement signal. Never +declare an early FIN to have reached a requested end. + +Recommendation: make each stream's admitted end unambiguous, then append its +tail into the existing copy. A stream-specific end bound in the unpublished +lite-07 GROUP header would let FIN below that bound finish the source group, +and FIN exactly at it leave the copy resumable. An explicit final-versus- +bounded terminal is an alternative. This is a wire decision, not an approved +format change. Reuse IETF's existing end bit/status rather than extending its +wire format, including emitting a genuine early end when the header did not +promise one. Keep published drafts compatible and mirror lite changes in JS +and its draft. + +For append ownership, transfer one private writer permission into cache-owned +group state between streams. Never park a producer/parent clone or permit two +writers. Resume atomically at the committed frame boundary, preserve the +compressed prefix and inflater, and leave cached payload subject to normal +eviction, abort, and route teardown. A replayed prefix must be decoded and +discarded without writing it twice. Replacement is less attractive: warm +compressed demand intentionally starts at N, so replacing a copy also needs +its prefix and correct cache coverage, fetch, and existing-reader recovery. + +Verify plain/compressed tails, real ends before/at/after the cap, empty tails, +duplicate concurrent streams, reset/eviction/teardown, and repeated demand +widening. Add a scripted case where the update overtakes an earlier GROUP +header, and the converse where a real early FIN follows the update. Neither +test may depend on sleeps or accepting a missing tail. Exercise the matching +JS path and cross-language interop after the boundary contract is settled. + +Public API: none expected. Wire: lite-07 boundary signaling needs a decision; +no format change is implemented by this checkpoint. ## Related diff --git a/rs/moq-net/tests/integration/bounded_tail.rs b/rs/moq-net/tests/integration/bounded_tail.rs new file mode 100644 index 00000000000..1180856cb21 --- /dev/null +++ b/rs/moq-net/tests/integration/bounded_tail.rs @@ -0,0 +1,151 @@ +#![cfg(feature = "async")] + +//! A request bound ends delivery, without declaring the source group complete. + +use crate::support::harness::{MockConnectOptions, connect_mock, spawn}; +use moq_net::{Hop, Timescale, Timestamp, Version, origin, track}; +use std::time::Duration; + +fn origin(hop: u64) -> origin::Producer { + let (origin, driver) = origin::Producer::new(origin::Config::new(Hop::new(hop).unwrap())); + spawn(driver); + origin +} + +fn range(start: u64, end: u64) -> track::Subscription { + track::Subscription::default().with_ranges([track::Range { + start: track::Anchor::Absolute(track::Position { group: 5, frame: start }), + end: Some(track::Anchor::Absolute(track::Position { group: 5, frame: end })), + }]) +} + +async fn bounded_tail(version: Option, compressed: bool, frames: u64) { + let source = origin(1); + let relay = origin(2); + let reader = origin(3); + let broadcast = source.create_broadcast("bcast").unwrap(); + let producer = broadcast + .create_track( + "data", + track::Info::default() + .with_timescale(Timescale::MILLI) + .with_compress(compressed), + ) + .unwrap(); + broadcast.announce(Default::default()).unwrap(); + let mut links = Vec::new(); + let remote = if let Some(version) = version { + for (upstream, downstream) in [(&source, &relay), (&relay, &reader)] { + let mut options = MockConnectOptions::new(version); + options.server_publish = Some(upstream.consume()); + options.client_subscribe = Some(downstream.clone()); + links.push(connect_mock(options).await); + } + reader.consume().routed("bcast").await.unwrap(); + reader.consume().request_broadcast("bcast", None).await.unwrap() + } else { + broadcast.consume() + }; + let track = remote.track("data").unwrap(); + let mut first = track.subscribe(range(0, 3)).await.unwrap(); + crate::support::harness::subscribed(&producer).await; + let mut group = producer.create_group(moq_net::group::Info { sequence: 5 }).unwrap(); + for index in 0..frames { + group + .write_frame(Timestamp::from_millis(index).unwrap(), vec![index as u8; 100]) + .unwrap(); + } + group.finish().unwrap(); + let mut prefix = first.recv_group().await.unwrap().unwrap(); + for index in 0..3.min(frames) { + assert_eq!( + prefix.read_frame().await.unwrap().unwrap().payload, + vec![index as u8; 100] + ); + } + assert!(prefix.read_frame().await.unwrap().is_none()); + let mut second = track.subscribe(range(3, 5)).await.unwrap(); + let mut tail = second.recv_group().await.unwrap().unwrap(); + // IETF passthrough readers expose frame ordinals independently of Object IDs; + // narrow this cursor too so that a stale prefix cannot hide the missing tail. + tail.set_frames(3..5); + for index in 3..5.min(frames) { + let frame = tail.read_frame().await.unwrap().expect("the requested tail frame"); + assert_eq!(frame.payload, vec![index as u8; 100]); + } + assert!(tail.read_frame().await.unwrap().is_none()); + drop((tail, prefix, second, first, links)); +} + +#[moq_clock::test] +async fn a_bounded_plain_copy_accepts_its_tail() { + moq_clock::timeout( + Duration::from_secs(1), + bounded_tail(Some("moq-lite-07-wip".parse().unwrap()), false, 5), + ) + .await + .expect("a later subscriber receives the plain tail"); +} + +#[moq_clock::test] +async fn a_bounded_compressed_copy_accepts_its_tail() { + moq_clock::timeout( + Duration::from_secs(1), + bounded_tail(Some("moq-lite-07-wip".parse().unwrap()), true, 5), + ) + .await + .expect("a later subscriber receives and inflates the compressed tail"); +} + +#[moq_clock::test] +async fn a_bounded_draft20_copy_accepts_its_tail() { + moq_clock::timeout( + Duration::from_secs(1), + bounded_tail(Some("moq-transport-20".parse().unwrap()), false, 5), + ) + .await + .expect("a later subscriber receives the transport tail"); +} + +#[moq_clock::test] +async fn a_bounded_draft21_copy_accepts_its_tail() { + moq_clock::timeout( + Duration::from_secs(1), + bounded_tail(Some("moq-transport-21".parse().unwrap()), false, 5), + ) + .await + .expect("a later subscriber receives the transport tail"); +} + +#[moq_clock::test] +async fn a_bounded_draft22_copy_accepts_its_tail() { + moq_clock::timeout( + Duration::from_secs(1), + bounded_tail(Some("moq-transport-22".parse().unwrap()), false, 5), + ) + .await + .expect("a later subscriber receives the transport tail"); +} + +#[moq_clock::test] +async fn local_bounded_readers_keep_the_same_content() { + for compressed in [false, true] { + moq_clock::timeout(Duration::from_secs(1), bounded_tail(None, compressed, 5)) + .await + .expect("local readers receive the tail"); + } +} + +#[moq_clock::test] +async fn a_group_ending_before_or_at_the_request_bound_ends_cleanly() { + for compressed in [false, true] { + for frames in [2, 3] { + moq_clock::timeout( + Duration::from_secs(1), + bounded_tail(Some("moq-lite-07-wip".parse().unwrap()), compressed, frames), + ) + .await + .expect("a real end remains readable after demand widens"); + } + } +} diff --git a/rs/moq-net/tests/integration/main.rs b/rs/moq-net/tests/integration/main.rs index 136e961cf44..013125405da 100644 --- a/rs/moq-net/tests/integration/main.rs +++ b/rs/moq-net/tests/integration/main.rs @@ -9,6 +9,7 @@ mod announce_covering_route; mod announce_to_serve; mod auth; +mod bounded_tail; mod broadcast_close; mod catalog_acquisition; mod catalog_churn; From 546931f78dd93f666a2a92edcea22ec3223d768f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 20:13:06 -0700 Subject: [PATCH 3/5] test(net): cover published lite06 bounded tails Include lite06 in the unresolved completion-signal compatibility scope. Co-Authored-By: GPT-6 --- quest/m1/subscribe-ranges/bounded-tail.md | 20 ++++++++++++-------- rs/moq-net/tests/integration/bounded_tail.rs | 10 ++++++++++ 2 files changed, 22 insertions(+), 8 deletions(-) diff --git a/quest/m1/subscribe-ranges/bounded-tail.md b/quest/m1/subscribe-ranges/bounded-tail.md index 939c8858f12..7b84561b75f 100644 --- a/quest/m1/subscribe-ranges/bounded-tail.md +++ b/quest/m1/subscribe-ranges/bounded-tail.md @@ -13,8 +13,9 @@ plain and compressed tracks on every version that carries frame bounds. Reproduced on `ee1111a91b` through one relay: subscriber A reads group 5 frames 0..3, then subscriber B asks for 3..5. Local readers get all requested frames, but a relayed lite-07 copy loses the tail on both plain and compressed -tracks. Draft-20, 21, and 22 lose it too. The deterministic reproductions and -early/exact-end controls are in `rs/moq-net/tests/integration/bounded_tail.rs`. +tracks. Lite-06 plain and draft-20, 21, and 22 lose it too. The deterministic +reproductions and early/exact-end controls are in +`rs/moq-net/tests/integration/bounded_tail.rs`. The tail tests assert the intended behavior and currently fail; they are not ignored or changed to require the bug. @@ -26,7 +27,7 @@ the copy as covering every later request from its first frame. IETF serving a bounded slice. JS has the same unconditional close in its lite subscriber. -Settle the stream-boundary distinction before changing ownership. Lite-07's +Settle the stream-boundary distinction before changing ownership. Lite-06/07's GROUP header carries only subscription ID, sequence, and starting frame. It does not carry the request's end or revision. After an update widens 3 to 5, the same header, frames 0..3, and FIN can be: @@ -44,10 +45,13 @@ tail into the existing copy. A stream-specific end bound in the unpublished lite-07 GROUP header would let FIN below that bound finish the source group, and FIN exactly at it leave the copy resumable. An explicit final-versus- bounded terminal is an alternative. This is a wire decision, not an approved -format change. Reuse IETF's existing end bit/status rather than extending its -wire format, including emitting a genuine early end when the header did not -promise one. Keep published drafts compatible and mirror lite changes in JS -and its draft. +format change. Lite-06 also carries frame bounds and must remain compatible: +its solution needs negotiated signaling or request-identity correlation, not +an unconditional new header field. Changing only lite-07 would leave the Goal +unmet. Lite-05 and earlier round bounds outward to whole groups. Reuse IETF's +existing end bit/status rather than extending its wire format, including +emitting a genuine early end when the header did not promise one. Mirror lite +changes in JS and its draft. For append ownership, transfer one private writer permission into cache-owned group state between streams. Never park a producer/parent clone or permit two @@ -65,7 +69,7 @@ header, and the converse where a real early FIN follows the update. Neither test may depend on sleeps or accepting a missing tail. Exercise the matching JS path and cross-language interop after the boundary contract is settled. -Public API: none expected. Wire: lite-07 boundary signaling needs a decision; +Public API: none expected. Wire: lite boundary signaling needs a decision; no format change is implemented by this checkpoint. ## Related diff --git a/rs/moq-net/tests/integration/bounded_tail.rs b/rs/moq-net/tests/integration/bounded_tail.rs index 1180856cb21..3a5bbb952f7 100644 --- a/rs/moq-net/tests/integration/bounded_tail.rs +++ b/rs/moq-net/tests/integration/bounded_tail.rs @@ -77,6 +77,16 @@ async fn bounded_tail(version: Option, compressed: bool, frames: u64) { drop((tail, prefix, second, first, links)); } +#[moq_clock::test] +async fn a_bounded_lite06_copy_accepts_its_tail() { + moq_clock::timeout( + Duration::from_secs(1), + bounded_tail(Some("moq-lite-06".parse().unwrap()), false, 5), + ) + .await + .expect("a later subscriber receives the lite-06 tail"); +} + #[moq_clock::test] async fn a_bounded_plain_copy_accepts_its_tail() { moq_clock::timeout( From 4f47d1415c890469af04cac962f9ecf0f224c9f4 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 23 Oct 2026 11:20:25 -0700 Subject: [PATCH 4/5] quest: record bounded-tail wire decisions and proposed split Co-Authored-By: Claude Opus 5.5 --- quest/m1/subscribe-ranges/bounded-tail.md | 51 ++++++++++++++++------- 1 file changed, 36 insertions(+), 15 deletions(-) diff --git a/quest/m1/subscribe-ranges/bounded-tail.md b/quest/m1/subscribe-ranges/bounded-tail.md index 7b84561b75f..705a79e49c2 100644 --- a/quest/m1/subscribe-ranges/bounded-tail.md +++ b/quest/m1/subscribe-ranges/bounded-tail.md @@ -1,4 +1,4 @@ -# [M] A copy that stopped at a request's end bound accepts its tail +# [XL] A copy that stopped at a request's end bound accepts its tail ## Goal @@ -40,18 +40,34 @@ these cases: control and data streams can arrive out of order. Remembering every historical end has unbounded cost without a retirement signal. Never declare an early FIN to have reached a requested end. -Recommendation: make each stream's admitted end unambiguous, then append its -tail into the existing copy. A stream-specific end bound in the unpublished -lite-07 GROUP header would let FIN below that bound finish the source group, -and FIN exactly at it leave the copy resumable. An explicit final-versus- -bounded terminal is an alternative. This is a wire decision, not an approved -format change. Lite-06 also carries frame bounds and must remain compatible: -its solution needs negotiated signaling or request-identity correlation, not -an unconditional new header field. Changing only lite-07 would leave the Goal -unmet. Lite-05 and earlier round bounds outward to whole groups. Reuse IETF's -existing end bit/status rather than extending its wire format, including -emitting a genuine early end when the header did not promise one. Mirror lite -changes in JS and its draft. +Decided by the maintainer (2026-10-23), superseding the earlier park: + +- Lite-07-wip carries an explicit stream-level indication of bounded versus + final, updated in `drafts/draft-lcurley-moq-lite.md`. It is unpublished, so + its framing changes freely. Proposed shape: a GROUP `Frame End` (exclusive, + 0 when the stream runs to the group's end). A FIN below it ends the group; + a FIN at it ends only the stream, and the publisher serves the tail on a + later stream for the same sequence once the end widens. FRAME has no type + byte, so a trailing terminal message would need new framing. +- Lite-06 has no such field: it falls back to request correlation, a fresh + request identity for each end revision, so a stream's bound is its + request's. +- IETF reuses `has_end` / END_OF_GROUP, with no new wire, including emitting + a genuine early end that the header did not promise. +- `js/net` mirrors the lite-07 change. +- Too big for one PR: split rather than land part of it. Lite-05 and earlier + round bounds outward to whole groups and need nothing. + +Proposed split, awaiting the maintainer, in order: + +1. Lite-07 end to end: the draft, the GROUP codec in Rust and JS, Rust and JS + publishers labelling capped streams and serving the tail after the end + widens, and the Rust subscriber appending into the cached copy (append + ownership below). JS subscribers never ask for frame bounds, so they only + decode the field. +2. Lite-06 request correlation on top of 1's append path. +3. IETF `has_end` / END_OF_GROUP on top of 1's append path, with the + Object-ID/ordinal handling. For append ownership, transfer one private writer permission into cache-owned group state between streams. Never park a producer/parent clone or permit two @@ -69,8 +85,13 @@ header, and the converse where a real early FIN follows the update. Neither test may depend on sleeps or accepting a missing tail. Exercise the matching JS path and cross-language interop after the boundary contract is settled. -Public API: none expected. Wire: lite boundary signaling needs a decision; -no format change is implemented by this checkpoint. +The publisher half is missing too: the lite publisher marks a capped group +served and its cursor moves past it, so a widening SUBSCRIBE_UPDATE never +reaches the tail even on one hop, and the draft forbids a second Group Stream +for a sequence on one subscription. + +Public API: none expected. Wire: lite-07 GROUP gains the bounded/final +indication above; no format change is implemented by this checkpoint. ## Related From a526fe14eb138ee5dffd05cf60e1db7564471d64 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 23 Oct 2026 13:18:38 -0700 Subject: [PATCH 5/5] feat(net): lite-07 GROUP Frame End continues bounded groups A lite-07 Group Stream capped at a subscription's end frame now says so with `Frame End`. A FIN there suspends the receiver's cached copy instead of finishing it, and the publisher serves the rest on a later Group Stream once SUBSCRIBE_UPDATE moves the end past it, so a relay's copy cut for one request accepts the tail another request asks for. Lite-06 and moq-transport continuations move to their own quests. Co-Authored-By: Claude Opus 5.5 --- drafts/draft-lcurley-moq-lite.md | 14 +- js/net/src/lite/group.ts | 22 +- js/net/src/lite/message.test.ts | 36 ++ js/net/src/lite/publisher.test.ts | 35 +- js/net/src/lite/publisher.ts | 100 ++++- js/net/src/lite/subscriber.ts | 2 + js/net/src/lite/version.ts | 17 + quest/m1/compress/ietf-range-widen.md | 2 +- quest/m1/subscribe-ranges/README.md | 3 +- .../m1/subscribe-ranges/bounded-tail-ietf.md | 42 ++ .../subscribe-ranges/bounded-tail-lite06.md | 39 ++ quest/m1/subscribe-ranges/bounded-tail.md | 98 ----- rs/moq-net/src/fuzz.rs | 2 + rs/moq-net/src/lite/group.rs | 54 +++ rs/moq-net/src/lite/publisher.rs | 108 ++++- rs/moq-net/src/lite/subscriber.rs | 370 +++++++++++++++++- rs/moq-net/src/lite/version.rs | 9 + rs/moq-net/src/model/group.rs | 93 ++++- rs/moq-net/src/model/track.rs | 40 ++ rs/moq-net/src/recv.rs | 5 + rs/moq-net/src/test_interop.rs | 1 + rs/moq-net/tests/integration/bounded_tail.rs | 163 ++++---- 22 files changed, 1036 insertions(+), 219 deletions(-) create mode 100644 quest/m1/subscribe-ranges/bounded-tail-ietf.md create mode 100644 quest/m1/subscribe-ranges/bounded-tail-lite06.md delete mode 100644 quest/m1/subscribe-ranges/bounded-tail.md diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 6bf209eb2a6..0fbd59ef6e4 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -785,7 +785,7 @@ Receiving padding does not change session or subscription state. This allows bandwidth probes to measure delivery without requiring a media subscription. Versions before lite-07 do not define this stream type and reset only the stream, as required for unknown stream types. -### Group +### Group {#group-stream} A publisher creates Group Streams in response to a Subscribe Stream. A Group Stream MUST start with a GROUP message and MAY be followed by any number of FRAME messages. @@ -795,7 +795,8 @@ A frame MAY contain an empty payload, potentially indicating a gap in the group. The GROUP message's `Frame Start` gives the index of the first FRAME on the stream, so a stream carrying only part of a Group is self-describing. It is redundant with the subscription's own `Frame Start` in the steady state, and carried anyway because a SUBSCRIBE_UPDATE that moves the bound races the Group Streams already in flight: the two travel on different streams with no ordering between them, so the receiver would otherwise have to guess which bound a given Group Stream was opened under. -A publisher MUST NOT send two Group Streams for the same Group Sequence on one subscription. +A publisher MUST NOT send two Group Streams for the same Group Sequence on one subscription, except to continue one that stopped at its `Frame End`. +When a SUBSCRIBE_UPDATE moves the end past such a Group, the publisher sends the rest on a new Group Stream whose `Frame Start` is the earlier stream's `Frame End`. A subscriber assembling a Group from more than one publisher does so across separate subscriptions, and is responsible for concatenating the runs in index order and for ignoring any frame it has already received. Both the publisher and subscriber MAY reset the stream at any time. @@ -1293,6 +1294,7 @@ SUBSCRIBE_UPDATE Message { See [SUBSCRIBE](#subscribe) for information about each field. Moving `Frame Start` forward within a group that is already being delivered does not retract frames the publisher has already sent; like `Group Start`, it only bounds what the publisher sends from here on. +Moving the end past a Group whose stream stopped at its `Frame End` sends the rest of that Group (see [Group](#group-stream)). **Wanted Group**: @@ -1732,6 +1734,7 @@ GROUP Message { Subscribe ID (i) Group Sequence (i) Frame Start (i) + Frame End (i) } ~~~ @@ -1750,6 +1753,12 @@ A plain absolute index; 0 means the stream carries the group from its beginning, A non-zero value means the leading frames are not on this stream, either because the subscription started partway into this group or because the publisher only holds part of it. The subscriber MUST NOT assume it will receive the missing frames on another stream; they are a gap unless a separate subscription or FETCH covers them. +**Frame End**: +The subscription's end frame this stream stops at, encoded as the index + 1 like SUBSCRIBE's `Frame End`, or 0 when the stream runs to the end of the Group. +A FIN before `Frame End`, or any FIN when it is 0, ends the Group. +A FIN at `Frame End` ends only the stream: the Group may have more frames, which a later Group Stream delivers if the subscription's end moves past them (see [Group](#group-stream)). +A non-zero `Frame End` at or below `Frame Start` is malformed, as is a FRAME at or past it. + ## FRAME The FRAME message is a payload within a group. @@ -1789,6 +1798,7 @@ The `Message Length` describes the payload size on the wire. ## moq-lite-07 - Allow a negotiated extension to preserve timestamps with explicitly unchecked provenance. +- Added `Frame End` to GROUP: a FIN at it ends only the stream, and a later Group Stream continues the Group once the subscription's end moves past it. - Added Padding Stream (0x2), read and discarded incrementally by either endpoint. diff --git a/js/net/src/lite/group.ts b/js/net/src/lite/group.ts index 9cf14169299..20c6f2c0ebc 100644 --- a/js/net/src/lite/group.ts +++ b/js/net/src/lite/group.ts @@ -5,7 +5,7 @@ import { hooks } from "../internal.ts"; import type { Cursor, Reader, Writer } from "../stream.ts"; import * as Time from "../time.ts"; import * as Message from "./message.ts"; -import { hasFrameBounds, type Version } from "./version.ts"; +import { hasFrameBounds, hasFrameEnd, type Version } from "./version.ts"; export class Group { subscribe: bigint; @@ -21,18 +21,28 @@ export class Group { */ frameStart: number; + /** + * The exclusive frame this stream stops at because the subscription ends there, or + * undefined when it runs to the end of the group. A FIN at it ends only the stream, so + * a later stream may continue the group. Lite-07+. + */ + frameEnd?: number; + constructor({ subscribe, sequence, frameStart = 0, + frameEnd, }: { subscribe: bigint; sequence: number; frameStart?: number; + frameEnd?: number; }) { this.subscribe = subscribe; this.sequence = sequence; this.frameStart = frameStart; + this.frameEnd = frameEnd; } async #encode(w: Writer, version: Version) { @@ -44,13 +54,21 @@ export class Group { // The peer would number the frames from 0 and silently misalign the group. throw new Error("frame offsets not supported for this version"); } + if (hasFrameEnd(version)) { + await w.u53(this.frameEnd ?? 0); + } else if (this.frameEnd !== undefined) { + // The peer would read the FIN as the group's end. + throw new Error("frame end not supported for this version"); + } } static async #decode(r: Reader, version: Version): Promise { const subscribe = await r.u62(); const sequence = await r.u53(); const frameStart = hasFrameBounds(version) ? await r.u53() : 0; - return new Group({ subscribe, sequence, frameStart }); + const frameEnd = hasFrameEnd(version) ? await r.u53() : 0; + if (frameEnd !== 0 && frameEnd <= frameStart) throw new Error("frame end at or below frame start"); + return new Group({ subscribe, sequence, frameStart, frameEnd: frameEnd === 0 ? undefined : frameEnd }); } async encode(w: Writer, version: Version): Promise { diff --git a/js/net/src/lite/message.test.ts b/js/net/src/lite/message.test.ts index 3e385e347f4..4c0541983c5 100644 --- a/js/net/src/lite/message.test.ts +++ b/js/net/src/lite/message.test.ts @@ -1,6 +1,7 @@ import { expect, test } from "bun:test"; import { Reader, Writer } from "../stream.ts"; import { AnnounceInit } from "./announce.ts"; +import { Group } from "./group.ts"; import { Subscribe } from "./subscribe.ts"; import { Version } from "./version.ts"; @@ -34,3 +35,38 @@ test("control message length 65,535 passes the prefix check", async () => { const wire = await prefix(65_535, version); await expect(Subscribe.decode(new Reader(undefined, wire, version), version)).rejects.not.toThrow("too large"); }); + +async function encodeGroup(group: Group, version: Version): Promise { + const written: Uint8Array[] = []; + const writer = new Writer( + new WritableStream({ write: (chunk) => void written.push(new Uint8Array(chunk)) }), + version, + ); + await group.encode(writer, version); + writer.close(); + await writer.closed; + const wire = new Uint8Array(written.reduce((size, chunk) => size + chunk.byteLength, 0)); + let offset = 0; + for (const chunk of written) { + wire.set(chunk, offset); + offset += chunk.byteLength; + } + return wire; +} + +// Lite-07 carries where a capped stream stops; lite-06 cannot, so it refuses one rather than +// let the FIN read as the group's end. An end at or below the start is malformed. +test("GROUP Frame End is lite-07 only", async () => { + const capped = new Group({ subscribe: 1n, sequence: 7, frameStart: 4, frameEnd: 6 }); + const wire = await encodeGroup(capped, Version.DRAFT_07); + const decoded = await Group.decode(new Reader(undefined, wire, Version.DRAFT_07), Version.DRAFT_07); + expect([decoded.frameStart, decoded.frameEnd]).toEqual([4, 6]); + + await expect(encodeGroup(capped, Version.DRAFT_06)).rejects.toThrow("frame end"); + + const inverted = await encodeGroup(new Group({ subscribe: 1n, sequence: 7, frameStart: 4 }), Version.DRAFT_07); + inverted[inverted.length - 1] = 4; + await expect(Group.decode(new Reader(undefined, inverted, Version.DRAFT_07), Version.DRAFT_07)).rejects.toThrow( + "frame end", + ); +}); diff --git a/js/net/src/lite/publisher.test.ts b/js/net/src/lite/publisher.test.ts index 0af30562f78..233abf21922 100644 --- a/js/net/src/lite/publisher.test.ts +++ b/js/net/src/lite/publisher.test.ts @@ -651,7 +651,8 @@ async function servedSubscription( ) { const version = options.version ?? Version.DRAFT_05; const frames = options.frames ?? ["hello"]; - const pair = createMockTransportPair(version === Version.DRAFT_06 ? ALPN_06 : ALPN_05); + const alpn = { [Version.DRAFT_06]: ALPN_06, [Version.DRAFT_07]: ALPN_07_WIP }[version as number] ?? ALPN_05; + const pair = createMockTransportPair(alpn); const origin = new OriginProducer(); const publisher = new Publisher(pair.server, version, randomHop(), origin.consume()); @@ -755,7 +756,7 @@ async function servedSubscription( await reader.u62(); payloads.push(new TextDecoder().decode(await reader.read(await reader.u53()))); } - return { sequence: header.sequence, frameStart: header.frameStart, payloads }; + return { sequence: header.sequence, frameStart: header.frameStart, frameEnd: header.frameEnd, payloads }; }, // The sequence alone, for tests that only care which groups reached the wire. async servedSequence(): Promise { @@ -1331,6 +1332,34 @@ test("lite draft-06: a queued frame floor applies before the next group pop", as } }); +// A lite-07 stream stopped at the end frame says where, and each update moving the end past +// it continues the group on a new stream from there. +test("lite draft-07: a capped group continues once the end moves past it", async () => { + const sub = await servedSubscription({ + version: Version.DRAFT_07, + startGroup: 0, + endGroup: 0, + endFrame: 1, + frames: ["a", "b", "c"], + }); + try { + sub.serve(0); + expect(await sub.servedGroup()).toEqual({ sequence: 0, frameStart: 0, frameEnd: 2, payloads: ["a", "b"] }); + + await replayUpdate({ priority: 0, startGroup: 0, endGroup: 0, endFrame: 2 }).encode( + sub.client.writer, + Version.DRAFT_07, + ); + expect(await sub.servedGroup()).toEqual({ sequence: 0, frameStart: 2, frameEnd: 3, payloads: ["c"] }); + + // The group ends where the last stream stopped, which only an empty continuation says. + await replayUpdate({ priority: 0, startGroup: 0 }).encode(sub.client.writer, Version.DRAFT_07); + expect(await sub.servedGroup()).toEqual({ sequence: 0, frameStart: 3, payloads: [] }); + } finally { + await sub.close(); + } +}); + // The update and group are both ready when the write unblocks. Control is drained first, then // the group is popped and positioned synchronously under the new frame cap. test("lite draft-06: a queued frame update applies before the next group pop", async () => { @@ -1838,7 +1867,7 @@ test("blocked DROP responses stop opening later groups", async () => { }); /** One group stream the publisher put on the wire. */ -type Served = { sequence: number; frameStart: number; payloads: string[] }; +type Served = { sequence: number; frameStart: number; frameEnd?: number; payloads: string[] }; /** * Serves `groups` (frame payloads per group, keyed by sequence) under `bounds`, and diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index c655075ad1c..1aadac5570e 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -48,6 +48,7 @@ import { hasAnnounceRestart, hasCompression, hasDatagrams, + hasFrameEnd, hasLargest, hasProbeRtt, hasRouteCost, @@ -150,6 +151,12 @@ interface RunGroup { /** Last frame to send (inclusive), or undefined for the rest of the group. */ end?: number; + + /** Where the stream says it stops, so its FIN there ends only the stream (lite-07). */ + frameEnd?: number; + + /** The group's reader already sits at {@link start}, continuing an earlier stream. */ + continued?: boolean; } /** @@ -1191,6 +1198,62 @@ export class Publisher { const groups = new Set>(); const drops = new Signal(0); const pendingDrops: SubscribeDrop[] = []; + // The group whose stream stops at the end frame on lite-07, kept open so a later end + // can continue it: `stopped` settles with whether the stream stopped there. + let capped: { group: group.Consumer; end: number; stopped: Promise } | undefined; + + // Open a stream for a group and serve it. The FIN waits for every stream to finish or + // reset, including one that has not opened yet. + const serve = (options: RunGroup, after = Promise.resolve(true)) => { + const { group } = options; + if (options.end !== undefined && hasFrameEnd(this.version)) options.frameEnd = options.end + 1; + const stopped = Promise.withResolvers(); + if (options.frameEnd !== undefined) { + // Only the end group is capped, so a cap moved to another group frees the old one. + const old = capped; + if (old && old.group !== group) void old.stopped.then(() => old.group.close()); + capped = { group, end: options.frameEnd, stopped: stopped.promise }; + } + const task = after.then(async (ready) => { + if (!ready) return stopped.resolve(false); + const writer = await this.#openGroup(options); + const failed = writer instanceof Error ? writer : await this.#serveGroup(writer, options); + stopped.resolve(failed === "capped"); + if (emitRange && failed instanceof Error) { + pendingDrops.push(SubscribeDrop.given(group.sequence, group.sequence, failed, this.version)); + drops.update((revision) => revision + 1); + } + }); + groups.add(task); + void task.finally(() => groups.delete(task)); + }; + + // Serve the rest of the capped group once the end moved past where its stream + // stopped, after that stream let go of the group's reader. + const continueCapped = () => { + const held = capped; + if (!held) return; + const sequence = held.group.sequence; + if (bounds.startGroup !== undefined && bounds.startGroup > sequence) return; + if (bounds.endGroup !== undefined && bounds.endGroup < sequence) return; + const range = frameRange(bounds, sequence); + if (range.end !== undefined && range.end + 1 <= held.end) return; + capped = undefined; + serve( + { + sub, + group: held.group, + stamp, + compress, + priority, + unsubscribed, + start: Math.max(range.start, held.end), + end: range.end, + continued: true, + }, + held.stopped, + ); + }; // Cancels groups still queued for a stream slot. Only the subscriber leaving counts: // a track that ran out of groups still has to flush the ones already queued, and the @@ -1259,6 +1322,7 @@ export class Publisher { bounds.startFrame = update.startFrame; bounds.endGroup = update.endGroup; bounds.endFrame = update.endFrame; + continueCapped(); // A microtask is too short: decoding one framed update crosses several // awaits, each of which can requeue behind the serving continuation. A // task boundary lets the decoder finish whatever the transport already @@ -1404,7 +1468,7 @@ export class Publisher { } if (served) insertRun(served, group.sequence, group.sequence + 1); - const options: RunGroup = { + serve({ sub, group, stamp, @@ -1413,22 +1477,13 @@ export class Publisher { unsubscribed, start: range.start, end: range.end, - }; - // The FIN waits for every stream to finish or reset, including one that has not opened yet. - const opened = this.#openGroup(options); - const task = opened.then(async (writer) => { - const failed = writer instanceof Error ? writer : await this.#serveGroup(writer, options); - if (emitRange && failed instanceof Error) { - pendingDrops.push(SubscribeDrop.given(group.sequence, group.sequence, failed, this.version)); - drops.update((revision) => revision + 1); - } }); - groups.add(task); - void task.finally(() => groups.delete(task)); } } finally { if (!finished) unsubscribe(); priority.close(); + const left = capped; + if (left) void left.stopped.then(() => left.group.close()); } } @@ -1762,11 +1817,14 @@ export class Publisher { * * @internal */ - async #serveGroup(stream: Writer, options: RunGroup) { - const { sub, group, stamp, compress, priority, start: startFrame, end: endFrame } = options; + async #serveGroup(stream: Writer, options: RunGroup): Promise { + const { sub, group, stamp, compress, priority, start: startFrame, end: endFrame, frameEnd } = options; // This model holds whole groups, so frame `startFrame` is always reachable unless // the group ends first. Declaring it up front keeps the stream self-describing. - const msg = new GroupMessage({ subscribe: sub, sequence: group.sequence, frameStart: startFrame }); + const msg = new GroupMessage({ subscribe: sub, sequence: group.sequence, frameStart: startFrame, frameEnd }); + // Whether the stream stopped at its Frame End, leaving the group's reader open for the + // stream continuing it. + let capped = false; // Everything past this point runs inside the cleanup scope, so a failure never leaves // a finished group's stream being ranked. try { @@ -1782,7 +1840,7 @@ export class Publisher { let prevTs = 0n; // Whether the cursor ever reached the requested start, which decides how the // end of the group is read below. - let reached = startFrame === 0; + let reached = startFrame === 0 || options.continued === true; for (;;) { const read = await race([hooks.readGroupFrame(group, undefined, compress), stream.closed]); @@ -1801,7 +1859,6 @@ export class Publisher { // Frames below the requested start were excluded, and the receiver // numbers what it gets from `startFrame`. if (read.sequence < startFrame) continue; - if (endFrame !== undefined && read.sequence > endFrame) break; if (stamp !== undefined) { // Convert each frame to the track's wire timescale. @@ -1818,15 +1875,20 @@ export class Publisher { } finally { read.complete(); } + // Stop at the end frame rather than wait on the frame after it. + if (endFrame !== undefined && read.sequence >= endFrame) { + capped = frameEnd !== undefined; + break; + } } stream.close(); - group.close(); + if (!capped) group.close(); // A stream that fails after its FIN may have lost its header, so report it for a DROP. // Its bytes still hold the connection until acknowledged, so a group gone stale // meanwhile is reset like one still serving, as the Rust `GroupServe` does. await hooks.guardGroup(group, () => stream.closed); - return undefined; + return capped ? "capped" : undefined; } catch (err: unknown) { const e = error(err); stream.reset(e); diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 8639b8aa0db..9cd39cd0ff2 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -1621,6 +1621,8 @@ export class Subscriber { scale = timescale.peek(); } + // This subscriber never asks for an end frame, so no stream may stop at one. + if (group.frameEnd !== undefined) throw new ProtocolViolation("unrequested group frame end"); const compress = entry.compress; if (compress && group.frameStart !== 0) throw new StreamError(StreamCode.TooFarBehind, { message: "compressed group lacks its prefix" }); diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index b500d818d8b..e39142ec51f 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -257,6 +257,23 @@ export function hasFrameBounds(version: Version): boolean { } } +/** Whether GROUP carries `Frame End`, so a stream stopped at a subscription's end frame + * does not end its group. Added in lite-07. */ +export function hasFrameEnd(version: Version): boolean { + // Explicitly list older versions so future versions keep the lite-07+ behavior. + switch (version) { + case Version.DRAFT_01: + case Version.DRAFT_02: + case Version.DRAFT_03: + case Version.DRAFT_04: + case Version.DRAFT_05: + case Version.DRAFT_06: + return false; + default: + return true; + } +} + /** Whether SUBSCRIBE and SUBSCRIBE_UPDATE carry the subscriber's `Group Order`. Added in * lite-07; an earlier subscription is served newest first. */ export function hasOrder(version: Version): boolean { diff --git a/quest/m1/compress/ietf-range-widen.md b/quest/m1/compress/ietf-range-widen.md index 62b818c0fc0..d28aa0b614c 100644 --- a/quest/m1/compress/ietf-range-widen.md +++ b/quest/m1/compress/ietf-range-widen.md @@ -55,4 +55,4 @@ Public API: none. Wire: none. ## Related - [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - carries a live range change upstream on REQUEST_UPDATE -- [Bounded tails](/quest/m1/subscribe-ranges/bounded-tail.md) - a copy cut at a request's end must accept its tail before a warm prefix can be extended +- [IETF bounded tails](/quest/m1/subscribe-ranges/bounded-tail-ietf.md) - a moq-transport copy cut at a request's end must accept its tail before a warm prefix can be extended diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 77d4b797449..9a61962ea38 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -100,7 +100,8 @@ every sequence delivered or dropped. - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft - [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - range FETCH upstream per missing run, and non-blocking FETCH served downstream - [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity -- [Bounded tails](/quest/m1/subscribe-ranges/bounded-tail.md) - a copy cut at a request's mid-group end accepts the rest of its group instead of rejecting it as `Duplicate` +- [Lite-06 bounded tails](/quest/m1/subscribe-ranges/bounded-tail-lite06.md) - a lite-06 copy cut at a request's mid-group end accepts the rest of its group, correlating each end with its request +- [IETF bounded tails](/quest/m1/subscribe-ranges/bounded-tail-ietf.md) - a moq-transport copy cut at a request's mid-group end accepts the rest of its group, using END_OF_GROUP ## Related diff --git a/quest/m1/subscribe-ranges/bounded-tail-ietf.md b/quest/m1/subscribe-ranges/bounded-tail-ietf.md new file mode 100644 index 00000000000..bfb784794c6 --- /dev/null +++ b/quest/m1/subscribe-ranges/bounded-tail-ietf.md @@ -0,0 +1,42 @@ +# [M] A moq-transport copy that stopped at a request's end bound accepts its tail + +## Goal + +Over moq-transport drafts 20, 21, and 22, a group received for a request whose +end falls mid-group stays open to the rest of that group. When later demand +asks the same sequence past that end, a second subscriber on the same relay +reads the frames it asked for. + +## Plan + +Lite-07 landed the receive side in +[moq-dev/moq#6447](https://github.com/moq-dev/moq/pull/6447): a bounded FIN +suspends the cached copy (`group::Producer::suspend`), and the stream +continuing it takes the copy over (`track::Producer::poll_resume_group`), +waiting for the stream it overtook and dropping a replayed prefix. Reuse that +path; only the boundary signal differs. + +Decided by the maintainer (2026-10-23): reuse `GroupFlags.has_end` and +END_OF_GROUP, with no new wire. The IETF publisher already clears `has_end` +when serving a bounded slice, so a bare FIN on such a stream suspends and an +END_OF_GROUP finishes. A receiver-only change is not enough: a local source's +genuine early end emits no explicit END_OF_GROUP today, so the publisher must +send one whenever the header did not promise the end. + +`GroupIngest` collapses a bare FIN and END_OF_GROUP into `Ended::Group`; split +them. IETF readers get relayed groups whole because Object IDs may not be frame +ordinals (`track::Subscriber::deliver` skips the frame window when +`group.relayed()`), so continuing a copy needs the Object-ID to ordinal +mapping to agree across the two streams, including sparse IDs. + +Reproduce first with the `bounded_tail` harness in +`rs/moq-net/tests/integration/bounded_tail.rs` on `moq-transport-20`, `21`, and +`22`; the original failing cases are in commit d221093adb, whose readers also +narrow their frame cursor so a stale prefix cannot hide the missing tail. + +Public API: none. Wire: none. + +## Related + +- [Lite-06 bounded tails](/quest/m1/subscribe-ranges/bounded-tail-lite06.md) - the same continuation over lite-06 request correlation +- [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - carries a live range change upstream on REQUEST_UPDATE diff --git a/quest/m1/subscribe-ranges/bounded-tail-lite06.md b/quest/m1/subscribe-ranges/bounded-tail-lite06.md new file mode 100644 index 00000000000..60da1a10d2b --- /dev/null +++ b/quest/m1/subscribe-ranges/bounded-tail-lite06.md @@ -0,0 +1,39 @@ +# [M] A lite-06 copy that stopped at a request's end bound accepts its tail + +## Goal + +Over lite-06, a group received for a request whose end falls mid-group stays +open to the rest of that group. When later demand asks the same sequence past +that end, a second subscriber on the same relay reads the frames it asked for. +This holds for plain tracks; lite-06 has no compression. + +## Plan + +Lite-07 landed the receive side in +[moq-dev/moq#6447](https://github.com/moq-dev/moq/pull/6447): GROUP carries +`Frame End`, a FIN there suspends the cached copy +(`group::Producer::suspend`), and the stream continuing it takes the copy over +(`track::Producer::poll_resume_group`), waiting for the stream it overtook and +dropping a replayed prefix. Reuse that path; only the boundary signal differs. + +Decided by the maintainer (2026-10-23): lite-06 GROUP has no such field and +stays unchanged, so a stream's bound comes from request correlation. Each end +revision inside a group gets its own request identity: the relay moves an end +frame with a fresh SUBSCRIBE instead of a SUBSCRIBE_UPDATE, so the request a +stream belongs to fixes its end. A FIN at that end suspends; below it ends the +group. The fresh request starts at the copy's resume frame, so a lite-06 +publisher serves the tail with its existing `Frame Start` handling. Bound the +churn: only a move of the end frame within a group needs a new request, and +the replaced one is cancelled once its streams settle. + +Reproduce first with the `bounded_tail` harness in +`rs/moq-net/tests/integration/bounded_tail.rs` on `moq-lite-06`; the original +failing case is in commit d221093adb. Cover an early real end, an end exactly +at the cap, repeated widening, and the replaced request's streams arriving +after the new one's. + +Public API: none. Wire: none. + +## Related + +- [IETF bounded tails](/quest/m1/subscribe-ranges/bounded-tail-ietf.md) - the same continuation over moq-transport's existing end signal diff --git a/quest/m1/subscribe-ranges/bounded-tail.md b/quest/m1/subscribe-ranges/bounded-tail.md deleted file mode 100644 index 705a79e49c2..00000000000 --- a/quest/m1/subscribe-ranges/bounded-tail.md +++ /dev/null @@ -1,98 +0,0 @@ -# [XL] A copy that stopped at a request's end bound accepts its tail - -## Goal - -A group received for a request whose end falls mid-group stays open to the -rest of that group. When later demand asks the same sequence past that end, -the tail is accepted instead of rejected as `Duplicate`, so a second -subscriber on the same relay reads the frames it asked for. This holds for -plain and compressed tracks on every version that carries frame bounds. - -## Plan - -Reproduced on `ee1111a91b` through one relay: subscriber A reads group 5 -frames 0..3, then subscriber B asks for 3..5. Local readers get all requested -frames, but a relayed lite-07 copy loses the tail on both plain and compressed -tracks. Lite-06 plain and draft-20, 21, and 22 lose it too. The deterministic -reproductions and early/exact-end controls are in -`rs/moq-net/tests/integration/bounded_tail.rs`. -The tail tests assert the intended behavior and currently fail; they are not -ignored or changed to require the bug. - -`GroupRecv` (`rs/moq-net/src/lite/subscriber.rs`) always calls `group.finish()` -on FIN. `claim_sequence` and `covering_group` (`model/track.rs`) then treat -the copy as covering every later request from its first frame. IETF -`GroupIngest` similarly collapses a bare FIN and an explicit END_OF_GROUP into -`Ended::Group`, even though its publisher clears `GroupFlags.has_end` when -serving a bounded slice. JS has the same unconditional close in its lite -subscriber. - -Settle the stream-boundary distinction before changing ownership. Lite-06/07's -GROUP header carries only subscription ID, sequence, and starting frame. It -does not carry the request's end or revision. After an update widens 3 to 5, -the same header, frames 0..3, and FIN can be: - -- A stream admitted under the old end of 3, with more source frames to come. -- A stream admitted under the new end of 5, whose source really ends at 3. - -Latching the receiver's current end when the header arrives cannot distinguish -these cases: control and data streams can arrive out of order. Remembering -every historical end has unbounded cost without a retirement signal. Never -declare an early FIN to have reached a requested end. - -Decided by the maintainer (2026-10-23), superseding the earlier park: - -- Lite-07-wip carries an explicit stream-level indication of bounded versus - final, updated in `drafts/draft-lcurley-moq-lite.md`. It is unpublished, so - its framing changes freely. Proposed shape: a GROUP `Frame End` (exclusive, - 0 when the stream runs to the group's end). A FIN below it ends the group; - a FIN at it ends only the stream, and the publisher serves the tail on a - later stream for the same sequence once the end widens. FRAME has no type - byte, so a trailing terminal message would need new framing. -- Lite-06 has no such field: it falls back to request correlation, a fresh - request identity for each end revision, so a stream's bound is its - request's. -- IETF reuses `has_end` / END_OF_GROUP, with no new wire, including emitting - a genuine early end that the header did not promise. -- `js/net` mirrors the lite-07 change. -- Too big for one PR: split rather than land part of it. Lite-05 and earlier - round bounds outward to whole groups and need nothing. - -Proposed split, awaiting the maintainer, in order: - -1. Lite-07 end to end: the draft, the GROUP codec in Rust and JS, Rust and JS - publishers labelling capped streams and serving the tail after the end - widens, and the Rust subscriber appending into the cached copy (append - ownership below). JS subscribers never ask for frame bounds, so they only - decode the field. -2. Lite-06 request correlation on top of 1's append path. -3. IETF `has_end` / END_OF_GROUP on top of 1's append path, with the - Object-ID/ordinal handling. - -For append ownership, transfer one private writer permission into cache-owned -group state between streams. Never park a producer/parent clone or permit two -writers. Resume atomically at the committed frame boundary, preserve the -compressed prefix and inflater, and leave cached payload subject to normal -eviction, abort, and route teardown. A replayed prefix must be decoded and -discarded without writing it twice. Replacement is less attractive: warm -compressed demand intentionally starts at N, so replacing a copy also needs -its prefix and correct cache coverage, fetch, and existing-reader recovery. - -Verify plain/compressed tails, real ends before/at/after the cap, empty tails, -duplicate concurrent streams, reset/eviction/teardown, and repeated demand -widening. Add a scripted case where the update overtakes an earlier GROUP -header, and the converse where a real early FIN follows the update. Neither -test may depend on sleeps or accepting a missing tail. Exercise the matching -JS path and cross-language interop after the boundary contract is settled. - -The publisher half is missing too: the lite publisher marks a capped group -served and its cursor moves past it, so a widening SUBSCRIBE_UPDATE never -reaches the tail even on one hop, and the draft forbids a second Group Stream -for a sequence on one subscription. - -Public API: none expected. Wire: lite-07 GROUP gains the bounded/final -indication above; no format change is implemented by this checkpoint. - -## Related - -- [Compressed tracks](/quest/m1/compress/README.md) - its Ranges rule keeps a warm request at N, which this makes reachable diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index 8ca7c1a2d70..6d90a3652c5 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -482,6 +482,7 @@ impl LiteSample { subscribe: 3, sequence: 1_234, frame_start: 0, + frame_end: None, } .encode(w, version) .unwrap(), @@ -717,6 +718,7 @@ impl Default for Messages { subscribe: 7, sequence: 123_456, frame_start: 0, + frame_end: None, }, ietf_subscribe: ietf::Subscribe { max_age: None, diff --git a/rs/moq-net/src/lite/group.rs b/rs/moq-net/src/lite/group.rs index 3e18a108a07..9757b180775 100644 --- a/rs/moq-net/src/lite/group.rs +++ b/rs/moq-net/src/lite/group.rs @@ -15,6 +15,11 @@ pub struct Group { // missing, either because the subscription started partway in or because the // publisher only holds the tail. Lite06+ only; older versions always start at 0. pub frame_start: u64, + + // The exclusive frame this stream stops at because the subscription ends there, or + // `None` when it runs to the end of the group. A FIN at it ends only the stream, so a + // later stream may continue the group. Lite07+ only. + pub frame_end: Option, } impl Message for Group { @@ -25,11 +30,20 @@ impl Message for Group { true => r.varint()?, false => 0, }; + let frame_end = match version.has_frame_end() { + true => match r.varint()? { + 0 => None, + end if end <= frame_start => return Err(DecodeError::InvalidValue), + end => Some(end), + }, + false => None, + }; Ok(Self { subscribe, sequence, frame_start, + frame_end, }) } @@ -44,6 +58,13 @@ impl Message for Group { return Err(EncodeError::Version); } + match self.frame_end { + _ if version.has_frame_end() => w.varint(self.frame_end.unwrap_or(0))?, + None => {} + // The peer would read the FIN as the group's end. + Some(_) => return Err(EncodeError::Version), + } + Ok(()) } } @@ -59,6 +80,7 @@ mod test { subscribe: 1, sequence: 7, frame_start: 4, + frame_end: None, }; let mut buf = Vec::new(); msg.encode_msg(&mut Encoder::new(&mut buf, Version::Lite06.into()), Version::Lite06) @@ -74,6 +96,7 @@ mod test { subscribe: 1, sequence: 7, frame_start: 4, + frame_end: None, }; let mut buf = Vec::new(); assert!( @@ -89,4 +112,35 @@ mod test { let got = crate::coding::decode_buf(&mut buf.as_slice(), Version::Lite05, Group::decode_msg).unwrap(); assert_eq!(got.frame_start, 0); } + + /// Lite-07 carries where a capped stream stops; lite-06 cannot, so it refuses one + /// rather than let the FIN read as the group's end. An end at or below the start is + /// malformed. + #[test] + fn frame_end_is_lite07_only() { + let msg = Group { + subscribe: 1, + sequence: 7, + frame_start: 4, + frame_end: Some(6), + }; + let mut buf = Vec::new(); + msg.encode_msg(&mut Encoder::new(&mut buf, Version::Lite07.into()), Version::Lite07) + .unwrap(); + let got = crate::coding::decode_buf(&mut buf.as_slice(), Version::Lite07, Group::decode_msg).unwrap(); + assert_eq!((got.frame_start, got.frame_end), (4, Some(6))); + + let mut buf = Vec::new(); + assert!( + msg.encode_msg(&mut Encoder::new(&mut buf, Version::Lite06.into()), Version::Lite06) + .is_err() + ); + + let mut buf = Vec::new(); + let w = &mut Encoder::new(&mut buf, Version::Lite07.into()); + for value in [1, 7, 4, 4] { + w.varint(value).unwrap(); + } + assert!(crate::coding::decode_buf(&mut buf.as_slice(), Version::Lite07, Group::decode_msg).is_err()); + } } diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 561a8f475e7..f8caac5a6e7 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -8,7 +8,7 @@ use std::{ ops::{Bound, ControlFlow, Range}, sync::{ Arc, Mutex, - atomic::{AtomicUsize, Ordering}, + atomic::{AtomicBool, AtomicUsize, Ordering}, }, task::{Context, Poll, ready}, time::Duration, @@ -3765,6 +3765,7 @@ impl Subscription { sequence, frame_start, false, + None, Rank::Queue(priority), group, )); @@ -3813,6 +3814,39 @@ struct TrackRun { opening: Option>, /// The groups with a stream, serving or awaiting the acknowledgement of their FIN. children: kio::Tasks>, + /// The group whose last stream stops at the end frame, so a later end can continue it + /// (lite-07). + capped: Option, + /// The rest of `capped`, waiting to open once the end moved past it and its stream let go. + tail: Option<(group::Consumer, Released)>, +} + +/// A group whose stream stops at the subscription's end frame. +struct Capped { + group: group::Consumer, + /// The exclusive frame the stream stops at. + end: u64, + released: Released, +} + +/// Set once a capped stream wrote its FIN or failed, so the stream continuing its group opens +/// only after it: the subscriber appends a continuation once the stream before it let go. +#[derive(Clone, Default)] +struct Released(Arc); + +impl Released { + fn is_set(&self) -> bool { + self.0.load(Ordering::Acquire) + } +} + +/// Sets its [`Released`] when dropped. +struct Release(Released); + +impl Drop for Release { + fn drop(&mut self) { + self.0.0.store(true, Ordering::Release); + } } impl TrackRun { @@ -3841,6 +3875,8 @@ impl TrackRun { datagrams, opening: None, children: kio::Tasks::new(), + capped: None, + tail: None, } } @@ -3924,6 +3960,38 @@ impl TrackRun { .end_at(upd.end_group.map_or(Bound::Unbounded, Bound::Included)); self.start_frame = bounds.start_frame(); self.end_frame = bounds.end_frame(); + self.continue_capped(&upd); + } + + /// Queue the rest of the group a stream stopped at the end frame, once the end moved + /// past where it stopped. The continuation is capped again by the new end. + fn continue_capped(&mut self, upd: &lite::SubscribeUpdate) { + let Some(capped) = self.capped.take() else { + return; + }; + let sequence = capped.group.sequence; + let stopped = capped.end; + // A floor raised past the group no longer wants it. + if upd.start_group.is_some_and(|start| start > sequence) { + return; + } + let end = match self.end_frame { + Some((end, last)) if end == sequence => Some(last.saturating_add(1)), + _ => None, + }; + let within = upd.end_group.is_none_or(|end| end >= sequence); + if !within || end.is_some_and(|end| end <= stopped) { + self.capped = Some(capped); + return; + } + let mut tail = capped.group; + let from = match self.start_frame { + Some((start, frame)) if start == sequence => frame.max(stopped), + _ => stopped, + }; + tail.start_at(from); + tail.end_at(end.map_or(Bound::Unbounded, Bound::Excluded)); + self.tail = Some((tail, capped.released)); } /// Serve until the live edge reaches the track's boundary. @@ -4018,6 +4086,16 @@ impl TrackRun { return Poll::Ready(Ok(ControlFlow::Continue(()))); } + // A continuation goes out before the cursor's next group, once the stream before it + // let go. That stream is one of the children polled above, so its release wakes this. + if self.opening.is_none() + && self.tail.as_ref().is_some_and(|(_, released)| released.is_set()) + && let Some((tail, _)) = self.tail.take() + { + self.serve(tail); + return Poll::Ready(Ok(ControlFlow::Continue(()))); + } + // One cursor drives the whole subscription: poll the cap-aware arrival-order // group and, when enabled, the next best-effort datagram. Groups are polled // first so a datagram burst can't starve them; datagrams flow whenever no @@ -4248,6 +4326,22 @@ impl TrackRun { let frame_start = group.index(); // The stream stops at the end frame, before the group may end. let capped = self.end_frame.is_some_and(|(end, _)| end == sequence); + // Lite-07 says where, so its FIN there ends only the stream and a later end can + // continue the group. A group already known to end by then needs neither. + let frame_end = self + .end_frame + .filter(|(end, _)| *end == sequence && self.ctx.version.has_frame_end()) + .map(|(_, last)| last.saturating_add(1)) + .filter(|end| group.final_count().is_none_or(|fin| fin > *end)); + let frame_end = frame_end.map(|end| { + let released = Released::default(); + self.capped = Some(Capped { + group: group.clone(), + end, + released: released.clone(), + }); + (end, Release(released)) + }); self.mark_served(sequence); tracing::debug!(subscribe = self.ctx.id, track = %self.ctx.track_name, sequence, "serving group"); @@ -4272,6 +4366,7 @@ impl TrackRun { sequence, frame_start, capped, + frame_end, rank, group, )); @@ -4296,6 +4391,9 @@ struct GroupServe { frame_start: u64, /// The stream stops at a subscription's end frame, so its FIN cannot end the group. capped: bool, + /// Where `capped` stops, on a version whose GROUP says so, and what lets the stream + /// continuing the group open once this one wrote its FIN or failed. + frame_end: Option<(u64, Release)>, // Lite05+ delta-encodes per-frame timestamps within the group. The first // frame's delta is absolute (against an implicit prev value of 0), every // subsequent delta is signed against the previous frame. @@ -4340,6 +4438,7 @@ impl GroupServe { sequence: u64, frame_start: u64, capped: bool, + frame_end: Option<(u64, Release)>, rank: Rank>, group: group::Consumer, ) -> Self { @@ -4355,6 +4454,7 @@ impl GroupServe { sequence, frame_start, capped, + frame_end, prev_ts: 0, state: GroupState::Open, budget: kio::coop::Budget::new(32), @@ -4407,6 +4507,7 @@ impl GroupServe { subscribe: self.ctx.id, sequence: self.sequence, frame_start: self.frame_start, + frame_end: self.frame_end.as_ref().map(|(end, _)| *end), }; let header = writer.buffer(&lite::DataType::Group).and_then(|()| writer.buffer(&msg)); let records = header.and_then(|()| { @@ -4648,7 +4749,10 @@ impl GroupServe { // The buffer drained before the final frame resolved, so the // FIN follows the last byte. match writer.finish() { - Ok(()) => self.state = GroupState::Closed { writer }, + Ok(()) => { + self.frame_end = None; + self.state = GroupState::Closed { writer }; + } // The writer drops here: the Drop reset stands in for the // abort, exactly like the old `finish()?`. Err(err) => return Poll::Ready(Err(err)), diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 19bb2ab16cc..b20d8ee8424 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -98,6 +98,33 @@ struct TrackEntry { framing: lite::Framing, /// The groups received so far, so the subscription's end can wait for the ones owed. tail: kio::Producer, + /// Groups whose stream stopped at the subscription's end frame. + suspended: Suspended, +} + +/// Groups whose stream stopped at the subscription's end frame, aborted once the +/// subscription and its streams are gone unless a later stream continued them, so no +/// reader waits on a continuation that cannot come. +#[derive(Clone, Default)] +struct Suspended(Arc); + +#[derive(Default)] +struct SuspendedGroups(Lock>); + +impl Suspended { + fn push(&self, group: group::Handle) { + let mut groups = self.0.0.lock(); + groups.retain(|held| held.is_suspended() && !held.is_clone(&group)); + groups.push(group); + } +} + +impl Drop for SuspendedGroups { + fn drop(&mut self) { + for group in self.0.lock().drain(..) { + group.abort_suspended(Error::Cancel); + } + } } impl Subscriber { @@ -923,12 +950,18 @@ impl Drop for SetupWait { enum GroupRecvState { /// Reading the GROUP header. Header, + /// Waiting for the stream before it to stop writing the group this one may continue. + Claim { + hdr: lite::Group, + ietf: bool, + }, /// Filling the group, bailing if the track or group dies first. Serve { /// Guarded: dropping this machine mid-group is a cancellation, not a clean end. group: crate::recv::Group, track: track::Producer, ingest: FrameIngest, + suspended: Suspended, _reading: Reading, }, Done, @@ -957,16 +990,29 @@ impl GroupRecv { self.setup_wait = None; let mut cx = waiter.context(); let hdr = ready!(self.reader.poll_decode::(&mut cx))?; - - let (group, track, framing, reading) = { + self.state = GroupRecvState::Claim { hdr, ietf }; + } + GroupRecvState::Claim { hdr, ietf } => { + let (hdr, ietf) = (hdr.clone(), *ietf); + let (group, track, framing, reading, suspended, resumed) = { let mut subs = self.subscriber.subscribes.lock(); let entry = subs.get_mut(&hdr.subscribe).ok_or(Error::Cancel)?; + // A stream continuing a group whose earlier stream stopped at the + // subscription's end takes over that copy, once the earlier stream let + // go of it: the two travel on unordered streams. + let resumed = match self.subscriber.version.has_frame_end() { + true => ready!(entry.producer.poll_resume_group(hdr.sequence, hdr.frame_start, waiter)), + false => None, + }; let group_info = group::Info { sequence: hdr.sequence }; // Stats (groups/frames/bytes) are counted in the model as the group // is written, through the tagged `track::Producer`. The track counts // it as content once its first frame lands. - let received = entry.producer.create_group(group_info); + let (received, resumed) = match resumed { + Some(group) => (Ok(group), true), + None => (entry.producer.create_group(group_info), false), + }; // The subscription's end waits until this stream is read. Counted // whatever the cache said, since the publisher counts every stream it // opens (a resubscribe resends a cached group), and only after the @@ -1009,10 +1055,31 @@ impl GroupRecv { }; // The stream may carry only the tail of the group; number the frames // from where the publisher said they start so a front continuing the - // group across routes lines them up. - group.start_at(hdr.frame_start)?; - (group, entry.producer.clone(), entry.framing, reading) + // group across routes lines them up. A resumed copy already does. + if !resumed { + group.start_at(hdr.frame_start)?; + } + ( + group, + entry.producer.clone(), + entry.framing, + reading, + entry.suspended.clone(), + resumed, + ) }; + // Frames the stream replays below where the copy stopped are read and + // dropped. Object records would be applied twice, so that extension + // continues only exactly where the copy stopped. + let replayed = group.frame_count() as u64 - hdr.frame_start; + // A stream capped at or below what the copy holds adds nothing. + if replayed > 0 && (ietf || hdr.frame_end.is_some_and(|end| end <= group.frame_count() as u64)) { + let handle = group.handle(); + if group.suspend().is_ok() { + suspended.push(handle); + } + return Poll::Ready(Err(Error::Duplicate)); + } let (priority, order) = track .subscription() .map_or((0, Default::default()), |sub| (sub.priority, sub.order)); @@ -1022,10 +1089,15 @@ impl GroupRecv { // The timescale came from TRACK_INFO (read before this subscription was // even registered), so frames decode immediately. No SUBSCRIBE_OK to // wait on. + let mut ingest = FrameIngest::new(framing, self.subscriber.version, ietf, hdr.frame_start); + ingest.replay = replayed; + ingest.end = hdr.frame_end; + ingest.resumed = resumed.then_some(group.frame_count() as u64); self.state = GroupRecvState::Serve { group: crate::recv::Group::new(group), track, - ingest: FrameIngest::new(framing, self.subscriber.version, ietf, hdr.frame_start), + ingest, + suspended, _reading: reading, }; } @@ -1051,14 +1123,36 @@ impl GroupRecv { // short drops into an already-aborted group instead of reporting a loss. let GroupRecvState::Serve { group, - ingest: _ingest, + ingest, + suspended, _reading, .. } = std::mem::replace(&mut self.state, GroupRecvState::Done) else { unreachable!() }; + // A continuation that added nothing hands the copy back rather than end it + // with this stream's failure, since a later stream can still continue it. + let unchanged = ingest.resumed == Some(group.handle().frame_count() as u64); let res = match res { + // A FIN at the stream's end frame ends only the stream: the group stays + // open for the stream that continues it. + Ok(()) if ingest.end == Some(group.frame_count() as u64) => { + suspend(group, &suspended); + Ok(()) + } + // The stream ended inside the frames the copy already held. + Ok(()) if ingest.replay > 0 => { + suspend(group, &suspended); + Err(Error::Decode(crate::coding::DecodeError::InvalidValue)) + } + Err(err) if unchanged && group.has_writer() => { + suspend(group, &suspended); + match err { + Error::Cancel | Error::Stream(crate::StreamError::Cancel) => Ok(()), + err => Err(err), + } + } Ok(()) => { let _ = group.finish(); Ok(()) @@ -1081,6 +1175,14 @@ impl GroupRecv { } } +/// Leave `group` open for a stream continuing it, aborted with the subscription otherwise. +fn suspend(group: crate::recv::Group, suspended: &Suspended) { + let handle = group.handle().clone(); + if group.into_inner().suspend().is_ok() { + suspended.push(handle); + } +} + /// Pumps bare FRAME messages from a reader into a group producer: the wire /// format shared by GROUP streams and FETCH responses. struct FrameIngest { @@ -1099,6 +1201,13 @@ struct FrameIngest { /// Whether the session's timestamps are media time. Lite-05/06 ones may be send times, /// so they are read to stay in step and dropped. media_time: bool, + /// Leading frames the group already holds, read and dropped rather than written twice. + replay: u64, + /// The frame the stream stops at because its subscription ends there; see + /// [`lite::Group::frame_end`]. + end: Option, + /// Where a copy this stream continues had stopped. + resumed: Option, } /// A group's object records on ingest, under the `ietf` extension. @@ -1140,6 +1249,8 @@ enum IngestPhase { }, /// Streaming the frame payload. Payload { frame: frame::Producer }, + /// Dropping a replayed frame's payload. + Replay { remaining: usize }, } impl FrameIngest { @@ -1155,6 +1266,9 @@ impl FrameIngest { frame_start, records: None, media_time: version.has_media_time(), + replay: 0, + end: None, + resumed: None, } } @@ -1304,6 +1418,16 @@ impl FrameIngest { let Some(size) = ready!(self.poll_header_varint(reader, &mut cx))? else { return Poll::Ready(Ok(())); }; + if self.end.is_some_and(|end| group.frame_count() as u64 >= end) { + return Poll::Ready(Err(Error::Decode(crate::coding::DecodeError::InvalidValue))); + } + if self.replay > 0 { + self.replay -= 1; + let remaining = + usize::try_from(size).map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded))?; + self.phase = IngestPhase::Replay { remaining }; + continue; + } // A frame that says nothing a plain frame would not keeps no fields. let fields = match &mut self.records { Some(records) => { @@ -1331,6 +1455,10 @@ impl FrameIngest { })?; self.phase = IngestPhase::Payload { frame }; } + IngestPhase::Replay { remaining } => { + ready!(reader.poll_skip(&mut cx, remaining))?; + self.phase = self.next_frame(); + } IngestPhase::Payload { frame } => { let failed = ready!(reader.poll_read_frame(&mut cx, frame)).err(); @@ -2755,6 +2883,7 @@ mod tests { subscribe: 7, sequence: 3, frame_start: 0, + frame_end: None, } .encode(&mut crate::coding::Encoder::new(&mut bytes, version.into()), version) .unwrap(); @@ -2815,6 +2944,7 @@ mod tests { compressed: false, }, tail: Default::default(), + suspended: Default::default(), }, ); let mut accept = UniAccept::new(subscriber, early); @@ -2842,6 +2972,7 @@ mod tests { subscribe: 7, sequence: 3, frame_start: 0, + frame_end: None, } .encode(&mut crate::coding::Encoder::new(&mut script, version.into()), version) .unwrap(); @@ -2875,6 +3006,7 @@ mod tests { compressed: false, }, tail: Default::default(), + suspended: Default::default(), }, ); @@ -2922,6 +3054,7 @@ mod tests { subscribe: 7, sequence: 0, frame_start: 0, + frame_end: None, } .encode(&mut encoder, version) .unwrap(); @@ -2968,6 +3101,7 @@ mod tests { producer: track.clone(), framing, tail: Default::default(), + suspended: Default::default(), }, ); @@ -3108,6 +3242,7 @@ mod tests { subscribe: 7, sequence: 5, frame_start: 0, + frame_end: None, } .encode(&mut encoder, version) .unwrap(); @@ -3152,6 +3287,7 @@ mod tests { compressed: false, }, tail: Default::default(), + suspended: Default::default(), }, ); @@ -3209,6 +3345,220 @@ mod tests { assert!(matches!(res, Err(Error::BoundsExceeded(_))), "{res:?}"); } + /// GROUP streams for group 5 of subscription 7 on a timed Lite07 track, read by hand. + struct Bounded { + subscriber: Subscriber, + session: crate::lite::test_transport::ScriptedSession, + track: track::Producer, + } + + impl Bounded { + /// One stream per script, in order. + fn new(scripts: Vec>) -> Self { + let session = crate::lite::test_transport::ScriptedSession::per_stream_eof(scripts); + let subscriber = Subscriber::new(SubscriberConfig { + runtime: crate::time::sim(), + session: session.clone(), + origin: origin::Config::new(crate::Hop::new(1).unwrap()).produce(), + recv_bandwidth: None, + version: Version::Lite07, + peer_setup: crate::lite::PeerSetup::new(false, crate::transport::Binding::InBand), + peer_hop: None, + cost: None, + going_away: Default::default(), + auth: crate::auth::Handle::new(false), + }); + let track = track::Producer::new( + std::sync::Arc::new(crate::broadcast::Info::default()), + "video", + crate::track::Info::timed(), + ); + subscriber.subscribes.lock().insert( + 7, + TrackEntry { + producer: track.clone(), + framing: lite::Framing { + timescale: Some(Timescale::MILLI), + ietf: None, + compressed: false, + }, + tail: kio::Producer::new(Tail::default()), + suspended: Default::default(), + }, + ); + Self { + subscriber, + session, + track, + } + } + + /// A receiver for the next scripted stream. + async fn stream(&mut self) -> GroupRecv { + use crate::transport::poll::Session as _; + let (_, recv) = self.session.open_bi(Default::default()).await.unwrap(); + GroupRecv::new(self.subscriber.clone(), Reader::new(recv, Version::Lite07)) + } + + /// Receive the next scripted stream to its end. + async fn recv(&mut self) -> Result<(), Error> { + let mut stream = self.stream().await; + kio::wait(|waiter| stream.poll_serve(waiter)).await + } + + /// The payloads group 5 holds now, and how it ended. + fn read(&self) -> (Vec, Ended) { + let mut group = self.track.consume().peek_group(5).expect("group 5 is cached"); + let mut payloads = Vec::new(); + loop { + match group.read_frame().now_or_never() { + Some(Ok(Some(frame))) => payloads.push(frame.payload[0]), + Some(Ok(None)) => return (payloads, Ended::Finished), + Some(Err(err)) => return (payloads, Ended::Aborted(err.to_string())), + None => return (payloads, Ended::Open), + } + } + } + } + + /// How a [`Bounded`] group reads past its frames. + #[derive(Debug, PartialEq)] + enum Ended { + /// Waiting for a stream to continue it. + Open, + Finished, + Aborted(String), + } + + /// A GROUP stream for group 5 of subscription 7 carrying `frames`, each a one-byte + /// payload of its own index. + fn bounded_stream(frame_start: u64, frame_end: Option, frames: std::ops::Range) -> Vec { + let version = Version::Lite07; + let mut script = Vec::new(); + let mut encoder = crate::coding::Encoder::new(&mut script, version.into()); + lite::Group { + subscribe: 7, + sequence: 5, + frame_start, + frame_end, + } + .encode(&mut encoder, version) + .unwrap(); + for frame in frames { + encoder.varint(0).unwrap(); + encoder.bytes(&[frame]).unwrap(); + } + script + } + + /// A FIN at the stream's Frame End leaves the group open, and the stream continuing it + /// there completes it. A second continuation finds the group finished. + #[moq_clock::test] + async fn a_fin_at_the_frame_end_leaves_the_group_open() { + let mut bounded = Bounded::new(vec![ + bounded_stream(0, Some(3), 0..3), + bounded_stream(3, None, 3..5), + bounded_stream(3, None, 3..5), + ]); + bounded.recv().await.unwrap(); + assert_eq!(bounded.read(), (vec![0, 1, 2], Ended::Open)); + + bounded.recv().await.unwrap(); + assert_eq!(bounded.read(), (vec![0, 1, 2, 3, 4], Ended::Finished)); + + let res = bounded.recv().await; + assert!(matches!(res, Err(Error::Duplicate)), "{res:?}"); + } + + /// A FIN before the stream's Frame End is the group's real end. + #[moq_clock::test] + async fn a_fin_before_the_frame_end_ends_the_group() { + let mut bounded = Bounded::new(vec![bounded_stream(0, Some(5), 0..3)]); + bounded.recv().await.unwrap(); + assert_eq!(bounded.read(), (vec![0, 1, 2], Ended::Finished)); + } + + /// A continuation can overtake the stream it continues: it waits until that one lets + /// go of the group, then appends. + #[moq_clock::test] + async fn a_continuation_waits_for_the_stream_it_overtook() { + let mut bounded = Bounded::new(vec![bounded_stream(3, None, 3..5)]); + let mut writing = bounded.track.create_group(group::Info { sequence: 5 }).unwrap(); + for frame in 0..3u8 { + writing.write_frame(Timestamp::ZERO, vec![frame]).unwrap(); + } + + let mut tail = bounded.stream().await; + assert!(tail.poll_serve(&kio::Waiter::noop()).is_pending()); + writing.suspend().unwrap(); + kio::wait(|waiter| tail.poll_serve(waiter)).await.unwrap(); + assert_eq!(bounded.read(), (vec![0, 1, 2, 3, 4], Ended::Finished)); + } + + /// A stream replaying frames the copy holds is read past them, not written twice. + #[moq_clock::test] + async fn a_replayed_prefix_is_not_written_twice() { + let mut bounded = Bounded::new(vec![bounded_stream(0, Some(3), 0..3), bounded_stream(0, None, 0..5)]); + bounded.recv().await.unwrap(); + bounded.recv().await.unwrap(); + assert_eq!(bounded.read(), (vec![0, 1, 2, 3, 4], Ended::Finished)); + } + + /// A stream ending inside the frames the copy holds contradicts them. The copy stays + /// open for a real continuation. + #[moq_clock::test] + async fn a_replay_ending_early_keeps_the_copy_open() { + let mut bounded = Bounded::new(vec![ + bounded_stream(0, Some(3), 0..3), + bounded_stream(0, None, 0..2), + bounded_stream(3, None, 3..4), + ]); + bounded.recv().await.unwrap(); + assert!(bounded.recv().await.is_err()); + assert_eq!(bounded.read(), (vec![0, 1, 2], Ended::Open)); + bounded.recv().await.unwrap(); + assert_eq!(bounded.read(), (vec![0, 1, 2, 3], Ended::Finished)); + } + + /// A frame at or past the stream's Frame End is malformed. + #[moq_clock::test] + async fn a_frame_past_the_frame_end_is_malformed() { + let mut bounded = Bounded::new(vec![bounded_stream(0, Some(2), 0..3)]); + let res = bounded.recv().await; + assert!(matches!(res, Err(Error::Decode(_))), "{res:?}"); + } + + /// An evicted copy can no longer be continued, so the continuation starts a new one. + #[moq_clock::test] + async fn a_continuation_of_an_evicted_copy_starts_anew() { + let mut bounded = Bounded::new(vec![bounded_stream(0, Some(3), 0..3), bounded_stream(3, None, 3..5)]); + bounded.recv().await.unwrap(); + let held = bounded.subscriber.subscribes.lock()[&7].suspended.0.0.lock()[0].clone(); + held.abort(Error::Evicted).unwrap(); + bounded.recv().await.unwrap(); + + let mut group = bounded.track.consume().peek_group(5).unwrap(); + assert_eq!(group.first_frame(), 3); + group.start_at(3); + let mut payloads = Vec::new(); + while let Some(frame) = group.read_frame().now_or_never().unwrap().unwrap() { + payloads.push(frame.payload[0]); + } + assert_eq!(payloads, [3, 4]); + } + + /// A suspended group ends with its subscription, so no reader waits on a continuation + /// that cannot come. + #[moq_clock::test] + async fn a_suspended_group_ends_with_its_subscription() { + let mut bounded = Bounded::new(vec![bounded_stream(0, Some(3), 0..3)]); + bounded.recv().await.unwrap(); + let mut group = bounded.track.consume().peek_group(5).unwrap(); + bounded.subscriber.remove_subscribe(7); + let res = group.read_frame().now_or_never(); + assert!(matches!(res, Some(Err(Error::Cancel))), "{res:?}"); + } + /// An idle copy that resubscribes gets its newest cached group again. That stream only /// duplicates the cache, but lite-07's SUBSCRIBE_END still counts it, so the tail must /// too, or the subscription waits out the grace before it ends. @@ -3222,6 +3572,7 @@ mod tests { subscribe: 7, sequence: 3, frame_start: 0, + frame_end: None, } .encode(&mut crate::coding::Encoder::new(&mut script, version.into()), version) .unwrap(); @@ -3259,6 +3610,7 @@ mod tests { compressed: false, }, tail: tail.clone(), + suspended: Default::default(), }, ); @@ -3627,6 +3979,7 @@ mod tests { compressed: false, }, tail: Default::default(), + suspended: Default::default(), }, ); @@ -6399,6 +6752,7 @@ impl TrackServe { producer: producer.clone(), framing: lite::Framing::new(producer.info(), self.subscriber.version), tail: tail.clone(), + suspended: Default::default(), }, ); diff --git a/rs/moq-net/src/lite/version.rs b/rs/moq-net/src/lite/version.rs index 5694bbab7cf..75f1297eb7b 100644 --- a/rs/moq-net/src/lite/version.rs +++ b/rs/moq-net/src/lite/version.rs @@ -148,6 +148,15 @@ impl Version { ) } + /// Whether GROUP carries `Frame End`, so a stream stopped at a subscription's end frame + /// does not end its group. Added in lite-07. + pub(crate) fn has_frame_end(self) -> bool { + !matches!( + self, + Self::Lite01 | Self::Lite02 | Self::Lite03 | Self::Lite04 | Self::Lite05 | Self::Lite06 + ) + } + /// Whether SUBSCRIBE and SUBSCRIBE_UPDATE carry the subscriber's `Group Order`. /// Added in lite-07; an earlier subscription is served newest first. pub(crate) fn has_order(self) -> bool { diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index 1d59be2aef0..a6a967b3e34 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -205,6 +205,11 @@ pub(super) struct GroupState { // Whether an adopted head (see [`Handle::adopt_head`]) begins at the group's first // object, for a wire header that arrives after it. adopted: Option, + + // The writer stopped at a request's end without ending the group (see + // [`Producer::suspend`]): the group holds the write permission until + // [`Handle::resume`] hands it to the stream continuing it. + suspended: bool, } /// The frames a copy of a group holds, from where it starts, which a successor copy of @@ -998,7 +1003,7 @@ struct Alive { impl Drop for Producer { fn drop(&mut self) { if let Ok(state) = self.handle.state.write() { - if state.fin.is_some() || state.abort.is_some() { + if state.fin.is_some() || state.abort.is_some() || state.suspended { return; } tracing::warn!( @@ -1632,6 +1637,22 @@ impl Producer { Ok(()) } + /// Stop writing without ending the group, because the stream writing it stopped at a + /// request's end rather than the group's. The group keeps the write permission, so + /// readers wait past the last frame until [`Handle::resume`] hands it to the stream + /// continuing the group, or the group is aborted. + #[cfg(feature = "async")] + // private: capability gap + // A custom subscriber cannot continue a copy cut at a request's end until resuming goes public. + pub(crate) fn suspend(self) -> Result<()> { + let mut state = modify(&self.handle.state)?; + debug_assert!(state.partial.is_none(), "a frame is still open"); + state.suspended = true; + drop(state); + // Dropping the writer now leaves the group open. + Ok(()) + } + /// Abort the group with the given error. /// /// Consumes the writer. Drops the cached frames so a stale [`Consumer`] can't pin @@ -1962,9 +1983,69 @@ impl Handle { (state.committed > state.offset).then_some(state.committed) } + /// Take the write permission a [`Producer::suspend`] left with the group, for a stream + /// continuing it from frame `at`. `None` unless the group is suspended and `at` falls + /// within what it holds; the caller discards the frames it replays below + /// [`Self::frame_count`]. + #[cfg(feature = "async")] + pub(super) fn resume(&self, at: u64, stats: stats::Meter) -> Option { + let mut state = self.state.write().ok()?; + let held = state.offset as u64..=state.committed as u64; + if !state.suspended || state.abort.is_some() || state.fin.is_some() || !held.contains(&at) { + return None; + } + state.suspended = false; + drop(state); + Some(Producer { + handle: self.clone(), + stats, + }) + } + + /// Ready once a writer of the group starting at or below `at` is no longer writing it: + /// it suspended, finished, or aborted. `None` when no such writer exists, so a stream + /// continuing the group from `at` need not wait for one. + #[cfg(feature = "async")] + pub(super) fn poll_writer(&self, at: u64, waiter: &kio::Waiter) -> Option> { + let res = self.state.poll(waiter, |state| { + if state.suspended || state.fin.is_some() || state.abort.is_some() || state.offset as u64 > at { + Poll::Ready(()) + } else { + Poll::Pending + } + }); + match res { + Poll::Ready(_) => None, + Poll::Pending => Some(Poll::Pending), + } + } + + /// Abort the group with `err` if it is still suspended, so nobody waits on a + /// continuation that will not come. + #[cfg(feature = "async")] + // private: capability gap + // A custom subscriber cannot continue a copy cut at a request's end until resuming goes public. + pub(crate) fn abort_suspended(&self, err: Error) { + if let Ok(state) = self.state.write() + && state.suspended + { + self.commit_abort(state, err); + } + } + + /// Whether the group waits on a stream continuing it; see [`Producer::suspend`]. + #[cfg(feature = "async")] + // private: capability gap + // A custom subscriber cannot continue a copy cut at a request's end until resuming goes public. + pub(crate) fn is_suspended(&self) -> bool { + self.state.read().suspended + } + /// Whether `other` is a handle to this same group. #[cfg_attr(not(feature = "async"), allow(dead_code, reason = "async-only"))] - pub(super) fn is_clone(&self, other: &Self) -> bool { + // private: capability gap + // A custom subscriber cannot continue a copy cut at a request's end until resuming goes public. + pub(crate) fn is_clone(&self, other: &Self) -> bool { self.state.same_channel(&other.state) } @@ -2877,6 +2958,14 @@ impl Consumer { self.cursor.end = end.map(|end| usize::try_from(end).unwrap_or(usize::MAX)); } + /// The group's total frame count once it finished, `None` before. + #[cfg(feature = "async")] + // private: capability gap + // A custom publisher cannot label a stream capped at a request's end until resuming goes public. + pub(crate) fn final_count(&self) -> Option { + self.cursor.state.read().fin.map(|fin| fin as u64) + } + /// The number of frames written so far (completed plus any in-flight), independent of /// how many this consumer has read. The final total once the group is finished. pub fn frame_count(&self) -> usize { diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 33cf960b66e..4894ec9a1a8 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -2713,6 +2713,46 @@ impl Producer { Ok(group) } + /// Take over the cached copy of `sequence` a stream stopped writing at a request's end + /// (see [`group::Producer::suspend`]), for a stream continuing it from `frame_start`. + /// + /// Pending while a copy starting at or below `frame_start` is still being written: the + /// stream continuing a group can overtake the one stopping it. `None` when no suspended + /// copy can continue from there. + #[cfg(feature = "async")] + // private: capability gap + // A custom subscriber cannot continue a copy cut at a request's end until resuming goes public. + pub(crate) fn poll_resume_group( + &self, + sequence: u64, + frame_start: u64, + waiter: &kio::Waiter, + ) -> Poll> { + let copies: Vec = match self.state.read().lookup.get(&sequence) { + Some(slot) => slot.copies().cloned().collect(), + None => return Poll::Ready(None), + }; + let mut writing = false; + for copy in &copies { + if let Some(group) = copy.resume(frame_start, self.stats.meter()) { + return Poll::Ready(Some(group)); + } + match copy.poll_writer(frame_start, waiter) { + Some(pending) => writing |= pending.is_pending(), + // It stopped since the first try. + None => { + if let Some(group) = copy.resume(frame_start, self.stats.meter()) { + return Poll::Ready(Some(group)); + } + } + } + } + match writing { + true => Poll::Pending, + false => Poll::Ready(None), + } + } + /// Create a new group with the next sequence number: one past this producer's own writes. /// /// [`Error::Duplicate`] if an earlier producer of the name in this broadcast already wrote diff --git a/rs/moq-net/src/recv.rs b/rs/moq-net/src/recv.rs index d005fd209c9..c50e0421e08 100644 --- a/rs/moq-net/src/recv.rs +++ b/rs/moq-net/src/recv.rs @@ -46,6 +46,11 @@ impl Group { } } + /// Whether the writer is here rather than with an in-flight frame. + pub fn has_writer(&self) -> bool { + self.writer.is_some() + } + /// Hand ownership to the next receive stage without closing the group. pub fn into_inner(mut self) -> group::Producer { self.armed = false; diff --git a/rs/moq-net/src/test_interop.rs b/rs/moq-net/src/test_interop.rs index f99c3404ed1..5998d251f79 100644 --- a/rs/moq-net/src/test_interop.rs +++ b/rs/moq-net/src/test_interop.rs @@ -89,6 +89,7 @@ fn lite_group(version: lite::Version, frames: impl IntoIterator track::Subscription { }]) } -async fn bounded_tail(version: Option, compressed: bool, frames: u64) { +/// How the subscribers ask for group 5, one after another. +struct Demand { + /// Where each subscriber's range ends and the next one's starts; the last ends at 5. + cuts: &'static [u64], + /// Each subscriber leaves before the next one asks. + leave: bool, +} + +/// Subscribers read group 5 of a `frames`-frame source in consecutive ranges, through two +/// relays on `version` or locally on `None`, each getting exactly its own frames. +async fn bounded_tail(version: Option, compressed: bool, frames: u64, demand: Demand) { let source = origin(1); let relay = origin(2); let reader = origin(3); @@ -47,115 +57,106 @@ async fn bounded_tail(version: Option, compressed: bool, frames: u64) { broadcast.consume() }; let track = remote.track("data").unwrap(); - let mut first = track.subscribe(range(0, 3)).await.unwrap(); - crate::support::harness::subscribed(&producer).await; - let mut group = producer.create_group(moq_net::group::Info { sequence: 5 }).unwrap(); - for index in 0..frames { - group - .write_frame(Timestamp::from_millis(index).unwrap(), vec![index as u8; 100]) - .unwrap(); - } - group.finish().unwrap(); - let mut prefix = first.recv_group().await.unwrap().unwrap(); - for index in 0..3.min(frames) { - assert_eq!( - prefix.read_frame().await.unwrap().unwrap().payload, - vec![index as u8; 100] - ); - } - assert!(prefix.read_frame().await.unwrap().is_none()); - let mut second = track.subscribe(range(3, 5)).await.unwrap(); - let mut tail = second.recv_group().await.unwrap().unwrap(); - // IETF passthrough readers expose frame ordinals independently of Object IDs; - // narrow this cursor too so that a stale prefix cannot hide the missing tail. - tail.set_frames(3..5); - for index in 3..5.min(frames) { - let frame = tail.read_frame().await.unwrap().expect("the requested tail frame"); - assert_eq!(frame.payload, vec![index as u8; 100]); + + let mut bounds = vec![0]; + bounds.extend_from_slice(demand.cuts); + bounds.push(5); + let mut held = Vec::new(); + for (index, pair) in bounds.windows(2).enumerate() { + let (start, end) = (pair[0], pair[1]); + let mut subscriber = track.subscribe(range(start, end)).await.unwrap(); + if index == 0 { + crate::support::harness::subscribed(&producer).await; + let mut group = producer.create_group(moq_net::group::Info { sequence: 5 }).unwrap(); + for frame in 0..frames { + group + .write_frame(Timestamp::from_millis(frame).unwrap(), vec![frame as u8; 100]) + .unwrap(); + } + group.finish().unwrap(); + } + let mut group = subscriber.recv_group().await.unwrap().unwrap(); + // IETF passthrough readers expose frame ordinals independently of Object IDs; + // narrow this cursor too so that a stale prefix cannot hide a missing tail. + group.set_frames(start..end); + for frame in start..end.min(frames) { + let read = group.read_frame().await.unwrap().expect("the requested frame"); + assert_eq!(read.payload, vec![frame as u8; 100]); + } + assert!(group.read_frame().await.unwrap().is_none()); + match demand.leave { + true => drop((group, subscriber)), + false => held.push((group, subscriber)), + } } - assert!(tail.read_frame().await.unwrap().is_none()); - drop((tail, prefix, second, first, links)); + drop((held, links)); } -#[moq_clock::test] -async fn a_bounded_lite06_copy_accepts_its_tail() { +async fn run(version: Option, compressed: bool, frames: u64, demand: Demand) { moq_clock::timeout( Duration::from_secs(1), - bounded_tail(Some("moq-lite-06".parse().unwrap()), false, 5), + bounded_tail(version, compressed, frames, demand), ) .await - .expect("a later subscriber receives the lite-06 tail"); + .expect("every subscriber reads its frames"); } -#[moq_clock::test] -async fn a_bounded_plain_copy_accepts_its_tail() { - moq_clock::timeout( - Duration::from_secs(1), - bounded_tail(Some("moq-lite-07-wip".parse().unwrap()), false, 5), - ) - .await - .expect("a later subscriber receives the plain tail"); +fn lite07() -> Option { + Some("moq-lite-07-wip".parse().unwrap()) } +const TAIL: Demand = Demand { + cuts: &[3], + leave: false, +}; + #[moq_clock::test] -async fn a_bounded_compressed_copy_accepts_its_tail() { - moq_clock::timeout( - Duration::from_secs(1), - bounded_tail(Some("moq-lite-07-wip".parse().unwrap()), true, 5), - ) - .await - .expect("a later subscriber receives and inflates the compressed tail"); +async fn a_bounded_plain_copy_accepts_its_tail() { + run(lite07(), false, 5, TAIL).await; } #[moq_clock::test] -async fn a_bounded_draft20_copy_accepts_its_tail() { - moq_clock::timeout( - Duration::from_secs(1), - bounded_tail(Some("moq-transport-20".parse().unwrap()), false, 5), - ) - .await - .expect("a later subscriber receives the transport tail"); +async fn a_bounded_compressed_copy_accepts_its_tail() { + run(lite07(), true, 5, TAIL).await; } #[moq_clock::test] -async fn a_bounded_draft21_copy_accepts_its_tail() { - moq_clock::timeout( - Duration::from_secs(1), - bounded_tail(Some("moq-transport-21".parse().unwrap()), false, 5), - ) - .await - .expect("a later subscriber receives the transport tail"); +async fn local_bounded_readers_keep_the_same_content() { + for compressed in [false, true] { + run(None, compressed, 5, TAIL).await; + } } #[moq_clock::test] -async fn a_bounded_draft22_copy_accepts_its_tail() { - moq_clock::timeout( - Duration::from_secs(1), - bounded_tail(Some("moq-transport-22".parse().unwrap()), false, 5), - ) - .await - .expect("a later subscriber receives the transport tail"); +async fn a_group_ending_before_or_at_the_request_bound_ends_cleanly() { + for compressed in [false, true] { + for frames in [2, 3] { + run(lite07(), compressed, frames, TAIL).await; + } + } } +/// Each widening continues the copy where the last stream stopped. #[moq_clock::test] -async fn local_bounded_readers_keep_the_same_content() { +async fn repeated_widening_continues_the_copy() { for compressed in [false, true] { - moq_clock::timeout(Duration::from_secs(1), bounded_tail(None, compressed, 5)) - .await - .expect("local readers receive the tail"); + let demand = Demand { + cuts: &[2, 4], + leave: false, + }; + run(lite07(), compressed, 5, demand).await; } } +/// A copy whose subscription ended is not continued, and a later request still gets the +/// frames it asks for. #[moq_clock::test] -async fn a_group_ending_before_or_at_the_request_bound_ends_cleanly() { +async fn a_tail_after_the_subscription_ended_is_fetched_anew() { for compressed in [false, true] { - for frames in [2, 3] { - moq_clock::timeout( - Duration::from_secs(1), - bounded_tail(Some("moq-lite-07-wip".parse().unwrap()), compressed, frames), - ) - .await - .expect("a real end remains readable after demand widens"); - } + let demand = Demand { + cuts: &[3], + leave: true, + }; + run(lite07(), compressed, 5, demand).await; } }