From 739082cb2225d0a2a37d40cab622e79f28af39a8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 18 Oct 2026 22:12:11 -0700 Subject: [PATCH 01/36] quest(m1): claim subscribe-ranges/dynamic Co-Authored-By: Claude Opus 5.5 From 40d72f0e3454334d4c7bd897439409d63f38af7b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 18 Oct 2026 23:23:41 -0700 Subject: [PATCH 02/36] feat(moq-net): serve each subscriber its own ranges, with time anchors A subscriber now reads only the groups its own Subscription::ranges cover, resolved where it is served and again whenever they are replaced, instead of reading every group the aggregate brings in. A new Anchor::Time(pts) resolves from the cached groups' starts; an untimed track refuses it. The drop scan names only sequences inside the reader's ranges, so the SUBSCRIBE_DROP accounting can reuse it. moq-c and moq-mux stop re-resolving relative starts on their own. moq-ffi, moq-c, and the bindings carry the time anchor, and the FFI update now reports a refused one. Adds track_range_serving, which found and fixed two per-range walks per hand-out. Co-Authored-By: Claude Opus 5.5 --- dart/moq/MIGRATE.md | 6 + doc/concept/moq-lite.md | 15 +- go/wrapper/MIGRATE.md | 6 + go/wrapper/subscribe.go | 7 +- go/wrapper/types.go | 4 +- kt/moq/MIGRATE.md | 6 + .../kotlin/dev/moq/Aliases.kt | 4 +- py/moq-rs/MIGRATE.md | 6 + py/moq-rs/moq/subscribe.py | 5 +- quest/m1/subscribe-ranges/dynamic.md | 35 +- rs/moq-c/src/api.rs | 12 +- rs/moq-c/src/consume.rs | 41 +- rs/moq-c/src/test.rs | 83 +++ rs/moq-ffi/MIGRATE.md | 6 + rs/moq-ffi/src/consumer.rs | 66 ++- rs/moq-ffi/src/media.rs | 5 +- rs/moq-ffi/src/producer.rs | 2 +- rs/moq-ffi/src/test.rs | 64 +- rs/moq-mux/src/container/consumer.rs | 46 +- rs/moq-net/MIGRATE.md | 7 + rs/moq-net/benches/track.rs | 51 ++ rs/moq-net/src/model/resume.rs | 9 + rs/moq-net/src/model/subscription.rs | 111 ++-- rs/moq-net/src/model/subscription/window.rs | 430 ++++++++++++++ rs/moq-net/src/model/track.rs | 558 ++++++++++++++---- rs/moq-net/src/model/track/demand.rs | 22 +- rs/moq-net/src/model/track/demand/ranges.rs | 5 +- swift/MIGRATE.md | 6 + swift/Sources/Moq/Track.swift | 7 +- 29 files changed, 1353 insertions(+), 272 deletions(-) create mode 100644 rs/moq-net/src/model/subscription/window.rs diff --git a/dart/moq/MIGRATE.md b/dart/moq/MIGRATE.md index 965ea3b1cd..bae8c41459 100644 --- a/dart/moq/MIGRATE.md +++ b/dart/moq/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Ranges filter reads, and anchors can name a time + +A subscription now reads only the groups its own `ranges` cover. +`Anchor` gains a time variant, so exhaustive switches need a case for it. +`TrackConsumer.update` now throws on a time anchor an untimed track can't serve. + ### Subscriptions carry group ranges (#5732) `Subscription.groupStart` and `groupEnd` are replaced by `ranges`, a list of `Range` whose ends are an `Anchor` (an absolute group and frame, or a count of groups before the latest), and an optional `order`. diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index 4a207f1497..eb41810de2 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -216,12 +216,15 @@ Each subscription carries the knobs that decide behavior under congestion: | **Priority** (0..255) | Higher-priority tracks get bandwidth first. Audio above video, base layer above enhancement. From a native Rust publisher over QUIC or WebTransport (iroh excepted), subscriptions at the same priority split bandwidth evenly by bytes, streams and datagrams alike, however many groups each has queued. A browser publisher and other transports without send groups are not covered. | | **Order** | Which group to send first when several are pending. Newest first for live (the default), oldest first for catch-up. Needs moq-lite 07 or moq-transport; older moq-lite always sends newest first. | | **Max delay** | How long a non-latest group stays worth delivering before it is skipped. Zero means "live edge only", and raising it is also what asks for history. | -| **Ranges** | Which groups to ask for: from a group, or a number of groups back from the latest, optionally up to an end. The default asks for everything max delay allows. Max delay still caps every range. | - -Ranges are a request, not a filter. A subscriber reads from where its lowest -range starts, and may also get groups another subscriber of the same track -asked for. A relay merges its subscribers' ranges into one upstream request, -which today's wire versions carry as the single span covering them. +| **Ranges** | Which groups to ask for: from a group, a number of groups back from the latest, or the group containing a presentation time, optionally up to an end. The default asks for everything max delay allows. Max delay still caps every range. | + +Ranges are both a request and a filter. A subscriber reads only the groups its +own ranges cover, resolved again whenever it replaces them, even when another +subscriber of the same track asked for more. A relay merges its subscribers' +ranges into one upstream request, which today's wire versions carry as the +single span covering them. A time anchor is resolved from the groups the +serving hop holds; upstream it asks for everything max delay allows, and an +untimed track refuses it. Order only decides what goes first, never what is skipped, so catching up oldest first needs a max delay that covers the backlog. When a relay serves subscribers of diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index 8689478554..6c75b68765 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Ranges filter reads, and anchors can name a time + +A subscription now reads only the groups its own `Ranges` cover. +`AnchorTime` names the group containing a presentation time, and type switches over `Anchor` need a case for it. +`TrackConsumer.Update` also fails on a time anchor an untimed track can't serve. + ### Subscriptions carry group ranges (#5732) `Subscription.GroupStart` and `GroupEnd` are replaced by `Ranges`, a slice of `Range` whose ends are `AnchorAbsolute` or `AnchorRelative` (an absolute group and frame, or a count of groups before the latest), and an optional `Order`. diff --git a/go/wrapper/subscribe.go b/go/wrapper/subscribe.go index ce9764b6b7..74f3d7cdae 100644 --- a/go/wrapper/subscribe.go +++ b/go/wrapper/subscribe.go @@ -156,15 +156,14 @@ func (t *TrackConsumer) Info() (TrackInfo, error) { return trackInfoFromFFI(info), nil } -// Update changes this subscriber's delivery preferences. It fails only on a -// negative MaxDelay. +// Update changes this subscriber's delivery preferences. It fails on a +// negative MaxDelay, or a time anchor the track can't serve. func (t *TrackConsumer) Update(subscription Subscription) error { sub, err := subscription.ffi() if err != nil { return err } - t.inner.Update(sub) - return nil + return t.inner.Update(sub) } // Groups ranges over groups in sequence order. diff --git a/go/wrapper/types.go b/go/wrapper/types.go index 2139b6ed3b..53da90eeea 100644 --- a/go/wrapper/types.go +++ b/go/wrapper/types.go @@ -58,12 +58,14 @@ type ( // A relative start is the first frame of its group, a relative end includes its whole // group, and an absolute end is exclusive. Range = ffi.MoqRange - // Anchor is where one end of a Range falls: AnchorAbsolute or AnchorRelative. + // Anchor is where one end of a Range falls: AnchorAbsolute, AnchorRelative, or AnchorTime. Anchor = ffi.MoqAnchor // AnchorAbsolute is an exact group and frame. AnchorAbsolute = ffi.MoqAnchorAbsolute // AnchorRelative is the group Groups before the latest one, clamped to group 0: 0 is the latest. AnchorRelative = ffi.MoqAnchorRelative + // AnchorTime is the group containing TimestampUs, a presentation time in microseconds. + AnchorTime = ffi.MoqAnchorTime // Order is which of a subscription's pending groups the publisher sends first. Order = ffi.MoqOrder // VideoDecoderOutput configures what a video.Consumer delivers: an optional resize, a max delay, and whether frames keep the decoder's surface (macOS only; refused elsewhere). diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index 2552722b9e..8710934b05 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Ranges filter reads, and anchors can name a time + +A subscription now reads only the groups its own `ranges` cover. +`Anchor` gains `AnchorTime`, so exhaustive `when` expressions need a branch for it. +`TrackConsumer.update` now throws `MoqException` on a time anchor an untimed track can't serve. + ### Subscriptions carry group ranges (#5732) `Subscription.groupStart` and `groupEnd` are replaced by `ranges`, a list of `Range` whose ends are `AnchorAbsolute` or `AnchorRelative` (an absolute group and frame, or a count of groups before the latest), and an optional `order`. diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt index 87c02cff36..6430cf04fc 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt @@ -103,12 +103,14 @@ typealias Route = uniffi.moq.MoqRoute typealias Subscription = uniffi.moq.MoqSubscription /** A span of a track to deliver: from `start` up to `end`, or onward when `end` is null. */ typealias Range = uniffi.moq.MoqRange -/** Where one end of a [Range] falls: [AnchorAbsolute] or [AnchorRelative]. */ +/** Where one end of a [Range] falls: [AnchorAbsolute], [AnchorRelative], or [AnchorTime]. */ typealias Anchor = uniffi.moq.MoqAnchor /** An exact group and frame. */ typealias AnchorAbsolute = uniffi.moq.MoqAnchor.Absolute /** The group `groups` before the latest one, clamped to group 0: 0 is the latest. */ typealias AnchorRelative = uniffi.moq.MoqAnchor.Relative +/** The group containing `timestampUs`, a presentation time in microseconds. */ +typealias AnchorTime = uniffi.moq.MoqAnchor.Time /** Which pending group the publisher sends first: `ASCENDING` or `DESCENDING`. */ typealias Order = uniffi.moq.MoqOrder /** Options for fetching one past group by sequence. */ diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 980942e7d1..ed785df58a 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Ranges filter reads, and anchors can name a time + +A subscription now reads only the groups its own `ranges` cover. +`Anchor` gains a `TIME` variant, the group containing a presentation time. +`TrackConsumer.update` now raises on a time anchor an untimed track can't serve. + ### Subscriptions carry group ranges (#5732) `Subscription.group_start` and `group_end` are replaced by `ranges`, a list of `Range` whose ends are an `Anchor` (an absolute group and frame, or a count of groups before the latest), and an optional `order`. diff --git a/py/moq-rs/moq/subscribe.py b/py/moq-rs/moq/subscribe.py index 58cdd2901b..c425149408 100644 --- a/py/moq-rs/moq/subscribe.py +++ b/py/moq-rs/moq/subscribe.py @@ -149,7 +149,10 @@ def info(self) -> TrackInfo: return TrackInfo._from_ffi(self._inner.info()) def update(self, subscription: Subscription) -> None: - """Change this subscriber's delivery preferences.""" + """Change this subscriber's delivery preferences. + + Raises on a time anchor the track can't serve. + """ self._inner.update(subscription._ffi()) def cancel(self) -> None: diff --git a/quest/m1/subscribe-ranges/dynamic.md b/quest/m1/subscribe-ranges/dynamic.md index ad5fd0d567..fb2ee1b3fa 100644 --- a/quest/m1/subscribe-ranges/dynamic.md +++ b/quest/m1/subscribe-ranges/dynamic.md @@ -11,19 +11,13 @@ kept beside it. ## Plan -`track::Subscription` already carries aggregated `ranges`, but only as a wire -preference: locally a subscriber's ranges set where its reads start, and it -still reads every later group the aggregate brings in. Serving a range locally -means filtering a subscriber's reads to its own ranges, and a miss below the -cache becomes a range request on `Dynamic` rather than a group request. +A subscriber already reads only its own ranges, re-resolved when they are +replaced, and a `Time(pts)` anchor resolves from the cached groups' starts, or +is refused on an untimed track. What remains is the miss: one below the cache +becomes a range request on `Dynamic` rather than a group request. -- Add a `Time(pts)` anchor beside `Absolute` and `Relative` (see the - [line](/quest/m1/subscribe-ranges/README.md)), carried through moq-ffi and - every binding like the other anchors. A track resolves it from its cached - groups' timestamps and passes an unresolved one to its `track::Dynamic` - handler; an untimed track refuses it. Test a time start and end resolved - against the cache, an unresolved one passed through to a handler, and an - untimed track refusing one. +- Pass a time anchor the cache can't resolve to the `track::Dynamic` handler + in its range request, and test one passed through. - Replace `requested_group` and `fetch_group` rather than keep both. Update every caller (relay, moq-archive, hls, the ladder) in the same PR, including the archive consumer's time resolution from its key listing if it @@ -34,7 +28,9 @@ cache becomes a range request on `Dynamic` rather than a group request. `moq_relay::fetch_group` (added in the 2026-10-05 audit). - Every sequence in range either arrives or is named as dropped, so a reader never waits on a miss. Share the accounting with - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) rather than adding a second one. + [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) rather than adding a second one: + the cursor's drop scan already names only sequences inside the reader's own + ranges. - Decided in the 2026-10-10 audit: `max_delay` caps every range, including historical FETCH replacements, and publisher `max_age` always clamps it. A FETCH with no reader delay uses publisher max-age alone, not the live @@ -43,17 +39,8 @@ cache becomes a range request on `Dynamic` rather than a group request. budget or lets one expired caller cancel another still-valid reader. Reuse FETCH's optional per-reader budget and publisher clamp rather than restoring a historical exemption. -- Move every relative-start resolution into the model, which counts back - from produced groups only. Today two consumers re-resolve relative starts - on their own. moq-c's `apply_raw_subscription` (`rs/moq-c/src/consume.rs`) - counts back from `Ordered::latest()`, the shared group and datagram - sequence, so a datagram past the newest group skews the start. It also - re-resolves on every update, because the model doesn't. moq-mux's - `container::Consumer::new` treats a relative start as an unfloored join - and adopts the first arrival, which under newest-first delivery can skip - the older groups the range asked for. -- Benchmark serving and filling over range count and span as separate axes, - beside `track_subscription_ranges`, which covers aggregation. +- Benchmark filling over range count and span as separate axes, beside + `track_range_serving`, which covers serving. ## Related diff --git a/rs/moq-c/src/api.rs b/rs/moq-c/src/api.rs index 80890cd523..401caafc78 100644 --- a/rs/moq-c/src/api.rs +++ b/rs/moq-c/src/api.rs @@ -583,8 +583,8 @@ pub struct moq_subscription { /// A span of a track to deliver: from `start` up to `end`, or onward without one. /// -/// A relative start is the first frame of its group, and a relative end includes its whole -/// group. An absolute end is exclusive. +/// A relative or time start is the first frame of its group, and a relative or time end +/// includes its whole group. An absolute end is exclusive. #[repr(C)] #[allow(non_camel_case_types)] #[derive(Clone, Copy)] @@ -605,7 +605,8 @@ pub struct moq_anchor { /// `moq_anchor_kind` discriminant. pub kind: u32, /// The group, or with `MOQ_ANCHOR_RELATIVE` how many groups before the latest one - /// (0 is the latest, and an offset past the first group clamps to it). + /// (0 is the latest, and an offset past the first group clamps to it), or with + /// `MOQ_ANCHOR_TIME` a presentation time in microseconds. pub group: u64, /// The frame within `group`, read only with `MOQ_ANCHOR_ABSOLUTE`. pub frame: u64, @@ -620,6 +621,8 @@ pub enum moq_anchor_kind { MOQ_ANCHOR_ABSOLUTE = 0, /// A count of groups back from the latest one. MOQ_ANCHOR_RELATIVE = 1, + /// The group containing a presentation time. An untimed track refuses it. + MOQ_ANCHOR_TIME = 2, } /// Which of a subscription's pending groups the publisher sends first. Zero is the default. @@ -642,6 +645,9 @@ fn parse_anchor(anchor: &moq_anchor) -> Result { }) } v if v == moq_anchor_kind::MOQ_ANCHOR_RELATIVE as u32 => moq_net::track::Anchor::Relative(anchor.group), + v if v == moq_anchor_kind::MOQ_ANCHOR_TIME as u32 => { + moq_net::track::Anchor::Time(moq_net::Timestamp::from_micros(anchor.group)?) + } _ => return Err(Error::InvalidCode), }) } diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index 2597f60932..5638cc2083 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -1,7 +1,6 @@ use std::{ ffi::c_char, future::Future, - ops::Bound, pin::Pin, task::{Poll, ready}, }; @@ -683,37 +682,15 @@ impl Consume { } fn apply_raw_subscription(track: &mut moq_net::track::Ordered, subscription: Option) { - let subscription = subscription.unwrap_or_default(); - // Reads start where the lowest range does. Without a floor of its own (the oldest - // range, the default) a raw reader joins at the latest group. - let latest = match track.poll_largest(&kio::Waiter::noop()) { - std::task::Poll::Ready(Some(position)) => Some(position.group), - _ => None, - }; - let start = subscription - .ranges - .iter() - .map(|range| match range.start { - moq_net::track::Anchor::Absolute(start) => Some(start.group), - moq_net::track::Anchor::Relative(u64::MAX) => latest, - moq_net::track::Anchor::Relative(back) => latest.map(|latest| latest.saturating_sub(back)), - }) - .min() - .flatten(); - // They stop past the last absolute end; an open or relative one follows the live edge. - let end = subscription - .ranges - .iter() - .map(|range| match range.end { - Some(moq_net::track::Anchor::Absolute(end)) => Some(end), - _ => None, - }) - .collect::>>() - .map(|ends| ends.into_iter().max().unwrap_or_default()); - track.set_groups(( - start.map_or(Bound::Unbounded, Bound::Included), - end.map_or(Bound::Unbounded, moq_net::track::Position::group_end), - )); + let mut subscription = subscription.unwrap_or_default(); + // The subscription reads only its own ranges, resolved where it is served. A raw + // reader without a floor of its own (the oldest range, the default) joins at the + // latest group instead. + for range in &mut subscription.ranges { + if range.start == moq_net::track::Range::oldest().start { + range.start = moq_net::track::Range::live().start; + } + } // A closed track makes the update meaningless; the reader already sees the close. let _ = track.update(subscription); } diff --git a/rs/moq-c/src/test.rs b/rs/moq-c/src/test.rs index ac8de2aea6..4ea412b7b0 100644 --- a/rs/moq-c/src/test.rs +++ b/rs/moq-c/src/test.rs @@ -1804,6 +1804,89 @@ fn raw_track_subscription_options_and_update() { assert_eq!(moq_origin_close(origin), 0); } +/// A time anchor starts at the group containing that presentation time. +#[test] +fn raw_track_time_anchor() { + let origin = id(moq_origin_create()); + let path = b"raw-track-time"; + let broadcast = publish_broadcast(origin, path); + + let track_name = b"data"; + let info = moq_track_info { + priority: 3, + max_age_us: 1_000_000, + max_age_present: true, + timescale: 1_000_000, + timescale_present: true, + }; + let track = + id(unsafe { moq_publish_track(broadcast, track_name.as_ptr() as *const c_char, track_name.len(), &info) }); + + let payloads: [&[u8]; 3] = [b"zero", b"one", b"two"]; + for (i, payload) in payloads.into_iter().enumerate() { + assert_eq!( + unsafe { moq_publish_track_frame(track, payload.as_ptr(), payload.len(), i as u64 * 20_000) }, + 0 + ); + } + + let consume = request_broadcast(origin, path); + + let frame_cb = Callback::new(); + let ranges = [moq_range { + start: moq_anchor { + kind: moq_anchor_kind::MOQ_ANCHOR_TIME as u32, + group: 30_000, + frame: 0, + }, + end: moq_anchor { + kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, + group: 0, + frame: 0, + }, + end_present: false, + }]; + let subscription = moq_subscription { + priority: 5, + max_delay_us: 1_000_000, + ranges: ranges.as_ptr(), + ranges_len: ranges.len(), + order: 0, + }; + let consumer = id(unsafe { + moq_consume_track( + consume, + track_name.as_ptr() as *const c_char, + track_name.len(), + &subscription, + Some(channel_callback), + frame_cb.ptr, + ) + }); + + for expected in [&b"one"[..], b"two"] { + let frame_id = id(frame_cb.recv()); + let mut frame = moq_frame { + payload: std::ptr::null(), + payload_size: 0, + timestamp_us: 0, + timestamp_present: false, + keyframe: false, + }; + assert_eq!(unsafe { moq_consume_track_frame(frame_id, &mut frame) }, 0); + let received = unsafe { std::slice::from_raw_parts(frame.payload, frame.payload_size) }; + assert_eq!(received, expected); + assert_eq!(moq_consume_track_frame_free(frame_id), 0); + } + + assert_eq!(moq_consume_track_cancel(consumer), 0); + assert_eq!(frame_cb.recv_terminal(), 0); + assert_eq!(moq_publish_track_finish(track), 0); + assert_eq!(moq_consume_close(consume), 0); + assert_eq!(moq_publish_close(broadcast), 0); + assert_eq!(moq_origin_close(origin), 0); +} + #[test] fn json_snapshot_publish_consume() { let origin = id(moq_origin_create()); diff --git a/rs/moq-ffi/MIGRATE.md b/rs/moq-ffi/MIGRATE.md index e75eb72a71..56a0a708a0 100644 --- a/rs/moq-ffi/MIGRATE.md +++ b/rs/moq-ffi/MIGRATE.md @@ -5,6 +5,12 @@ These are the UniFFI names; each language wrapper has its own `MIGRATE.md` in it ## Unreleased +### Ranges filter reads, and anchors can name a time + +A subscription now reads only the groups its own `ranges` cover, rather than starting at the lowest one and taking whatever arrives. +`MoqAnchor` gains `Time { timestamp_us }`, the group containing that presentation time, so exhaustive matches need an arm for it. +`MoqTrackConsumer::update` now returns `Result<(), MoqError>`, failing with `TimestampMismatch` on a time anchor an untimed track can't serve. + ### Subscriptions carry group ranges (#5732) `MoqSubscription::group_start` and `group_end` are replaced by `ranges`, a list of `MoqRange { start, end }` whose ends are `MoqAnchor::Absolute { group, frame }` or `MoqAnchor::Relative { groups }`, and an optional `order`. diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index 4eb8e2945b..664e5165c2 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -38,9 +38,8 @@ pub struct MoqSubscription { /// The spans of the track to deliver, or null for everything `max_delay_us` still /// considers fresh. An empty list asks for nothing. /// - /// A request to the publisher, aggregated across every subscriber. The one place it - /// touches this subscription's reads is where they start: at the lowest group any - /// range starts at. + /// A request to the publisher, aggregated across every subscriber, and this + /// subscription's read filter: it reads only the groups its own ranges cover. #[uniffi(default = None)] pub ranges: Option>, /// Which pending group the publisher sends first, or null for newest first. @@ -50,8 +49,8 @@ pub struct MoqSubscription { /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is null. /// -/// A relative start is the first frame of its group, and a relative end includes its whole -/// group. An absolute end is exclusive. +/// A relative or time start is the first frame of its group, and a relative or time end +/// includes its whole group. An absolute end is exclusive. #[derive(Clone, Copy, uniffi::Record)] pub struct MoqRange { /// Where delivery starts. @@ -68,6 +67,9 @@ pub enum MoqAnchor { Absolute { group: u64, frame: u64 }, /// The group this many before the latest, clamped to group 0: `0` is the latest group. Relative { groups: u64 }, + /// The group containing this presentation time, in microseconds. An untimed track + /// refuses it. + Time { timestamp_us: u64 }, } /// Which of a subscription's pending groups the publisher sends first, mirroring @@ -80,21 +82,28 @@ pub enum MoqOrder { Descending, } -impl From for moq_net::track::Anchor { - fn from(anchor: MoqAnchor) -> Self { - match anchor { +impl TryFrom for moq_net::track::Anchor { + type Error = MoqError; + + fn try_from(anchor: MoqAnchor) -> Result { + Ok(match anchor { MoqAnchor::Absolute { group, frame } => Self::Absolute(moq_net::track::Position { group, frame }), MoqAnchor::Relative { groups } => Self::Relative(groups), - } + MoqAnchor::Time { timestamp_us } => { + Self::Time(moq_net::Timestamp::from_micros(timestamp_us).map_err(|_| MoqError::TimeOverflow)?) + } + }) } } -impl From for moq_net::track::Range { - fn from(range: MoqRange) -> Self { - Self { - start: range.start.into(), - end: range.end.map(Into::into), - } +impl TryFrom for moq_net::track::Range { + type Error = MoqError; + + fn try_from(range: MoqRange) -> Result { + Ok(Self { + start: range.start.try_into()?, + end: range.end.map(TryInto::try_into).transpose()?, + }) } } @@ -128,18 +137,25 @@ impl From for moq_net::group::Fetch { } } -impl From for moq_net::track::Subscription { - fn from(s: MoqSubscription) -> Self { +impl TryFrom for moq_net::track::Subscription { + type Error = MoqError; + + fn try_from(s: MoqSubscription) -> Result { let mut subscription = moq_net::track::Subscription::default() .with_priority(s.priority) .with_max_delay(std::time::Duration::from_micros(s.max_delay_us)); if let Some(ranges) = s.ranges { - subscription = subscription.with_ranges(ranges.into_iter().map(Into::into)); + subscription = subscription.with_ranges( + ranges + .into_iter() + .map(TryInto::try_into) + .collect::, _>>()?, + ); } if let Some(order) = s.order { subscription = subscription.with_order(order.into()); } - subscription + Ok(subscription) } } @@ -255,7 +271,7 @@ impl MoqBroadcastConsumer { ) -> Result, MoqError> { self.driver .watch(async { - let subscription = subscription.map(moq_net::track::Subscription::from); + let subscription = subscription.map(moq_net::track::Subscription::try_from).transpose()?; let track = self.inner.track(&name)?.subscribe(subscription).await?; Ok(Arc::new(MoqTrackConsumer::new(track, self.driver.clone()))) }) @@ -663,9 +679,13 @@ impl MoqTrackConsumer { /// Change this subscriber's delivery preferences. /// /// Silently ignored if the track already ended; the update is meaningless at - /// that point. - pub fn update(&self, subscription: MoqSubscription) { - let _ = self.control.update(subscription.into()); + /// that point. Errors on a time anchor the track can't serve, leaving the + /// preferences as they were. + pub fn update(&self, subscription: MoqSubscription) -> Result<(), MoqError> { + match self.control.update(subscription.try_into()?) { + Ok(()) | Err(moq_net::Error::Closed) => Ok(()), + Err(err) => Err(err.into()), + } } /// Cancel all current and future reads. diff --git a/rs/moq-ffi/src/media.rs b/rs/moq-ffi/src/media.rs index d080bf6573..8b75a9da21 100644 --- a/rs/moq-ffi/src/media.rs +++ b/rs/moq-ffi/src/media.rs @@ -1411,7 +1411,10 @@ impl MoqMediaTrackConsumer { // subscription if init parsing fails. let media = media_container(container)?; let resolved = broadcast.resolve_inner(reference.as_deref()).await?; - let subscription = subscription.map(moq_net::track::Subscription::from).unwrap_or_default(); + let subscription = subscription + .map(moq_net::track::Subscription::try_from) + .transpose()? + .unwrap_or_default(); let track = broadcast .driver .watch(async { Ok(resolved.track(&name)?.subscribe(subscription).await?) }) diff --git a/rs/moq-ffi/src/producer.rs b/rs/moq-ffi/src/producer.rs index 9e30534e81..a435dc166c 100644 --- a/rs/moq-ffi/src/producer.rs +++ b/rs/moq-ffi/src/producer.rs @@ -517,7 +517,7 @@ impl MoqTrackProducer { let _guard = crate::ffi::enter(); let guard = self.inner.lock().unwrap(); let track = guard.as_ref().ok_or(MoqError::Closed)?; - let subscription = subscription.map(moq_net::track::Subscription::from); + let subscription = subscription.map(moq_net::track::Subscription::try_from).transpose()?; Ok(Arc::new(MoqTrackConsumer::new( track.subscribe(subscription), Background::default(), diff --git a/rs/moq-ffi/src/test.rs b/rs/moq-ffi/src/test.rs index 001a0c6616..ef18837228 100644 --- a/rs/moq-ffi/src/test.rs +++ b/rs/moq-ffi/src/test.rs @@ -919,7 +919,7 @@ async fn raw_track_update_does_not_wait_for_pending_read() { end: None, }]), order: None, - }); + }).unwrap(); let payload = b"updated subscription".to_vec(); track @@ -3911,7 +3911,7 @@ async fn raw_track_update_during_pending_group_still_reads_datagram() { end: None, }]), order: None, - }); + }).unwrap(); let sequence = track.append_datagram(datagram_frame(b"after-update")).unwrap(); let datagram = tokio::time::timeout(TIMEOUT, consumer.recv_datagram()) @@ -6591,7 +6591,7 @@ async fn ascending_subscription_reads_a_cached_backlog_oldest_first() { }]), order: Some(MoqOrder::Ascending), }; - let converted = moq_net::track::Subscription::from(subscription.clone()); + let converted = moq_net::track::Subscription::try_from(subscription.clone()).unwrap(); assert_eq!(converted.order, moq_net::track::Order::Ascending); assert_eq!( converted.ranges, @@ -6613,3 +6613,61 @@ async fn ascending_subscription_reads_a_cached_backlog_oldest_first() { } assert_eq!(sequences, vec![0, 1, 2]); } + +/// A time anchor starts at the group containing it, and an untimed track refuses one. +#[tokio::test] +async fn time_anchors_start_at_the_containing_group() { + let broadcast = MoqBroadcastProducer::new().unwrap(); + let track = broadcast.publish_track("events".into(), Some(timed_info())).unwrap(); + for timestamp_us in [0, 1_000, 2_000] { + let group = track.append_group().unwrap(); + group + .write_frame(MoqFrame { + payload: b"x".to_vec(), + timestamp_us: Some(timestamp_us), + }) + .unwrap(); + group.finish().unwrap(); + } + + let since = |timestamp_us| MoqSubscription { + priority: 0, + max_delay_us: 10_000_000, + ranges: Some(vec![MoqRange { + start: MoqAnchor::Time { timestamp_us }, + end: None, + }]), + order: Some(MoqOrder::Ascending), + }; + let consumer = track.consume(Some(since(1_500))).unwrap(); + let mut sequences = Vec::new(); + for _ in 0..2 { + let group = tokio::time::timeout(TIMEOUT, consumer.recv_group()) + .await + .expect("timed out waiting for a cached group") + .unwrap() + .expect("expected a group"); + sequences.push(group.sequence()); + } + assert_eq!(sequences, vec![1, 2]); + + let untimed = broadcast + .publish_track( + "status".into(), + Some(MoqTrackInfo { + timescale: None, + ..timed_info() + }), + ) + .unwrap(); + let consumer = untimed.consume(None).unwrap(); + assert!(matches!( + consumer.update(since(0)), + Err(MoqError::Protocol { + details: MoqProtocolError { + kind: MoqProtocolKind::TimestampMismatch, + .. + } + }) + )); +} diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index fe26ab0ec3..c6de8b9180 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -161,17 +161,7 @@ impl Consumer { // edge from there. The budget is clamped to the track's retention window, // since history the publisher no longer keeps can't be waited for and would // otherwise stall the catch-up by the excess. - // Only absolute starts name a floor: a relative one counts back from wherever the - // publisher is, which the first arrivals say. - let start = subscription - .ranges - .iter() - .map(|range| match range.start { - moq_net::track::Anchor::Absolute(start) => Some(start.group), - moq_net::track::Anchor::Relative(_) => None, - }) - .collect::>>() - .and_then(|starts| starts.into_iter().min()); + let start = track.floor(); let max_delay = subscription .max_delay .min(track.info().max_age.unwrap_or(std::time::Duration::MAX)); @@ -290,6 +280,15 @@ impl Consumer { // (a gap below the served window looks like in-flight groups until the budget // expires, or forever on a quiet track), so the cursor adopts it from the // first arrivals instead of assuming group 0. Skipping stays the budget's job. + // A relative start names its floor once the first group to arrive resolves it. + if self.startup + && !self.newest + && let Some(start) = self.track.floor() + { + self.pending.retain(|group| group.sequence >= start); + self.current = start; + self.startup = false; + } if self.startup { // NOTE: poll_min_timestamp buffers at least one frame per group and // registers the waiter on the ones still empty. Any wire frame counts, so a @@ -2888,6 +2887,31 @@ mod tests { assert_eq!(micros, vec![0, 100_000], "the open group's late frames are delivered"); } + /// A relative start names its floor once resolved: delivery starts that many groups + /// back from the latest, rather than at whichever group arrives first. + #[tokio::test] + async fn startup_at_a_relative_start() { + tokio::time::pause(); + let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); + for sequence in 5..=9 { + write_group(&mut track, sequence, &[ts(sequence * 100_000)]); + } + let subscription = moq_net::track::Subscription::default() + .with_max_delay(Duration::from_secs(10)) + .with_ranges([moq_net::track::Range { + start: moq_net::track::Anchor::Relative(2), + end: None, + }]); + let consumer_track = track.subscribe(subscription); + assert_eq!(consumer_track.floor(), Some(7)); + let mut consumer = Consumer::new(consumer_track, Container::Legacy(crate::container::Kind::Data)); + track.finish().unwrap(); + + let frames = read_all(&mut consumer).await.unwrap(); + let micros: Vec = frames.iter().map(|f| f.timestamp.as_micros()).collect(); + assert_eq!(micros, vec![700_000, 800_000, 900_000]); + } + #[tokio::test] async fn startup_single_group_mid_stream() { let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index bdcf4edb0b..1aadbbf5f3 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -4,6 +4,13 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Ranges filter each subscriber's reads + +A `track::Subscriber` now reads only the groups its own `Subscription::ranges` cover, resolved again whenever `update` replaces them; before, they only set where its reads started. +A reader that relied on groups another subscriber asked for must widen its own ranges. +`track::Anchor` gains `Time(Timestamp)`, the group containing a presentation time, so exhaustive matches need an arm for it. +Subscribing or updating with one on an untimed track fails with `Error::TimestampMismatch`. + ### Track readers follow `largest()` instead of reading `latest()` (#5725) `latest()` on `track::{Producer, Consumer, Subscriber, Ordered}` is gone. diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index 73756c0f41..f0887e1e6a 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -56,6 +56,10 @@ //! disjoint range, swept over how many ranges and how many groups each spans: reading the //! union must cost the same per range whatever it spans. //! +//! `track_range_serving` drains one reader whose own ranges are disjoint, swept over how +//! many ranges and how many cached groups each spans: each group handed out must cost the +//! same, so a walk over the ranges or the groups between them shows up as a slope. +//! //! Run with `cargo bench -p moq-net --bench track`. // Custom timings measure the real time the work took, not the ambient clock. @@ -107,6 +111,10 @@ const RANGES: [u64; 4] = [1, 16, 256, 4096]; /// Groups each aggregated range spans. const SPAN: [u64; 3] = [1, 64, 4096]; +/// Disjoint ranges one reader is served, and the cached groups each spans. +const SERVED_RANGES: [u64; 3] = [1, 16, 256]; +const SERVED_SPAN: [u64; 3] = [1, 8, 64]; + /// Concurrent publishers sharing one relay-style cache pool. const WRITERS: [usize; 4] = [1, 2, 4, 8]; @@ -811,6 +819,48 @@ fn bench_subscription_ranges(c: &mut Criterion) { group.finish(); } +fn bench_range_serving(c: &mut Criterion) { + let mut group = c.benchmark_group("track_range_serving"); + for span in SERVED_SPAN { + for ranges in SERVED_RANGES { + let broadcast = broadcast::Info::default().produce(); + let track = broadcast.create_track("bench", None).unwrap(); + // A gap between neighbors keeps every range distinct, and its group cached. + for _ in 0..ranges * (span + 1) { + let mut group = track.append_group().unwrap(); + group.write_frame(None, Bytes::from_static(b"x")).unwrap(); + group.finish().unwrap(); + } + let subscription = track::Subscription::default() + .with_max_delay(AGED_BUDGET) + .with_ranges((0..ranges).map(|i| { + let start = i * (span + 1); + track::Range::groups(start..start + span) + })); + + group.throughput(Throughput::Elements(ranges * span)); + group.bench_function(BenchmarkId::new(format!("span_{span}"), ranges), |b| { + b.iter_batched( + || track.subscribe(subscription.clone()), + |mut reader| { + let waiter = kio::Waiter::noop(); + let mut served = 0; + while let Poll::Ready(Ok(Some(group))) = reader.poll_recv_group(&waiter) { + black_box(group); + served += 1; + } + assert_eq!(served, ranges * span); + reader + }, + BatchSize::SmallInput, + ); + }); + drop((track, broadcast)); + } + } + group.finish(); +} + /// Late joins scan a bounded datagram ring; sweep both the backlog and the joining audience. fn bench_datagram_subscriber_join(c: &mut Criterion) { let mut group = c.benchmark_group("track_datagram_subscriber_join"); @@ -1213,6 +1263,7 @@ criterion_group!( bench_subscriber_join, bench_range_demand, bench_subscription_ranges, + bench_range_serving, bench_subscriber_preferences, bench_datagram_subscriber_join, bench_parked_read, diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index c958c4b764..04a9bbe886 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -397,6 +397,15 @@ impl Subscriber { } } + /// The floor the first copy answered with; see [`track::Subscriber::floor`]. Later + /// copies start where the reader got to, which names no floor of the reader's own. + pub(super) fn floor(&self) -> Option { + self.reader().copies.iter().find_map(|copy| match ©.sub { + Sub::Ready(sub) if copy.floor.is_none() => Some(sub.floor()), + _ => None, + })? + } + /// Poll for sequences the serving copy gave up; see [`track::Subscriber::poll_drop`]. #[cfg_attr(not(feature = "async"), allow(dead_code))] pub(super) fn poll_drop(&mut self, waiter: &kio::Waiter) -> Poll> { diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index a7f8c6adc8..2cf19e0ce1 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -3,6 +3,11 @@ use std::{ time::Duration, }; +use crate::Timestamp; + +mod window; +pub(super) use window::{Containing, Holdings, Window}; + /// Subscriber-side preferences for receiving a track. /// /// Each subscriber holds its own [`Subscription`]; the publisher observes an @@ -70,11 +75,10 @@ pub struct Subscription { /// for nothing. /// /// A request to the publisher, aggregated across every live subscriber as the union of - /// their ranges, so it says what the publisher sends, not what any one subscriber sees. - /// The one place it touches this subscriber's reads is where its cursor starts: at the - /// lowest group any of its ranges names. [`crate::track::Subscriber::set_groups`] is the - /// local read filter. See [Local cursor vs wire - /// preference](crate::track::Subscriber#local-cursor-vs-wire-preference). + /// their ranges, and this subscriber's read filter: it reads only the groups its own + /// ranges cover, whatever another subscriber widens the union to. + /// [`crate::track::Subscriber::set_groups`] narrows the reads further. See [Local cursor + /// vs wire preference](crate::track::Subscriber#local-cursor-vs-wire-preference). pub ranges: Vec, /// Which pending group the publisher sends first; see [`Order`]. /// @@ -140,28 +144,37 @@ pub enum Order { Descending, } -/// Where one end of a [`Range`] falls: a fixed position, or a count of groups back from the -/// latest one. +/// Where one end of a [`Range`] falls: a fixed position, a count of groups back from the +/// latest one, or a presentation time. /// -/// A relative anchor resolves against the latest group of the hop that serves it, once, -/// when it starts serving. Before any group exists it resolves at the first one, so -/// [`Range::live`] starts there. +/// A relative or time anchor resolves at the hop that serves it, when it starts serving +/// and again whenever the subscriber replaces its ranges. Before any group exists a +/// relative anchor resolves at the first one, so [`Range::live`] starts there. /// -/// No published wire carries one yet: upstream, a relative start is no floor, the same -/// request as [`Range::oldest`], and a relative end is no end. Behind a moq-transport upstream, -/// which joins without a floor at the live group, `Relative(n)..` starts there. +/// No published wire carries either yet: upstream, a relative or time start is no floor, +/// the same request as [`Range::oldest`], and a relative or time end is no end. Behind a +/// moq-transport upstream, which joins without a floor at the live group, `Relative(n)..` +/// starts there. #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)] pub enum Anchor { /// This exact position. Absolute(Position), /// The group this many before the latest, clamped to group 0: `Relative(0)` is the latest group. Relative(u64), + /// The group containing this presentation time: the newest one starting at or before it. + /// + /// Resolved from the starts of the groups the serving hop holds. When none starts at + /// or before it, a start reaches back to the oldest group, and while no group has + /// started at all it waits like `Relative(0)`. An end stays open until a group + /// starting after it is known. An untimed track refuses it with + /// [`Error::TimestampMismatch`](crate::Error::TimestampMismatch). + Time(Timestamp), } /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is `None`. /// -/// Relative anchors cover whole groups: a relative start is frame 0 of its group, and a -/// relative end includes its group, so `Absolute(0)..Relative(0)` delivers through the latest +/// Relative and time anchors cover whole groups: such a start is frame 0 of its group, and +/// such an end includes its group, so `Absolute(0)..Relative(0)` delivers through the latest /// group. An absolute end is exclusive, like the end of a [`std::ops::Range`]: see /// [`Position::after`] and [`Position::after_group`]. #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)] @@ -208,16 +221,6 @@ impl Range { } } - /// The first position this range delivers, or `None` while it waits on a latest group. - fn resolve_start(&self, latest: Option) -> Option { - match self.start { - Anchor::Absolute(start) => Some(start), - // Nothing sits below group 0, so the farthest offset needs no latest group. - Anchor::Relative(u64::MAX) => Some(Position::default()), - Anchor::Relative(back) => Some(Position::group(latest?.saturating_sub(back))), - } - } - /// Whether this range certainly delivers nothing, wherever the latest group lands. pub(super) fn is_empty(&self) -> bool { match (self.start, self.end) { @@ -279,6 +282,8 @@ pub(super) fn union(ranges: impl IntoIterator) -> Vec { (Anchor::Absolute(_), None | Some(Anchor::Absolute(_))) | (Anchor::Relative(_), None) => false, (Anchor::Absolute(start), _) => tail.is_none_or(|tail| start < tail), (Anchor::Relative(back), _) => farthest.is_none_or(|farthest| back > farthest), + // A time can't be placed against the others without the hop's groups. + (Anchor::Time(_), _) => true, })); union.extend(farthest.map(|back| Range { start: Anchor::Relative(back), @@ -396,8 +401,8 @@ impl Subscription { /// These preferences with every range raised to start no earlier than `floor` and /// stop no later than `end`, the exclusive end. /// - /// A relative anchor becomes the bound itself, since which of the two is tighter - /// depends on a latest group this hop may not know. That asks for no less than the + /// A relative or time anchor becomes the bound itself, since which of the two is tighter + /// depends on groups this hop may not know. That asks for no less than the /// range would. pub(super) fn clamped(&self, floor: Option, end: Option) -> Self { let ranges = self @@ -407,7 +412,7 @@ impl Subscription { start: match (range.start, floor) { (start, None) => start, (Anchor::Absolute(start), Some(floor)) => Anchor::Absolute(start.max(floor)), - (Anchor::Relative(_), Some(floor)) => Anchor::Absolute(floor), + (Anchor::Relative(_) | Anchor::Time(_), Some(floor)) => Anchor::Absolute(floor), }, end: match (range.end, end) { (requested, None) => requested, @@ -419,17 +424,11 @@ impl Subscription { Self { ranges, ..self.clone() } } - /// The lowest group this subscription's ranges start at, resolving relative starts - /// against `latest`. `None` while one waits on a latest group that does not exist yet. - /// - /// Empty ranges ask for nothing, and no ranges at all floor nowhere: group 0. - pub(super) fn floor(&self, latest: Option) -> Option { - let mut floor = None; - for range in self.ranges.iter().filter(|range| !range.is_empty()) { - let start = range.resolve_start(latest)?.group; - floor = Some(floor.map_or(start, |floor: u64| floor.min(start))); - } - Some(floor.unwrap_or(0)) + /// Whether any range ends at a presentation time, which an untimed track refuses. + pub(super) fn has_time(&self) -> bool { + self.ranges.iter().any(|range| { + matches!(range.start, Anchor::Time(_)) || matches!(range.end, Some(Anchor::Time(_))) + }) } } @@ -666,26 +665,40 @@ mod tests { /// group exists only the farthest offset resolves, since it is group 0 regardless. #[test] fn relative_starts_resolve_against_the_latest_group() { + struct Untimed; + impl Holdings for Untimed { + fn timed(&self) -> bool { + false + } + fn containing(&self, _: Timestamp) -> Containing { + Containing::Unknown + } + fn after(&self, _: Timestamp, _: u64) -> Option { + None + } + } + let floor = |sub: &Subscription, latest| Window::new(&sub.ranges, latest, &Untimed).unwrap().floor(); + let live = Subscription::default().with_ranges([Range::live()]); - assert_eq!(live.floor(Some(7)), Some(7)); - assert_eq!(live.floor(None), None); + assert_eq!(floor(&live, Some(7)), Some(7)); + assert_eq!(floor(&live, None), None); let buffering = Subscription::default().with_ranges([range(Anchor::Relative(3), None)]); - assert_eq!(buffering.floor(Some(7)), Some(4)); - assert_eq!(buffering.floor(Some(1)), Some(0), "an offset past the start clamps"); + assert_eq!(floor(&buffering, Some(7)), Some(4)); + assert_eq!(floor(&buffering, Some(1)), Some(0), "an offset past the start clamps"); - assert_eq!(Subscription::default().floor(None), Some(0)); + assert_eq!(floor(&Subscription::default(), None), Some(0)); // The lowest range wins, and one unresolved range leaves the whole floor unresolved. let mixed = Subscription::default().with_ranges([range(absolute(4, 0), None), Range::live()]); - assert_eq!(mixed.floor(Some(3)), Some(3)); - assert_eq!(mixed.floor(Some(9)), Some(4)); - assert_eq!(mixed.floor(None), None); + assert_eq!(floor(&mixed, Some(3)), Some(3)); + assert_eq!(floor(&mixed, Some(9)), Some(4)); + assert_eq!(floor(&mixed, None), None); // Nothing asked for floors nowhere, and an empty range asks for nothing. - assert_eq!(Subscription::default().with_ranges([]).floor(None), Some(0)); + assert_eq!(floor(&Subscription::default().with_ranges([]), None), Some(0)); let empty = Subscription::default().with_ranges([Range::groups(5..5), Range::groups(10..)]); - assert_eq!(empty.floor(None), Some(10)); + assert_eq!(floor(&empty, None), Some(10)); } /// The bounds a wire without ranges carries: the lowest absolute start and the highest diff --git a/rs/moq-net/src/model/subscription/window.rs b/rs/moq-net/src/model/subscription/window.rs new file mode 100644 index 0000000000..a149bce120 --- /dev/null +++ b/rs/moq-net/src/model/subscription/window.rs @@ -0,0 +1,430 @@ +//! A subscriber's own ranges, resolved to the group sequences it reads. +//! +//! The aggregate a publisher sees is the union of every subscriber's ranges, so it says +//! what arrives, not what one subscriber asked for. A [`Window`] is the per-subscriber +//! side: resolved at the hop serving the subscriber, it filters that subscriber's reads. + +use std::ops; + +use rangemap::RangeInclusiveSet; + +use super::{Anchor, Range}; +use crate::{Error, Result, Timestamp}; + +/// When a serving hop's groups start, for resolving time anchors. +pub(in crate::model) trait Holdings { + /// Whether the track carries timestamps at all. + fn timed(&self) -> bool; + + /// Where `pts` falls among the groups with a known start. + fn containing(&self, pts: Timestamp) -> Containing; + + /// The lowest group at or above `from` known to start after `pts`. + fn after(&self, pts: Timestamp, from: u64) -> Option; +} + +/// Where a presentation time falls among the groups a hop holds. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(in crate::model) enum Containing { + /// The newest group starting at or before it. + Group(u64), + /// Every group with a known start begins after it. + Before, + /// No group has a known start. + Unknown, +} + +/// A subscriber's [`Range`]s resolved to the group sequences it reads. +#[derive(Debug)] +pub(in crate::model) struct Window { + /// What these were resolved from, so only replacing the ranges resolves again. + ranges: Vec, + spans: Vec, + /// The groups the spans cover, an unresolved time end counting as open. + groups: RangeInclusiveSet, + /// A relative anchor waits on a first group, so nothing is filtered yet. + pending: bool, + /// Nothing is left to resolve: see [`Self::is_settled`]. Kept so a read checks it once + /// per group, not once per range. + settled: bool, +} + +/// One non-empty range, resolved. +#[derive(Clone, Copy, Debug)] +struct Span { + start: u64, + end: End, + /// Whether the start names a group, rather than reaching back to the oldest one. + floored: bool, +} + +#[derive(Clone, Copy, Debug)] +enum End { + /// Before this group. + Before(u64), + /// Past every group. + Open, + /// Before the first group starting after this time, once one is known. + Time(Timestamp), +} + +impl Window { + /// Resolve `ranges`, counting relative anchors back from `latest`. + /// + /// Refuses a time anchor on an untimed track with [`Error::TimestampMismatch`]. + pub fn new(ranges: &[Range], latest: Option, holdings: &impl Holdings) -> Result { + let mut window = Self { + ranges: ranges.to_vec(), + spans: Vec::new(), + groups: RangeInclusiveSet::new(), + pending: false, + settled: false, + }; + window.place(latest, holdings)?; + Ok(window) + } + + /// Whether nothing is left to resolve as groups arrive. + pub fn is_settled(&self) -> bool { + self.settled + } + + /// Whether a relative anchor still waits on a first group. + pub fn is_pending(&self) -> bool { + self.pending + } + + /// Resolve what the groups now held settle: relative anchors once `latest` exists, and + /// time ends once a group starting after them is known. Returns whether anything moved. + pub fn settle(&mut self, latest: Option, holdings: &impl Holdings) -> bool { + if self.pending { + if latest.is_none() { + return false; + } + // Checked when the ranges were first resolved, against the same track. + let _ = self.place(latest, holdings); + return !self.pending; + } + let mut moved = false; + for span in &mut self.spans { + if let End::Time(pts) = span.end + && let Some(end) = holdings.after(pts, span.start) + { + span.end = End::Before(end); + moved = true; + } + } + if moved { + self.rebuild(); + } + moved + } + + /// Whether group `sequence` is in range. Everything is while a relative anchor waits. + pub fn contains(&self, sequence: u64) -> bool { + self.pending || self.groups.contains(&sequence) + } + + /// The lowest group in range, `None` while a relative anchor waits. Nothing in range + /// floors nowhere: group 0. + pub fn floor(&self) -> Option { + match self.pending { + true => None, + false => Some(self.groups.iter().next().map_or(0, |run| *run.start())), + } + } + + /// The lowest group the ranges name, `None` while one reaches back to the oldest group + /// or a relative anchor waits on a first group. + pub fn start(&self) -> Option { + if self.pending || self.spans.iter().any(|span| !span.floored) { + return None; + } + self.spans.iter().map(|span| span.start).min() + } + + /// The exclusive end past every group in range, `None` when one stays open. + pub fn end(&self) -> Option { + if self.pending { + return None; + } + match self.groups.iter().next_back() { + Some(run) => run.end().checked_add(1), + None => Some(0), + } + } + + /// The groups in range from `from` on, lowest first. + pub fn runs(&self, from: u64) -> impl Iterator> + '_ { + let all = self.pending.then(|| from..=u64::MAX); + let runs = (!self.pending).then(|| { + self.groups + .overlapping(from..=u64::MAX) + .map(move |run| (*run.start()).max(from)..=*run.end()) + }); + all.into_iter().chain(runs.into_iter().flatten()) + } + + /// The first run of `range` that is in range. + pub fn clip(&self, range: ops::Range) -> Option> { + let last = range.end.checked_sub(1).filter(|last| *last >= range.start)?; + if self.pending { + return Some(range); + } + let run = self.groups.overlapping(range.start..=last).next()?; + let end = match run.end().checked_add(1) { + Some(end) => end.min(range.end), + None => range.end, + }; + Some((*run.start()).max(range.start)..end) + } + + fn place(&mut self, latest: Option, holdings: &impl Holdings) -> Result<()> { + let time = |anchor: Option| matches!(anchor, Some(Anchor::Time(_))); + if !holdings.timed() && self.ranges.iter().any(|range| time(Some(range.start)) || time(range.end)) { + return Err(Error::TimestampMismatch); + } + self.spans.clear(); + self.pending = false; + for range in self.ranges.iter().filter(|range| !range.is_empty()) { + let (Some((start, floored)), Some(end)) = (start(range.start, latest, holdings), end(range.end, latest)) + else { + self.pending = true; + continue; + }; + let end = match end { + End::Time(pts) => holdings.after(pts, start).map_or(end, End::Before), + end => end, + }; + self.spans.push(Span { start, end, floored }); + } + self.rebuild(); + Ok(()) + } + + fn rebuild(&mut self) { + self.settled = !self.pending && !self.spans.iter().any(|span| matches!(span.end, End::Time(_))); + self.groups = RangeInclusiveSet::new(); + for span in &self.spans { + let last = match span.end { + End::Before(end) => match end.checked_sub(1) { + Some(last) if last >= span.start => last, + _ => continue, + }, + End::Open | End::Time(_) => u64::MAX, + }; + self.groups.insert(span.start..=last); + } + } +} + +/// The first group a range starts at, and whether it names that group rather than +/// reaching back to the oldest one. `None` while it waits on a latest group. +fn start(anchor: Anchor, latest: Option, holdings: &impl Holdings) -> Option<(u64, bool)> { + match anchor { + Anchor::Absolute(start) => Some((start.group, true)), + // Nothing sits below group 0, so the farthest offset needs no latest group. + Anchor::Relative(u64::MAX) => Some((0, false)), + Anchor::Relative(back) => Some((latest?.saturating_sub(back), true)), + Anchor::Time(pts) => match holdings.containing(pts) { + Containing::Group(group) => Some((group, true)), + // The group holding it is older than any held, so reach back to the oldest. + Containing::Before => Some((0, false)), + // Nothing has started yet: like the live edge, the latest group. + Containing::Unknown => Some((latest?, true)), + }, + } +} + +/// Where a range stops, or `None` while it waits on a latest group. +fn end(anchor: Option, latest: Option) -> Option { + Some(match anchor { + None => End::Open, + Some(Anchor::Absolute(end)) => end.final_sequence().map_or(End::Open, End::Before), + Some(Anchor::Relative(back)) => latest? + .saturating_sub(back) + .checked_add(1) + .map_or(End::Open, End::Before), + Some(Anchor::Time(pts)) => End::Time(pts), + }) +} + +#[cfg(test)] +mod tests { + use super::super::Position; + use super::*; + + /// Groups by sequence with their start in milliseconds, `None` for one not started. + struct Starts(Vec<(u64, Option)>); + + impl Holdings for Starts { + fn timed(&self) -> bool { + true + } + + fn containing(&self, pts: Timestamp) -> Containing { + let mut stamped = false; + for (sequence, start) in self.0.iter().rev() { + let Some(start) = start else { continue }; + stamped = true; + if Timestamp::from_millis(*start).unwrap() <= pts { + return Containing::Group(*sequence); + } + } + match stamped { + true => Containing::Before, + false => Containing::Unknown, + } + } + + fn after(&self, pts: Timestamp, from: u64) -> Option { + self.0 + .iter() + .filter(|(sequence, _)| *sequence >= from) + .find_map(|(sequence, start)| { + start + .filter(|start| Timestamp::from_millis(*start).unwrap() > pts) + .map(|_| *sequence) + }) + } + } + + struct Untimed; + + impl Holdings for Untimed { + fn timed(&self) -> bool { + false + } + fn containing(&self, _: Timestamp) -> Containing { + Containing::Unknown + } + fn after(&self, _: Timestamp, _: u64) -> Option { + None + } + } + + fn time(millis: u64) -> Anchor { + Anchor::Time(Timestamp::from_millis(millis).unwrap()) + } + + fn range(start: Anchor, end: impl Into>) -> Range { + Range { start, end: end.into() } + } + + fn covered(window: &Window, upto: u64) -> Vec { + (0..upto).filter(|sequence| window.contains(*sequence)).collect() + } + + /// Disjoint ranges filter to exactly their groups; the gap between them is out. + #[test] + fn ranges_cover_only_their_groups() { + let window = Window::new(&[Range::groups(2..4), Range::groups(7..)], Some(9), &Untimed).unwrap(); + assert_eq!(covered(&window, 10), vec![2, 3, 7, 8, 9]); + assert_eq!(window.floor(), Some(2)); + assert_eq!(window.end(), None); + assert_eq!(window.runs(3).collect::>(), vec![3..=3, 7..=u64::MAX]); + assert_eq!(window.clip(0..10), Some(2..4)); + assert_eq!(window.clip(4..10), Some(7..10)); + assert_eq!(window.clip(4..7), None); + + // A mid-group end includes its group, a head-of-group end excludes it. + let window = Window::new( + &[range( + Anchor::Absolute(Position { group: 1, frame: 2 }), + Anchor::Absolute(Position { group: 3, frame: 1 }), + )], + None, + &Untimed, + ) + .unwrap(); + assert_eq!(covered(&window, 6), vec![1, 2, 3]); + assert_eq!(window.end(), Some(4)); + + // Nothing asked for is nothing in range, floored nowhere. + let window = Window::new(&[], None, &Untimed).unwrap(); + assert_eq!(covered(&window, 4), Vec::::new()); + assert_eq!((window.floor(), window.end()), (Some(0), Some(0))); + } + + /// Relative anchors count back from the latest group, both ends whole groups, and wait + /// for a first group when there is none. + #[test] + fn relative_anchors_count_back_from_the_latest_group() { + let window = Window::new(&[range(Anchor::Relative(3), Anchor::Relative(1))], Some(7), &Untimed).unwrap(); + assert_eq!(covered(&window, 10), vec![4, 5, 6]); + + let mut window = Window::new(&[Range::live()], None, &Untimed).unwrap(); + assert!(window.is_pending() && window.contains(3) && window.floor().is_none()); + assert!(!window.settle(None, &Untimed)); + assert!(window.settle(Some(5), &Untimed)); + assert_eq!(covered(&window, 8), vec![5, 6, 7]); + assert!(window.is_settled()); + + // The farthest offset needs no latest group, and names no floor. + let window = Window::new(&[Range::oldest()], None, &Untimed).unwrap(); + assert_eq!((window.floor(), window.start()), (Some(0), None)); + } + + /// The named floor is the lowest start, unless a range reaches back to the oldest + /// group or still waits on a first one; an explicit group 0 names one. + #[test] + fn the_named_start_is_the_lowest_unless_one_reaches_back() { + let start = |ranges: &[Range], latest| Window::new(ranges, latest, &Untimed).unwrap().start(); + assert_eq!(start(&[Range::groups(4..), Range::live()], Some(9)), Some(4)); + assert_eq!(start(&[Range::groups(4..), Range::live()], Some(2)), Some(2)); + assert_eq!(start(&[Range::groups(0..)], None), Some(0)); + assert_eq!(start(&[Range::groups(4..), Range::live()], None), None); + assert_eq!(start(&[Range::groups(4..), Range::oldest()], Some(9)), None); + assert_eq!(start(&[Range::groups(4..), Range::groups(..2)], Some(9)), None); + } + + /// A time start is the group containing it, an end stops just past that group, and an + /// end nothing has started after yet stays open until something does. + #[test] + fn time_anchors_resolve_from_group_starts() { + let starts = Starts(vec![(2, Some(2000)), (3, Some(3000)), (5, Some(5000)), (6, None)]); + let window = Window::new(&[range(time(3500), time(4500))], Some(6), &starts).unwrap(); + assert_eq!(covered(&window, 8), vec![3, 4]); + assert!(window.is_settled()); + + // Group 6 may yet start at 5000 too, so an end there waits on it. + let window = Window::new(&[range(time(3500), time(5000))], Some(6), &starts).unwrap(); + assert!(!window.is_settled()); + assert_eq!(covered(&window, 8), vec![3, 4, 5, 6, 7]); + + // Before every start held: reach back to the oldest. + let window = Window::new(&[range(time(1000), time(2500))], Some(6), &starts).unwrap(); + assert_eq!(covered(&window, 8), vec![0, 1, 2]); + + // Past the newest start: open until a later group is known to start after it. + let mut starts = Starts(vec![(2, Some(2000)), (3, Some(3000))]); + let mut window = Window::new(&[range(time(3500), time(3600))], Some(3), &starts).unwrap(); + assert!(!window.is_settled()); + assert!(window.contains(4)); + starts.0.push((4, None)); + assert!(!window.settle(Some(4), &starts), "an unstarted group proves nothing"); + starts.0.push((5, Some(5000))); + assert!(window.settle(Some(5), &starts)); + assert_eq!(covered(&window, 8), vec![3, 4]); + + // Nothing started yet: a time start waits like the live edge. + let starts = Starts(Vec::new()); + let mut window = Window::new(&[range(time(1000), None)], None, &starts).unwrap(); + assert!(window.is_pending()); + let starts = Starts(vec![(4, None)]); + assert!(window.settle(Some(4), &starts)); + assert_eq!(window.floor(), Some(4)); + } + + #[test] + fn an_untimed_track_refuses_time_anchors() { + assert!(matches!( + Window::new(&[range(time(0), None)], None, &Untimed), + Err(Error::TimestampMismatch) + )); + assert!(matches!( + Window::new(&[range(Anchor::Relative(0), time(0))], None, &Untimed), + Err(Error::TimestampMismatch) + )); + } +} diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 75dfa3981e..766c328e7a 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -21,7 +21,7 @@ use super::ledger::{self, Ledger}; use super::{Datagram, Requests}; use super::Cap; -use super::subscription::NewGroup; +use super::subscription::{Containing, Holdings, NewGroup, Window}; pub use super::subscription::{Anchor, Order, Position, Range, Subscription}; use rangemap::RangeSet; @@ -564,6 +564,15 @@ impl Slot { } } + /// Where the copy readers get starts in presentation time, if it has started. + fn start(&self) -> Option { + let group = self.serving(); + match group.is_aborted() { + true => None, + false => group.timestamp(), + } + } + /// The checked start of whichever copy began at frame 0. fn bound(&self) -> Option { self.copies().find_map(group::Producer::bound) @@ -626,6 +635,49 @@ pub(crate) struct FetchOutcome { accepted: bool, } +/// Whether `time` is strictly after `other`, as instants: exact across scales, unlike +/// [`Timestamp`]'s `Ord`, which breaks a tie between scales. +fn later(time: Timestamp, other: Timestamp) -> bool { + u128::from(time.value()) * u128::from(other.scale().as_u64()) + > u128::from(other.value()) * u128::from(time.scale().as_u64()) +} + +/// Where the groups this hop holds start, for resolving time anchors. Walks down from the +/// newest group, so a time near the live edge costs a few groups whatever the cache holds. +impl Holdings for TrackState { + fn timed(&self) -> bool { + self.info.as_ref().is_none_or(|info| info.timescale.is_some()) + } + + fn containing(&self, pts: Timestamp) -> Containing { + let mut started = false; + for (&sequence, slot) in self.lookup.iter().rev() { + let Some(start) = slot.start() else { continue }; + if !later(start, pts) { + return Containing::Group(sequence); + } + started = true; + } + match started { + true => Containing::Before, + false => Containing::Unknown, + } + } + + fn after(&self, pts: Timestamp, from: u64) -> Option { + let mut after = None; + for (&sequence, slot) in self.lookup.range(from..).rev() { + match slot.start() { + Some(start) if later(start, pts) => after = Some(sequence), + Some(_) => break, + // Not started, so it proves nothing either way. + None => {} + } + } + after + } +} + impl TrackState { /// The starts a first frame of group `sequence` must fall between: the nearest checked /// starts cached on either side, since groups land out of sequence order, and the @@ -810,7 +862,7 @@ impl TrackState { self.shown(sequence); } - /// Find the smallest-sequence cached group satisfying + /// Find the smallest-sequence cached group in `window` satisfying /// `next_sequence <= seq < end_sequence (if set)`. Used by /// [`Subscriber::next_group`] so the range can be widened (or unset) /// after the fact and previously-skipped cached groups become available @@ -823,6 +875,7 @@ impl TrackState { &self, next_sequence: u64, end_sequence: Option, + window: &Window, ) -> Poll>> { // Nothing more can arrive once the track was sealed (dropped with its end // declared) or aborted. Either closed the channel, so parking is not an option: @@ -836,10 +889,14 @@ impl TrackState { }; // If the exclusive end is already at or below where we'd resume, no - // group can ever satisfy this call until the cap rises. Pending (not - // None) so the consumer is parked rather than told the stream is over. - // An empty range (`end == 0`) parks even at the first sequence. - if let Some(cap) = end_sequence + // group can ever satisfy this call until the cap rises or the ranges widen. + // Pending (not None) so the consumer is parked rather than told the stream + // is over. An empty range (`end == 0`) parks even at the first sequence. + let cap = match (end_sequence, window.end()) { + (Some(cap), Some(end)) => Some(cap.min(end)), + (cap, end) => cap.or(end), + }; + if let Some(cap) = cap && cap <= next_sequence { if closed { @@ -848,13 +905,16 @@ impl TrackState { return Poll::Pending; } - let best = self - .lookup - .range(next_sequence.max(self.live_floor.unwrap_or(0))..) + let best = window + .runs(next_sequence.max(self.live_floor.unwrap_or(0))) .filter(|_| self.readable()) - .map(|(_, slot)| slot.serving()) - .take_while(|group| super::subscription::before_end(group.sequence, end_sequence)) - .find(|group| !group.is_aborted()); + .find_map(|run| { + self.lookup + .range(run) + .map(|(_, slot)| slot.serving()) + .take_while(|group| super::subscription::before_end(group.sequence, end_sequence)) + .find(|group| !group.is_aborted()) + }); if let Some(group) = best { // Deliberately no cache refresh here: this is a pure seek, and the caller may @@ -2759,10 +2819,10 @@ impl Producer { /// The info is fixed at creation, so there's nothing to wait for (no /// SUBSCRIBE_OK round trip). Pass `None` for [`Subscription::default`]. /// - /// The read cursor starts at the group the subscription named (its floor), or 0. - /// [`Subscription::max_delay`] is what asks for data: delivery skips everything above - /// the floor that the budget convicts, so the default budget of zero delivers only - /// the latest group and a larger one reaches back over what it can still use. + /// Reads only the groups the subscription's ranges cover. + /// [`Subscription::max_delay`] is what asks for data: delivery skips everything in range + /// that the budget convicts, so the default budget of zero delivers only the latest + /// group and a larger one reaches back over what it can still use. pub fn subscribe(&self, subscription: impl Into>) -> Subscriber { let preferences = subscription.into().unwrap_or_default(); @@ -3548,10 +3608,10 @@ impl Consumer { /// [`Subscriber`] once the track info is available, or the track's abort error (or /// [`Error::Dropped`]) if it is already closed. /// - /// The read cursor starts at the group the subscription named (its floor), or 0. - /// [`Subscription::max_delay`] is what asks for data: delivery skips everything above - /// the floor that the budget convicts, so the default budget of zero delivers only - /// the latest group and a larger one reaches back over what it can still use. + /// Reads only the groups the subscription's ranges cover. + /// [`Subscription::max_delay`] is what asks for data: delivery skips everything in range + /// that the budget convicts, so the default budget of zero delivers only the latest + /// group and a larger one reaches back over what it can still use. pub fn subscribe(&self, subscription: impl Into>) -> kio::Pending { let state = self.resolve(); let subscription = Preferences::register(state.read(), subscription.into().unwrap_or_default()); @@ -3936,6 +3996,7 @@ impl Subscribing { // Wait until the track info is available let info = ready!(self.state.poll(waiter, |state| state.poll_info())) .map_err(|e| e.abort.clone().unwrap_or(Error::Dropped))??; + refuse_untimed(&info, &self.subscription.read())?; let resume = self.state.read().routes.clone(); let inner = match resume { @@ -4359,25 +4420,20 @@ impl kio::Task for Fetching { /// /// # Local cursor vs wire preference /// -/// Group bounds exist at two levels, and setting one does not imply the other: -/// -/// - [`Self::set_groups`] limits **this subscriber's reads**, filtering exactly what -/// this handle returns without changing the publisher's demand. -/// - [`Subscription::ranges`], set via [`Self::update`], are a **request to the -/// publisher**. They're aggregated across every live subscriber (their union), so they -/// say what the publisher should send, not what this subscriber sees. +/// Group bounds exist at two levels: /// -/// They stay separate because their scopes differ: a subscriber can't filter by the -/// aggregate, since another subscriber can widen it, and the publisher can't honor a -/// cursor it's never told about. So setting only the cursor still transfers the skipped -/// groups, and setting only the preference still returns groups another subscriber asked -/// for. Set both to skip them *and* avoid the transfer. +/// - [`Subscription::ranges`], set via [`Self::update`], are both a **request to the +/// publisher** and **this subscriber's read filter**. The publisher sees the union across +/// every live subscriber, but each subscriber reads only the groups its own ranges cover, +/// resolved where it is served: a relative anchor counted back from the latest group (or +/// the first to arrive, before any exists) and a time anchor from the cached groups' +/// starts, again whenever the ranges are replaced. +/// - [`Self::set_groups`] narrows **this subscriber's reads** further, without changing the +/// publisher's demand. /// -/// The one place they meet is where the cursor comes from. A new subscriber's cursor is -/// floored at the lowest group its own ranges start at, a relative start counted back from -/// the latest group (or the first to arrive, before any exists), and its -/// [`Subscription::max_delay`] decides what above the floor is worth delivering. Every later -/// move is the caller's. +/// A group outside either is held rather than skipped past, so widening either one later +/// makes it available again. Within both, [`Subscription::max_delay`] decides what is +/// still worth delivering, measured against the track's live edge. pub struct Subscriber { name: Arc, // The broadcast this track belongs to; see [`Self::broadcast`]. @@ -4549,9 +4605,18 @@ struct Cursor { datagram_index: usize, /// Minimum sequence to return from any `recv` method. Set by `start_at`. min_sequence: u64, - /// Whether the subscription's floor still waits on the track's first group: a range - /// counting back from the latest group cannot resolve before one exists. - floor_pending: bool, + /// This subscriber's own ranges, resolved: only the groups in it are read. + window: Window, + /// The [`Preferences::ranges_revision`] the window was resolved from. + ranges_revision: u64, + /// Bumped whenever [`Self::window`] moves, so the drop scan knows to look again. + window_revision: u64, + /// Why the ranges can't be served here: a time anchor on an untimed track. + refused: Option, + /// The window revision and cap at which no parked group was in reach. A group is only + /// parked out of reach, so the walk waits for either to move rather than running once + /// per hand-out over every range. + reoffer_idle: Option<(u64, Option)>, /// One past the highest sequence returned by `next_group`, or `None` once it returned /// `u64::MAX` and no group can follow. /// Used only by that method to skip late arrivals; does not affect `recv_group`. @@ -4589,6 +4654,13 @@ struct Cursor { drops: Option>, } +/// The sequences a [`DropCursor`] reports from: a cursor's span, narrowed to its window. +struct Within<'a> { + span: ops::Range, + window: &'a Window, + revision: u64, +} + /// What a wire publisher's cursor gave up, found as it changes rather than by rescanning /// the cache on every poll. struct DropCursor { @@ -4597,9 +4669,10 @@ struct DropCursor { cuts: super::aborts::Watch, /// Sequences given up and not reported yet: stale skips, cuts, and upstream drops. pending: RangeSet, - /// The [`TrackState::drops_revision`] and window last folded into `pending`. + /// The [`TrackState::drops_revision`], span, and window revision last folded into + /// `pending`. #[cfg_attr(not(feature = "async"), allow(dead_code))] - scanned: Option<(u64, ops::Range)>, + scanned: Option<(u64, ops::Range, u64)>, /// Every sequence this cursor already reported. #[cfg_attr(not(feature = "async"), allow(dead_code))] reported: RangeSet, @@ -4618,7 +4691,8 @@ impl DropCursor { } /// The lowest unreported run of `within` given up, now marked reported. - fn next(&mut self, state: &TrackState, cut: BTreeSet, within: &ops::Range) -> Option> { + fn next(&mut self, state: &TrackState, cut: BTreeSet, within: &Within<'_>) -> Option> { + let (span, window) = (&within.span, within.window); for sequence in cut { // A cut already reclaimed from the cache is in the track's drops instead. if state.is_cut(sequence) @@ -4629,14 +4703,14 @@ impl DropCursor { } // Rescan only after the drops change or the window moves; out-of-window runs were // skipped and may be owed now. - let scan = (state.drops_revision, within.clone()); + let scan = (state.drops_revision, span.clone(), within.revision); if self.scanned.as_ref() != Some(&scan) { self.scanned = Some(scan); - for dropped in state.drops.overlapping(within) { + for dropped in state.drops.overlapping(span) { self.pending - .insert(dropped.start.max(within.start)..dropped.end.min(within.end)); + .insert(dropped.start.max(span.start)..dropped.end.min(span.end)); } - for (&sequence, slot) in state.lookup.range(within.clone()) { + for (&sequence, slot) in state.lookup.range(span.clone()) { if slot.visible && slot.group.is_cut() && let Some(end) = sequence.checked_add(1) @@ -4647,9 +4721,13 @@ impl DropCursor { } while let Some(range) = self.pending.iter().next().cloned() { self.pending.remove(range.clone()); - let clipped = range.start.max(within.start)..range.end.min(within.end); - if clipped.is_empty() { + // Only what this subscriber asked for is owed: the first run of its ranges in it, + // the rest of the range waiting its turn. + let Some(clipped) = window.clip(range.start.max(span.start)..range.end.min(span.end)) else { continue; + }; + if clipped.end < range.end { + self.pending.insert(clipped.end..range.end); } if let Some(gap) = self.reported.gaps(&clipped).next() { if gap.end < range.end { @@ -4668,10 +4746,14 @@ impl Cursor { // The ranges floor the cursor, and the budget picks where above it to start, timed // or not: delivery skips the cached groups it already judges stale, on whichever // clock convicts them. - let (floor, datagram_index) = { + let (window, ranges_revision, refused, datagram_index) = { let preferences = subscription.read(); + let ranges_revision = subscription.ranges_revision(); let state = state.read(); - let floor = preferences.floor(state.newest_produced); + let (window, refused) = match Window::new(&preferences.ranges, state.newest_produced, &*state) { + Ok(window) => (window, None), + Err(err) => (Window::new(&[], None, &*state).expect("nothing to refuse"), Some(err)), + }; // Arrival order need not be timestamp order. Only skip the initial stale prefix; // subsequent delivery stays in arrival order, including late datagrams. let newest = state.datagrams.iter().filter_map(|datagram| datagram.timestamp).max(); @@ -4687,13 +4769,17 @@ impl Cursor { }) .expect("the newest datagram is within max delay") }); - (floor, state.datagram_offset + start) + (window, ranges_revision, refused, state.datagram_offset + start) }; Self { state, subscription, - min_sequence: floor.unwrap_or(0), - floor_pending: floor.is_none(), + min_sequence: 0, + window, + ranges_revision, + window_revision: 0, + reoffer_idle: None, + refused, index: 0, datagram_index, next_sequence: Some(0), @@ -4709,7 +4795,12 @@ impl Cursor { #[cfg_attr(not(feature = "async"), allow(dead_code))] fn poll_drop(&mut self, waiter: &kio::Waiter) -> Poll> { - let within = self.min_sequence..self.end_sequence.unwrap_or(u64::MAX); + self.sync(); + let within = Within { + span: self.floor()..self.end_sequence.unwrap_or(u64::MAX), + window: &self.window, + revision: self.window_revision, + }; let drops = match &mut self.drops { Some(drops) => drops, None => self.drops.insert(Box::new(DropCursor::new(&self.state.read()))), @@ -4738,19 +4829,66 @@ impl Cursor { } } - /// Raise the floor to where the subscription's ranges start, once a group exists to - /// count a relative start back from. Returns whether the floor is settled. - fn resolve_floor(&mut self) -> bool { - if self.floor_pending { - // The track had no produced group at subscribe, so the newest one then was the - // first to arrive, however many have arrived or left since. - let latest = self.state.read().first_produced; - if let Some(floor) = self.subscription.read().floor(latest) { - self.min_sequence = self.min_sequence.max(floor); - self.floor_pending = false; + /// Bring [`Self::window`] up to date: resolved again once the subscriber replaces its + /// ranges, counting relative anchors back from the latest group then, and settled as + /// the groups it waits on arrive. + fn sync(&mut self) { + let replaced = { + let preferences = self.subscription.read(); + let revision = self.subscription.ranges_revision(); + (revision != self.ranges_revision).then(|| (revision, preferences.ranges.clone())) + }; + let state = self.state.read(); + let moved = match replaced { + Some((revision, ranges)) => { + self.ranges_revision = revision; + match Window::new(&ranges, state.newest_produced, &*state) { + Ok(window) => { + self.window = window; + self.refused = None; + } + Err(err) => self.refused = Some(err), + } + true + } + None if self.window.is_settled() => false, + // The track had no produced group when the ranges resolved, so the newest one + // then was the first to arrive, however many have arrived or left since. + None if self.window.is_pending() => self.window.settle(state.first_produced, &*state), + None => self.window.settle(state.newest_produced, &*state), + }; + drop(state); + if moved { + self.window_revision += 1; + } + } + + /// The lowest group this cursor reads: its own floor, raised to its window's. + fn floor(&self) -> u64 { + self.min_sequence.max(self.window.floor().unwrap_or(0)) + } + + /// Whether group `sequence`, at or above [`Self::floor`], is one this cursor reads now: + /// in its window and below its cap. One that isn't may be later. + fn admits(&self, sequence: u64) -> bool { + super::subscription::before_end(sequence, self.end_sequence) && self.window.contains(sequence) + } + + /// The lowest parked group this cursor reads now. + fn reoffer(&mut self) -> Option { + let at = (self.window_revision, self.end_sequence); + if self.parked.is_empty() || self.reoffer_idle == Some(at) { + return None; + } + for run in self.window.runs(self.floor()) { + if let Some((&sequence, _)) = self.parked.range(run).next() + && super::subscription::before_end(sequence, self.end_sequence) + { + return Some(sequence); } } - !self.floor_pending + self.reoffer_idle = Some(at); + None } // A helper to automatically apply Dropped if the state is closed without an error. @@ -4871,7 +5009,11 @@ impl Cursor { } fn poll_recv_group(&mut self, waiter: &kio::Waiter) -> Poll>> { - self.resolve_floor(); + self.sync(); + if let Some(err) = &self.refused { + return Poll::Ready(Err(err.clone())); + } + let floor = self.floor(); if let Some(aborts) = &self.aborts { // A sequence may name both a live group and its fetched head. Only the // consumer we parked decides whether this notification invalidates it. @@ -4884,7 +5026,7 @@ impl Cursor { // Each overtaken entry is removed once, rather than scanning every entry // again for every delivered group. while let Some((&sequence, _)) = self.parked.first_key_value() - && sequence < self.min_sequence + && sequence < floor { self.parked.pop_first(); aborts.remove(sequence); @@ -4898,10 +5040,10 @@ impl Cursor { let drift = ready!(self.poll_drift(self.end_sequence, waiter))?; loop { - // Re-offer the lowest parked group back inside the cap once it rises, - // ahead of the arrival cursor: it is the oldest thing still owed. - let consumer = match self.parked.keys().next().copied() { - Some(sequence) if super::subscription::before_end(sequence, self.end_sequence) => { + // Re-offer the lowest parked group back in range once the cap rises or the ranges + // widen, ahead of the arrival cursor: it is the oldest thing still owed. + let consumer = match self.reoffer() { + Some(sequence) => { let group = self.parked.remove(&sequence).expect("just looked it up"); self.aborts .as_ref() @@ -4912,8 +5054,9 @@ impl Cursor { group } _ => { + let floor = self.floor(); let Some((producer, found_index)) = - ready!(self.poll(waiter, |state| state.poll_recv_group(self.index, self.min_sequence))?) + ready!(self.poll(waiter, |state| state.poll_recv_group(self.index, floor))?) else { // Parked groups survive a finished track: they become deliverable // again if the cap rises, so the stream isn't over while any are held. @@ -4929,14 +5072,17 @@ impl Cursor { self.index = found_index + 1; // The first group to arrive resolves a relative floor, which may land - // above it. - if self.floor_pending && self.resolve_floor() && consumer.sequence < self.min_sequence { - continue; + // above it, and a later one may settle where a time end falls. + if !self.window.is_settled() { + self.sync(); + if consumer.sequence < self.floor() { + continue; + } } - // Park a group beyond the cap instead of dropping it, and keep scanning - // so an in-range group that arrived behind it still flows. - if !super::subscription::before_end(consumer.sequence, self.end_sequence) { + // Park a group beyond the cap or outside the ranges instead of dropping it, + // and keep scanning so an in-range group that arrived behind it still flows. + if !self.admits(consumer.sequence) { self.park(consumer, waiter); continue; @@ -4974,17 +5120,14 @@ impl Cursor { return Poll::Ready(Ok(None)); }; self.datagram_index = found_index + 1; - self.resolve_floor(); + self.sync(); let sequence = datagram.sequence; let mid_group = self.subscription.read().start().is_some_and(|start| { start.group == sequence && start.frame > datagram.wire.as_ref().and_then(|wire| wire.object_id).unwrap_or(0) }); - if sequence >= self.min_sequence - && super::subscription::before_end(sequence, self.end_sequence) - && !mid_group - { + if sequence >= self.floor() && self.admits(sequence) && !mid_group { return Poll::Ready(Ok(Some(datagram))); } } @@ -4996,14 +5139,19 @@ impl Cursor { let Some(next) = self.next_sequence else { return Poll::Ready(Ok(None)); }; - self.resolve_floor(); - let mut floor = next.max(self.min_sequence); + self.sync(); + if let Some(err) = &self.refused { + return Poll::Ready(Err(err.clone())); + } + let mut floor = next.max(self.floor()); let end = self.end_sequence; // One scan for the whole poll, so walking a backlog off stays linear in its size. let drift = ready!(self.poll_drift(end, waiter))?; let group = loop { - let Some(producer) = ready!(self.poll(waiter, |state| state.poll_next_in_range(floor, end))?) else { + let window = &self.window; + let Some(producer) = ready!(self.poll(waiter, |state| state.poll_next_in_range(floor, end, window))?) + else { // Deliberately no flush of `seek_pending` here: only a delivery commit // may count a conviction. This `None` can be an artifact of a floor // that will lower again. A conviction never committed is dropped @@ -5012,10 +5160,14 @@ impl Cursor { }; let group = producer.consume(); - // The first group to arrive resolves a relative floor, which may land above it. - if self.floor_pending && self.resolve_floor() && group.sequence < self.min_sequence { - floor = floor.max(self.min_sequence); - continue; + // The first group to arrive resolves a relative floor, which may land above it, and + // a later one may settle where a time end falls. + if !self.window.is_settled() { + self.sync(); + if group.sequence < self.floor() || !self.window.contains(group.sequence) { + floor = floor.max(self.floor()).max(group.sequence.saturating_add(1)); + continue; + } } // Skip a group the budget has given up on and keep scanning, so one poll @@ -5069,6 +5221,8 @@ impl Cursor { #[derive(Clone)] pub struct Control { subscription: Preferences, + // Whether the track is timed, so a time anchor on an untimed one is refused. + timed: bool, } impl Control { @@ -5080,12 +5234,24 @@ impl Control { /// Replace this subscriber's preferences, updating the producer's aggregate. /// /// Returns [`Error::Closed`] if the track already ended; the update is - /// meaningless at that point and can usually be ignored. + /// meaningless at that point and can usually be ignored. Refuses a time anchor on + /// an untimed track with [`Error::TimestampMismatch`], leaving the preferences as they were. pub fn update(&self, subscription: Subscription) -> Result<()> { + if !self.timed && subscription.has_time() { + return Err(Error::TimestampMismatch); + } self.subscription.update(subscription) } } +/// Refuse `subscription` when it asks an untimed track for a presentation time. +fn refuse_untimed(info: &Info, subscription: &Subscription) -> Result<()> { + match info.timescale.is_none() && subscription.has_time() { + true => Err(Error::TimestampMismatch), + false => Ok(()), + } +} + impl Subscriber { /// The track's [`Info`], resolved when the subscription was established. /// @@ -5130,6 +5296,7 @@ impl Subscriber { Inner::Plain(cursor) => cursor.subscription.clone(), Inner::Resume(resume, _) => resume.subscription().clone(), }, + timed: self.info.timescale.is_some(), } } @@ -5246,10 +5413,11 @@ impl Subscriber { Inner::Resume(resume, _) => return resume.has_unread_group(unread), }; let state = cursor.state.read(); - let floor = cursor.min_sequence.max(state.live_floor.unwrap_or(0)).max(unread.start); + let floor = cursor.floor().max(state.live_floor.unwrap_or(0)).max(unread.start); let eligible = |sequence: u64| { sequence >= floor && super::subscription::before_end(sequence, unread.end) + && cursor.window.contains(sequence) && !unread.delivered.contains(sequence) }; cursor @@ -5377,8 +5545,21 @@ impl Subscriber { Inner::Resume(resume, _) => return resume.poll_start(waiter), }; let start = ready!(Consumer::poll_state_start(&cursor.state, waiter)); - cursor.resolve_floor(); - Poll::Ready(start.map(|start| start.max(cursor.min_sequence))) + cursor.sync(); + Poll::Ready(start.map(|start| start.max(cursor.floor()))) + } + + /// The lowest group this subscriber's ranges name, as resolved here when it last read: + /// `None` while one reaches back to the oldest group still around, or a relative + /// start still waits on the first group to arrive. + /// + /// Where a reader that waits on each group in sequence starts, rather than at whatever + /// arrives first, which newest-first delivery makes the latest. + pub fn floor(&self) -> Option { + match &self.inner { + Inner::Plain(cursor) => cursor.window.start(), + Inner::Resume(resume, _) => resume.floor(), + } } /// Poll for sequences this subscription gave up without serving. @@ -5441,7 +5622,7 @@ impl Subscriber { /// `2..=5` includes groups 2 through 5; `2..5` excludes group 5. An omitted /// start preserves the current floor, and an omitted end removes the cap. /// Groups above the end remain buffered and can be read after raising the cap. - /// This changes only local delivery; use [`Subscription::ranges`] and + /// This narrows only local delivery, within [`Subscription::ranges`]; use those and /// [`Self::update`] to change the requested groups. pub fn set_groups(&mut self, groups: impl RangeBounds) { let (start, end) = super::subscription::sequence_bounds(groups); @@ -5453,7 +5634,7 @@ impl Subscriber { /// /// A local filter, not a request: it doesn't tell the publisher anything, so the /// skipped groups are still delivered and simply not returned. To ask the publisher - /// to start there instead, set [`Subscription::ranges`] via [`Self::update`]. + /// to start there too, set [`Subscription::ranges`] via [`Self::update`]. /// See [Local cursor vs wire preference](Self#local-cursor-vs-wire-preference). #[cfg_attr(not(feature = "async"), allow(dead_code))] // pending: /quest/m1/public-model/serve.md @@ -5508,14 +5689,22 @@ impl Subscriber { /// Replace this subscriber's delivery preferences. /// /// Stored verbatim; the publisher's max age window is applied to the aggregate, not - /// here (see [`Producer::subscription`]). Returns [`Error::Closed`] if the track - /// already ended; the update is meaningless at that point and can usually be ignored. + /// here (see [`Producer::subscription`]). Replaced ranges resolve again here, a + /// relative anchor counting back from the latest group now. Returns [`Error::Closed`] + /// if the track already ended; the update is meaningless at that point and can usually + /// be ignored. Refuses a time anchor on an untimed track with + /// [`Error::TimestampMismatch`], leaving the preferences as they were. pub fn update(&mut self, subscription: Subscription) -> Result<()> { + refuse_untimed(&self.info, &subscription)?; let channel = match &self.inner { Inner::Plain(cursor) => &cursor.subscription, Inner::Resume(resume, _) => resume.subscription(), }; - channel.update(subscription) + channel.update(subscription)?; + if let Inner::Plain(cursor) = &mut self.inner { + cursor.sync(); + } + Ok(()) } /// Poll for the track's largest location: the current one, then each newer one, and `None` once the track ends. @@ -9166,12 +9355,14 @@ mod test { append_at(&mut producer, second * 1000); } - // A range's end is a request to the publisher, folded in with every other - // subscriber's, and it does not filter this handle: the groups above it arrive - // anyway. Capping the live edge with it would pin the edge below them, leaving - // everything past it with nothing newer to be late against. + // The budget caps every range: a range ending below the live edge is judged against + // that edge, not its own end, so with no budget nothing in it is fresh. let mut subscriber = producer.subscribe(Subscription::default().with_end(Position::after_group(1))); - assert_eq!(drain(&mut subscriber), vec![3]); + assert_eq!(drain(&mut subscriber), Vec::::new()); + + // A budget that covers it delivers the range, and nothing above it. + let mut subscriber = producer.subscribe(replay().with_end(Position::after_group(1))); + assert_eq!(drain(&mut subscriber), vec![0, 1]); } #[test] @@ -9183,7 +9374,7 @@ mod test { // Capped at group 0: the route running on past the cap is data this subscriber // can never be served, so it isn't a live edge to be late against. - let mut subscriber = producer.subscribe(Subscription::default().with_end(Position::after_group(0))); + let mut subscriber = producer.subscribe(Subscription::default()); subscriber.set_groups(..1); assert_eq!(drain(&mut subscriber), vec![0]); @@ -10779,7 +10970,7 @@ mod test { fn empty_local_cap_holds_while_another_subscriber_requests_everything() { let producer = track_producer("test", Info::timed()); let mut everything = producer.subscribe(replay()); - let mut empty = producer.subscribe(replay().with_end(Position::group(0))); + let mut empty = producer.subscribe(replay()); empty.set_groups((Bound::Unbounded, Position::group(0).group_end())); for s in 0..3 { @@ -13337,4 +13528,157 @@ mod test { group.write_frame(None, &b"x"[..]).unwrap(); } } + + /// A subscriber reads only its own ranges, whatever another subscriber's widen the + /// aggregate to, in arrival order and in sequence order alike. + #[test] + #[cfg(feature = "async")] + fn a_subscriber_reads_only_its_own_ranges() { + let mut producer = track_producer("test", Info::timed()); + let mut everything = producer.subscribe(replay()); + let ranges = [Range::groups(1..3), Range::groups(5..)]; + let mut mine = producer.subscribe(replay().with_ranges(ranges)); + let mut ordered = producer.subscribe(replay().with_ranges(ranges)).ordered(); + for second in 0..7 { + append_at(&mut producer, second * 1000); + } + assert_eq!(drain(&mut everything), vec![0, 1, 2, 3, 4, 5, 6]); + assert_eq!(drain(&mut mine), vec![1, 2, 5, 6]); + assert_eq!(drain_ordered(&mut ordered), vec![1, 2, 5, 6]); + } + + /// Replacing the ranges re-offers what the old ones held back, and a relative anchor + /// counts back from the latest group as of the update, not the subscribe. + #[test] + #[cfg(feature = "async")] + fn replaced_ranges_resolve_again() { + let mut producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(replay().with_groups(0..2)); + for second in 0..4 { + append_at(&mut producer, second * 1000); + } + assert_eq!(drain(&mut subscriber), vec![0, 1]); + + subscriber.update(replay().with_groups(0..)).unwrap(); + assert_eq!(drain(&mut subscriber), vec![2, 3], "held back, now in range"); + + // The latest group is 3 now, so one back is 2, already read: only new groups follow. + let mut late = producer.subscribe(replay().with_groups(..1)); + assert_eq!(drain(&mut late), vec![0]); + late.update(replay().with_ranges([Range { + start: Anchor::Relative(1), + end: None, + }])) + .unwrap(); + assert_eq!(drain(&mut late), vec![2, 3]); + append_at(&mut producer, 4000); + assert_eq!(drain(&mut late), vec![4]); + } + + /// Datagrams outside a subscriber's ranges are dropped like the groups. + #[test] + #[cfg(feature = "async")] + fn datagrams_outside_the_ranges_are_skipped() { + let mut producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(replay().with_ranges([Range::groups(1..2), Range::groups(3..)])); + for millis in 0..5 { + producer + .append_datagram(Timestamp::from_millis(millis).unwrap(), &b"x"[..]) + .unwrap(); + } + let mut sequences = Vec::new(); + while let Some(Ok(Some(datagram))) = subscriber.recv_datagram().now_or_never() { + sequences.push(datagram.sequence); + } + assert_eq!(sequences, vec![1, 3, 4]); + } + + /// Only the sequences a subscriber asked for are named dropped to it. + #[test] + #[cfg(feature = "async")] + fn drops_are_named_only_within_the_ranges() { + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(replay().with_ranges([Range::groups(2..4), Range::groups(6..8)])); + producer.drop_groups(0..10); + let waiter = kio::Waiter::noop(); + let mut dropped = Vec::new(); + while let Poll::Ready(range) = subscriber.poll_drop(&waiter) { + dropped.push(range); + } + assert_eq!(dropped, vec![2..4, 6..8]); + } + + /// A time start is the cached group containing it and a time end stops just past that + /// group; one past every start held stays open until a group starts after it. + #[test] + #[cfg(feature = "async")] + fn time_anchors_resolve_against_the_cache() { + let mut producer = track_producer("test", Info::timed()); + for second in 0..4 { + append_at(&mut producer, second * 1000); + } + let time = |millis| Anchor::Time(Timestamp::from_millis(millis).unwrap()); + + let mut seek = producer.subscribe(replay().with_ranges([Range { + start: time(1500), + end: Some(time(2000)), + }])); + assert_eq!(drain(&mut seek), vec![1, 2]); + + // A start before every group reaches back to the oldest one. + let mut early = producer.subscribe(replay().with_ranges([Range { + start: Anchor::Time(Timestamp::from_micros(0).unwrap()), + end: Some(time(500)), + }])); + assert_eq!(drain(&mut early), vec![0]); + + // Past the newest start: open until a group is known to start after the end. + let mut live = producer.subscribe(replay().with_ranges([Range { + start: time(3000), + end: Some(time(4500)), + }])); + assert_eq!(drain(&mut live), vec![3]); + append_at(&mut producer, 4000); + append_at(&mut producer, 5000); + append_at(&mut producer, 6000); + assert_eq!(drain(&mut live), vec![4]); + } + + /// An untimed track refuses a time anchor wherever the ranges reach it. + #[test] + #[cfg(feature = "async")] + fn an_untimed_track_refuses_time_anchors() { + let producer = untimed_producer(); + let timed = replay().with_ranges([Range { + start: Anchor::Time(Timestamp::from_millis(0).unwrap()), + end: None, + }]); + + let pending = producer.consume().subscribe(timed.clone()); + assert!(matches!( + pending.poll_ok(&kio::Waiter::noop()), + Poll::Ready(Err(Error::TimestampMismatch)) + )); + + let mut subscriber = producer.subscribe(replay()); + assert!(matches!(subscriber.update(timed.clone()), Err(Error::TimestampMismatch))); + assert!(matches!( + subscriber.control().update(timed.clone()), + Err(Error::TimestampMismatch) + )); + assert_eq!(subscriber.subscription(), replay(), "a refused update changes nothing"); + + let mut refused = producer.subscribe(timed); + assert!(matches!( + refused.recv_group().now_or_never(), + Some(Err(Error::TimestampMismatch)) + )); + + // Ranges it can serve lift the refusal. + refused.update(replay()).unwrap(); + let mut group = producer.append_group().unwrap(); + group.finish().unwrap(); + let read = refused.recv_group().now_or_never().expect("the group is cached"); + assert_eq!(read.unwrap().map(|group| group.sequence), Some(0)); + } } diff --git a/rs/moq-net/src/model/track/demand.rs b/rs/moq-net/src/model/track/demand.rs index 071b50838d..db85879e00 100644 --- a/rs/moq-net/src/model/track/demand.rs +++ b/rs/moq-net/src/model/track/demand.rs @@ -1,5 +1,13 @@ //! Incremental demand, separate from the track's cached content. -use std::{collections::BTreeMap, sync::Arc, task::Poll, time::Duration}; +use std::{ + collections::BTreeMap, + sync::{ + Arc, + atomic::{self, AtomicU64}, + }, + task::Poll, + time::Duration, +}; use super::{NewGroup, Order, Subscription, TrackState, clamp_combined}; @@ -138,6 +146,8 @@ pub(in crate::model) struct Preferences(Arc); struct Preference { value: kio::Producer, subscriptions: Option>, + /// Bumped under the write whenever the ranges change. + ranges_revision: AtomicU64, } impl Preferences { @@ -151,6 +161,7 @@ impl Preferences { Self(Arc::new(Preference { value: kio::Producer::new(value), subscriptions, + ranges_revision: AtomicU64::new(0), })) } @@ -186,6 +197,9 @@ impl Preferences { .subscriptions .as_ref() .map(|subs| subs.lock().replace(Some(&value), Some(&next))); + if value.ranges != next.ranges { + self.0.ranges_revision.fetch_add(1, atomic::Ordering::Relaxed); + } *value = next; drop(value); if let Some(wakes) = &mut wakes { @@ -202,6 +216,11 @@ impl Preferences { pub(in crate::model) fn read(&self) -> kio::Ref<'_, Subscription> { self.0.value.read() } + /// Counts changes to the ranges, so a reader can tell them apart without comparing every + /// range. Read it while holding [`Self::read`] to get the count matching those ranges. + pub(in crate::model) fn ranges_revision(&self) -> u64 { + self.0.ranges_revision.load(atomic::Ordering::Relaxed) + } pub(in crate::model) fn consume(&self) -> kio::Consumer { self.0.value.consume() } @@ -232,6 +251,7 @@ impl From> for Preferences { Self(Arc::new(Preference { value, subscriptions: None, + ranges_revision: AtomicU64::new(0), })) } } diff --git a/rs/moq-net/src/model/track/demand/ranges.rs b/rs/moq-net/src/model/track/demand/ranges.rs index 120723920d..88bae86d29 100644 --- a/rs/moq-net/src/model/track/demand/ranges.rs +++ b/rs/moq-net/src/model/track/demand/ranges.rs @@ -4,7 +4,8 @@ //! where it ends, so a position is covered while the running sum stays above zero. A treap //! keeps each subtree's sum and lowest running sum, so a change touches two boundaries and //! asks one range-minimum question, whatever else the audience holds. Ranges anchored to -//! the latest group can't be placed on that line and are counted by value instead. +//! the latest group or to a time can't be placed on that line and are counted by value +//! instead. use std::cell::OnceCell; use std::collections::BTreeMap; @@ -139,6 +140,8 @@ impl Ranges { match range.start { Anchor::Absolute(start) => self.absolute.tail().is_some_and(|tail| tail <= start), Anchor::Relative(back) => self.farthest().is_some_and(|farthest| back <= farthest), + // Where a time falls among the groups is the serving hop's to say. + Anchor::Time(_) => false, } } } diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 9227b0b2d5..42b00baf51 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Ranges filter reads, and anchors can name a time + +A subscription now reads only the groups its own `ranges` cover. +`Subscription.Anchor` gains `.time(timestampUs:)`, so exhaustive switches need a case for it. +`TrackConsumer.update(subscription:)` now `throws`, on a time anchor an untimed track can't serve; call it with `try`. + ### Subscriptions carry group ranges (#5732) `Subscription.groupStart` and `groupEnd` are replaced by `ranges`, a list of `Subscription.Range` whose ends are a `Subscription.Anchor` (`.absolute(group:frame:)` or `.relative(groups:)`), and an optional `order`. diff --git a/swift/Sources/Moq/Track.swift b/swift/Sources/Moq/Track.swift index f5883967f1..440c91d6c9 100644 --- a/swift/Sources/Moq/Track.swift +++ b/swift/Sources/Moq/Track.swift @@ -44,9 +44,10 @@ public final class TrackConsumer: AsyncSequence, Sendable { try ffi.info() } - /// Change this subscriber's delivery preferences. - public func update(subscription: Subscription) { - ffi.update(subscription: subscription) + /// Change this subscriber's delivery preferences. Throws on a time anchor the track + /// can't serve. + public func update(subscription: Subscription) throws { + try ffi.update(subscription: subscription) } /// Cancel all current and future reads. From d68f823cfdce1ff100d5c58fe4d20e623fbbf7e3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 18 Oct 2026 23:51:46 -0700 Subject: [PATCH 03/36] chore(moq-net): box the cursor's range filter, regenerate Dart bindings The resolved window and its bookkeeping live behind one box so every reader that embeds a cursor stays its old size. Co-Authored-By: Claude Opus 5.5 --- dart/moq_ffi/lib/src/moq.dart | 48 ++++++++- rs/moq-ffi/src/test.rs | 40 ++++---- rs/moq-net/src/model/subscription.rs | 6 +- rs/moq-net/src/model/subscription/window.rs | 9 +- rs/moq-net/src/model/track.rs | 106 +++++++++++--------- 5 files changed, 137 insertions(+), 72 deletions(-) diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index 3f990238d9..0762e1aba2 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -4238,6 +4238,12 @@ class FfiConverterMoqAnchor { lifted.value, lifted.bytesRead - subview.offsetInBytes + 4, ); + case 3: + final lifted = TimeMoqAnchor.read(subview); + return LiftRetVal( + lifted.value, + lifted.bytesRead - subview.offsetInBytes + 4, + ); default: throw UniffiInternalError( UniffiInternalError.unexpectedEnumCase, @@ -4347,6 +4353,44 @@ class RelativeMoqAnchor extends MoqAnchor { } } +class TimeMoqAnchor extends MoqAnchor { + final int timestampUs; + TimeMoqAnchor(int this.timestampUs); + TimeMoqAnchor._(int this.timestampUs); + static LiftRetVal read(Uint8List buf) { + int new_offset = buf.offsetInBytes; + final timestampUs_lifted = FfiConverterUInt64.read( + Uint8List.view(buf.buffer, new_offset), + ); + final timestampUs = timestampUs_lifted.value; + new_offset += timestampUs_lifted.bytesRead; + return LiftRetVal(TimeMoqAnchor._(timestampUs), new_offset); + } + + @override + RustBuffer lower() { + final buf = Uint8List(allocationSize()); + write(buf); + return toRustBuffer(buf); + } + + @override + int allocationSize() { + return FfiConverterUInt64.allocationSize(timestampUs) + 4; + } + + @override + int write(Uint8List buf) { + buf.buffer.asByteData(buf.offsetInBytes).setInt32(0, 3); + int new_offset = buf.offsetInBytes + 4; + new_offset += FfiConverterUInt64.write( + timestampUs, + Uint8List.view(buf.buffer, new_offset), + ); + return new_offset; + } +} + enum MoqOrder { ascending, descending } class FfiConverterMoqOrder { @@ -8322,7 +8366,7 @@ class MoqTrackConsumer implements MoqTrackConsumerInterface { FfiConverterMoqSubscription.lower(subscription), status, ); - }, null); + }, moqExceptionErrorHandler); } } @@ -18637,7 +18681,7 @@ void _checkApiChecksums() { if (uniffi_moq_ffi_checksum_method_moqtrackconsumer_recv_group() != 28712) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } - if (uniffi_moq_ffi_checksum_method_moqtrackconsumer_update() != 24851) { + if (uniffi_moq_ffi_checksum_method_moqtrackconsumer_update() != 39083) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_method_moqgroupdemand_is_used() != 55523) { diff --git a/rs/moq-ffi/src/test.rs b/rs/moq-ffi/src/test.rs index ef18837228..0a34eca2bb 100644 --- a/rs/moq-ffi/src/test.rs +++ b/rs/moq-ffi/src/test.rs @@ -911,15 +911,17 @@ async fn raw_track_update_does_not_wait_for_pending_read() { tokio::spawn(async move { consumer.read_frame().await }) }; - consumer.update(MoqSubscription { - priority: 10, - max_delay_us: 25_000, - ranges: Some(vec![MoqRange { - start: MoqAnchor::Absolute { group: 0, frame: 0 }, - end: None, - }]), - order: None, - }).unwrap(); + consumer + .update(MoqSubscription { + priority: 10, + max_delay_us: 25_000, + ranges: Some(vec![MoqRange { + start: MoqAnchor::Absolute { group: 0, frame: 0 }, + end: None, + }]), + order: None, + }) + .unwrap(); let payload = b"updated subscription".to_vec(); track @@ -3903,15 +3905,17 @@ async fn raw_track_update_during_pending_group_still_reads_datagram() { spawn_parked(async move { consumer.next_group().await }).await }; - consumer.update(MoqSubscription { - priority: 10, - max_delay_us: 25_000, - ranges: Some(vec![MoqRange { - start: MoqAnchor::Absolute { group: 0, frame: 0 }, - end: None, - }]), - order: None, - }).unwrap(); + consumer + .update(MoqSubscription { + priority: 10, + max_delay_us: 25_000, + ranges: Some(vec![MoqRange { + start: MoqAnchor::Absolute { group: 0, frame: 0 }, + end: None, + }]), + order: None, + }) + .unwrap(); let sequence = track.append_datagram(datagram_frame(b"after-update")).unwrap(); let datagram = tokio::time::timeout(TIMEOUT, consumer.recv_datagram()) diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index 2cf19e0ce1..692953a992 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -426,9 +426,9 @@ impl Subscription { /// Whether any range ends at a presentation time, which an untimed track refuses. pub(super) fn has_time(&self) -> bool { - self.ranges.iter().any(|range| { - matches!(range.start, Anchor::Time(_)) || matches!(range.end, Some(Anchor::Time(_))) - }) + self.ranges + .iter() + .any(|range| matches!(range.start, Anchor::Time(_)) || matches!(range.end, Some(Anchor::Time(_)))) } } diff --git a/rs/moq-net/src/model/subscription/window.rs b/rs/moq-net/src/model/subscription/window.rs index a149bce120..aa9052323c 100644 --- a/rs/moq-net/src/model/subscription/window.rs +++ b/rs/moq-net/src/model/subscription/window.rs @@ -156,7 +156,7 @@ impl Window { /// The groups in range from `from` on, lowest first. pub fn runs(&self, from: u64) -> impl Iterator> + '_ { - let all = self.pending.then(|| from..=u64::MAX); + let all = self.pending.then_some(from..=u64::MAX); let runs = (!self.pending).then(|| { self.groups .overlapping(from..=u64::MAX) @@ -181,7 +181,12 @@ impl Window { fn place(&mut self, latest: Option, holdings: &impl Holdings) -> Result<()> { let time = |anchor: Option| matches!(anchor, Some(Anchor::Time(_))); - if !holdings.timed() && self.ranges.iter().any(|range| time(Some(range.start)) || time(range.end)) { + if !holdings.timed() + && self + .ranges + .iter() + .any(|range| time(Some(range.start)) || time(range.end)) + { return Err(Error::TimestampMismatch); } self.spans.clear(); diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 766c328e7a..ba0ab98d94 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -21,8 +21,8 @@ use super::ledger::{self, Ledger}; use super::{Datagram, Requests}; use super::Cap; -use super::subscription::{Containing, Holdings, NewGroup, Window}; pub use super::subscription::{Anchor, Order, Position, Range, Subscription}; +use super::subscription::{Containing, Holdings, NewGroup, Window}; use rangemap::RangeSet; mod demand; @@ -4605,18 +4605,8 @@ struct Cursor { datagram_index: usize, /// Minimum sequence to return from any `recv` method. Set by `start_at`. min_sequence: u64, - /// This subscriber's own ranges, resolved: only the groups in it are read. - window: Window, - /// The [`Preferences::ranges_revision`] the window was resolved from. - ranges_revision: u64, - /// Bumped whenever [`Self::window`] moves, so the drop scan knows to look again. - window_revision: u64, - /// Why the ranges can't be served here: a time anchor on an untimed track. - refused: Option, - /// The window revision and cap at which no parked group was in reach. A group is only - /// parked out of reach, so the walk waits for either to move rather than running once - /// per hand-out over every range. - reoffer_idle: Option<(u64, Option)>, + /// This subscriber's own ranges, resolved. Boxed: every reader embeds a cursor. + filter: Box, /// One past the highest sequence returned by `next_group`, or `None` once it returned /// `u64::MAX` and no group can follow. /// Used only by that method to skip late arrivals; does not affect `recv_group`. @@ -4741,12 +4731,28 @@ impl DropCursor { } } +/// A cursor's own ranges, resolved, and what keeps them current. +struct Filter { + /// Only the groups in it are read. + window: Window, + /// The [`Preferences::ranges_revision`] the window was resolved from. + resolved: u64, + /// Bumped whenever the window moves, so the drop scan knows to look again. + revision: u64, + /// Why the ranges can't be served here: a time anchor on an untimed track. + refused: Option, + /// The revision and cap at which no parked group was in reach. A group is only parked + /// out of reach, so the walk waits for either to move rather than running once per + /// hand-out over every range. + reoffer_idle: Option<(u64, Option)>, +} + impl Cursor { fn new(state: kio::Consumer, subscription: Preferences) -> Self { // The ranges floor the cursor, and the budget picks where above it to start, timed // or not: delivery skips the cached groups it already judges stale, on whichever // clock convicts them. - let (window, ranges_revision, refused, datagram_index) = { + let (filter, datagram_index) = { let preferences = subscription.read(); let ranges_revision = subscription.ranges_revision(); let state = state.read(); @@ -4769,17 +4775,20 @@ impl Cursor { }) .expect("the newest datagram is within max delay") }); - (window, ranges_revision, refused, state.datagram_offset + start) + let filter = Filter { + window, + resolved: ranges_revision, + revision: 0, + refused, + reoffer_idle: None, + }; + (filter, state.datagram_offset + start) }; Self { state, subscription, min_sequence: 0, - window, - ranges_revision, - window_revision: 0, - reoffer_idle: None, - refused, + filter: Box::new(filter), index: 0, datagram_index, next_sequence: Some(0), @@ -4798,8 +4807,8 @@ impl Cursor { self.sync(); let within = Within { span: self.floor()..self.end_sequence.unwrap_or(u64::MAX), - window: &self.window, - revision: self.window_revision, + window: &self.filter.window, + revision: self.filter.revision, }; let drops = match &mut self.drops { Some(drops) => drops, @@ -4836,58 +4845,58 @@ impl Cursor { let replaced = { let preferences = self.subscription.read(); let revision = self.subscription.ranges_revision(); - (revision != self.ranges_revision).then(|| (revision, preferences.ranges.clone())) + (revision != self.filter.resolved).then(|| (revision, preferences.ranges.clone())) }; let state = self.state.read(); let moved = match replaced { Some((revision, ranges)) => { - self.ranges_revision = revision; + self.filter.resolved = revision; match Window::new(&ranges, state.newest_produced, &*state) { Ok(window) => { - self.window = window; - self.refused = None; + self.filter.window = window; + self.filter.refused = None; } - Err(err) => self.refused = Some(err), + Err(err) => self.filter.refused = Some(err), } true } - None if self.window.is_settled() => false, + None if self.filter.window.is_settled() => false, // The track had no produced group when the ranges resolved, so the newest one // then was the first to arrive, however many have arrived or left since. - None if self.window.is_pending() => self.window.settle(state.first_produced, &*state), - None => self.window.settle(state.newest_produced, &*state), + None if self.filter.window.is_pending() => self.filter.window.settle(state.first_produced, &*state), + None => self.filter.window.settle(state.newest_produced, &*state), }; drop(state); if moved { - self.window_revision += 1; + self.filter.revision += 1; } } /// The lowest group this cursor reads: its own floor, raised to its window's. fn floor(&self) -> u64 { - self.min_sequence.max(self.window.floor().unwrap_or(0)) + self.min_sequence.max(self.filter.window.floor().unwrap_or(0)) } /// Whether group `sequence`, at or above [`Self::floor`], is one this cursor reads now: /// in its window and below its cap. One that isn't may be later. fn admits(&self, sequence: u64) -> bool { - super::subscription::before_end(sequence, self.end_sequence) && self.window.contains(sequence) + super::subscription::before_end(sequence, self.end_sequence) && self.filter.window.contains(sequence) } /// The lowest parked group this cursor reads now. fn reoffer(&mut self) -> Option { - let at = (self.window_revision, self.end_sequence); - if self.parked.is_empty() || self.reoffer_idle == Some(at) { + let at = (self.filter.revision, self.end_sequence); + if self.parked.is_empty() || self.filter.reoffer_idle == Some(at) { return None; } - for run in self.window.runs(self.floor()) { + for run in self.filter.window.runs(self.floor()) { if let Some((&sequence, _)) = self.parked.range(run).next() && super::subscription::before_end(sequence, self.end_sequence) { return Some(sequence); } } - self.reoffer_idle = Some(at); + self.filter.reoffer_idle = Some(at); None } @@ -5010,7 +5019,7 @@ impl Cursor { fn poll_recv_group(&mut self, waiter: &kio::Waiter) -> Poll>> { self.sync(); - if let Some(err) = &self.refused { + if let Some(err) = &self.filter.refused { return Poll::Ready(Err(err.clone())); } let floor = self.floor(); @@ -5073,7 +5082,7 @@ impl Cursor { // The first group to arrive resolves a relative floor, which may land // above it, and a later one may settle where a time end falls. - if !self.window.is_settled() { + if !self.filter.window.is_settled() { self.sync(); if consumer.sequence < self.floor() { continue; @@ -5140,7 +5149,7 @@ impl Cursor { return Poll::Ready(Ok(None)); }; self.sync(); - if let Some(err) = &self.refused { + if let Some(err) = &self.filter.refused { return Poll::Ready(Err(err.clone())); } let mut floor = next.max(self.floor()); @@ -5149,7 +5158,7 @@ impl Cursor { let drift = ready!(self.poll_drift(end, waiter))?; let group = loop { - let window = &self.window; + let window = &self.filter.window; let Some(producer) = ready!(self.poll(waiter, |state| state.poll_next_in_range(floor, end, window))?) else { // Deliberately no flush of `seek_pending` here: only a delivery commit @@ -5162,9 +5171,9 @@ impl Cursor { // The first group to arrive resolves a relative floor, which may land above it, and // a later one may settle where a time end falls. - if !self.window.is_settled() { + if !self.filter.window.is_settled() { self.sync(); - if group.sequence < self.floor() || !self.window.contains(group.sequence) { + if group.sequence < self.floor() || !self.filter.window.contains(group.sequence) { floor = floor.max(self.floor()).max(group.sequence.saturating_add(1)); continue; } @@ -5417,7 +5426,7 @@ impl Subscriber { let eligible = |sequence: u64| { sequence >= floor && super::subscription::before_end(sequence, unread.end) - && cursor.window.contains(sequence) + && cursor.filter.window.contains(sequence) && !unread.delivered.contains(sequence) }; cursor @@ -5557,7 +5566,7 @@ impl Subscriber { /// arrives first, which newest-first delivery makes the latest. pub fn floor(&self) -> Option { match &self.inner { - Inner::Plain(cursor) => cursor.window.start(), + Inner::Plain(cursor) => cursor.filter.window.start(), Inner::Resume(resume, _) => resume.floor(), } } @@ -13661,7 +13670,10 @@ mod test { )); let mut subscriber = producer.subscribe(replay()); - assert!(matches!(subscriber.update(timed.clone()), Err(Error::TimestampMismatch))); + assert!(matches!( + subscriber.update(timed.clone()), + Err(Error::TimestampMismatch) + )); assert!(matches!( subscriber.control().update(timed.clone()), Err(Error::TimestampMismatch) @@ -13676,7 +13688,7 @@ mod test { // Ranges it can serve lift the refusal. refused.update(replay()).unwrap(); - let mut group = producer.append_group().unwrap(); + let group = producer.append_group().unwrap(); group.finish().unwrap(); let read = refused.recv_group().now_or_never().expect("the group is cached"); assert_eq!(read.unwrap().map(|group| group.sequence), Some(0)); From 81f8a87e3c6e4e8824e6dd246dd1d2b4af5ea63d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sun, 18 Oct 2026 23:57:12 -0700 Subject: [PATCH 04/36] docs: link the MIGRATE entries to #5929 Co-Authored-By: Claude Opus 5.5 --- dart/moq/MIGRATE.md | 2 +- go/wrapper/MIGRATE.md | 2 +- kt/moq/MIGRATE.md | 2 +- py/moq-rs/MIGRATE.md | 2 +- rs/moq-ffi/MIGRATE.md | 2 +- rs/moq-net/MIGRATE.md | 2 +- swift/MIGRATE.md | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/dart/moq/MIGRATE.md b/dart/moq/MIGRATE.md index bae8c41459..6f2865ee07 100644 --- a/dart/moq/MIGRATE.md +++ b/dart/moq/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time +### Ranges filter reads, and anchors can name a time (#5929) A subscription now reads only the groups its own `ranges` cover. `Anchor` gains a time variant, so exhaustive switches need a case for it. diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index 6c75b68765..59a807e1bb 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time +### Ranges filter reads, and anchors can name a time (#5929) A subscription now reads only the groups its own `Ranges` cover. `AnchorTime` names the group containing a presentation time, and type switches over `Anchor` need a case for it. diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index 8710934b05..0b152c1cad 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time +### Ranges filter reads, and anchors can name a time (#5929) A subscription now reads only the groups its own `ranges` cover. `Anchor` gains `AnchorTime`, so exhaustive `when` expressions need a branch for it. diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index ed785df58a..98f5c02a2d 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time +### Ranges filter reads, and anchors can name a time (#5929) A subscription now reads only the groups its own `ranges` cover. `Anchor` gains a `TIME` variant, the group containing a presentation time. diff --git a/rs/moq-ffi/MIGRATE.md b/rs/moq-ffi/MIGRATE.md index 56a0a708a0..2928644260 100644 --- a/rs/moq-ffi/MIGRATE.md +++ b/rs/moq-ffi/MIGRATE.md @@ -5,7 +5,7 @@ These are the UniFFI names; each language wrapper has its own `MIGRATE.md` in it ## Unreleased -### Ranges filter reads, and anchors can name a time +### Ranges filter reads, and anchors can name a time (#5929) A subscription now reads only the groups its own `ranges` cover, rather than starting at the lowest one and taking whatever arrives. `MoqAnchor` gains `Time { timestamp_us }`, the group containing that presentation time, so exhaustive matches need an arm for it. diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index 1aadbbf5f3..e55ba5a50e 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter each subscriber's reads +### Ranges filter each subscriber's reads (#5929) A `track::Subscriber` now reads only the groups its own `Subscription::ranges` cover, resolved again whenever `update` replaces them; before, they only set where its reads started. A reader that relied on groups another subscriber asked for must widen its own ranges. diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 42b00baf51..1609ea9a46 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time +### Ranges filter reads, and anchors can name a time (#5929) A subscription now reads only the groups its own `ranges` cover. `Subscription.Anchor` gains `.time(timestampUs:)`, so exhaustive switches need a case for it. From feb8a72023a51921914677394376210e50305a9e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 09:15:03 -0700 Subject: [PATCH 05/36] feat(moq-c): give the time anchor its own timestamp_us field Also document the extra group a time end may let through at the live edge. Co-Authored-By: Claude Opus 5.5 --- doc/concept/moq-lite.md | 4 +++- rs/moq-c/MIGRATE.md | 5 +++++ rs/moq-c/src/api.rs | 7 ++++--- rs/moq-c/src/test.rs | 6 +++++- rs/moq-net/src/model/subscription.rs | 5 +++-- 5 files changed, 20 insertions(+), 7 deletions(-) diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index eb41810de2..69770ec533 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -224,7 +224,9 @@ subscriber of the same track asked for more. A relay merges its subscribers' ranges into one upstream request, which today's wire versions carry as the single span covering them. A time anchor is resolved from the groups the serving hop holds; upstream it asks for everything max delay allows, and an -untimed track refuses it. +untimed track refuses it. At the live edge, a time end may let through one extra +group that turns out to start past it, since holding each group until its first +frame arrives would cost latency. Order only decides what goes first, never what is skipped, so catching up oldest first needs a max delay that covers the backlog. When a relay serves subscribers of diff --git a/rs/moq-c/MIGRATE.md b/rs/moq-c/MIGRATE.md index 83b105d149..74e5dab5b2 100644 --- a/rs/moq-c/MIGRATE.md +++ b/rs/moq-c/MIGRATE.md @@ -4,6 +4,11 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Ranges filter reads, and anchors can name a time (#5929) + +A raw track consumer now reads only the groups its `ranges` cover, rather than every group from the lowest start to the last absolute end. +`moq_anchor` gains `timestamp_us`, read with the new `MOQ_ANCHOR_TIME` kind, which changes its ABI. + ### OpenH264 is a build feature (#5771) The release archives and the CMake build still carry the software H.264 fallback. diff --git a/rs/moq-c/src/api.rs b/rs/moq-c/src/api.rs index 401caafc78..8c6c0e9c01 100644 --- a/rs/moq-c/src/api.rs +++ b/rs/moq-c/src/api.rs @@ -605,11 +605,12 @@ pub struct moq_anchor { /// `moq_anchor_kind` discriminant. pub kind: u32, /// The group, or with `MOQ_ANCHOR_RELATIVE` how many groups before the latest one - /// (0 is the latest, and an offset past the first group clamps to it), or with - /// `MOQ_ANCHOR_TIME` a presentation time in microseconds. + /// (0 is the latest, and an offset past the first group clamps to it). pub group: u64, /// The frame within `group`, read only with `MOQ_ANCHOR_ABSOLUTE`. pub frame: u64, + /// The presentation time in microseconds, read only with `MOQ_ANCHOR_TIME`. + pub timestamp_us: u64, } /// How a [moq_anchor] names its position. @@ -646,7 +647,7 @@ fn parse_anchor(anchor: &moq_anchor) -> Result { } v if v == moq_anchor_kind::MOQ_ANCHOR_RELATIVE as u32 => moq_net::track::Anchor::Relative(anchor.group), v if v == moq_anchor_kind::MOQ_ANCHOR_TIME as u32 => { - moq_net::track::Anchor::Time(moq_net::Timestamp::from_micros(anchor.group)?) + moq_net::track::Anchor::Time(moq_net::Timestamp::from_micros(anchor.timestamp_us)?) } _ => return Err(Error::InvalidCode), }) diff --git a/rs/moq-c/src/test.rs b/rs/moq-c/src/test.rs index 4ea412b7b0..40febbd3e0 100644 --- a/rs/moq-c/src/test.rs +++ b/rs/moq-c/src/test.rs @@ -1729,11 +1729,13 @@ fn raw_track_subscription_options_and_update() { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 1, frame: 0, + timestamp_us: 0, }, end: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 2, frame: 0, + timestamp_us: 0, }, end_present: true, }]; @@ -1836,13 +1838,15 @@ fn raw_track_time_anchor() { let ranges = [moq_range { start: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_TIME as u32, - group: 30_000, + group: 0, frame: 0, + timestamp_us: 30_000, }, end: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 0, frame: 0, + timestamp_us: 0, }, end_present: false, }]; diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index 692953a992..47fffdc0aa 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -166,8 +166,9 @@ pub enum Anchor { /// Resolved from the starts of the groups the serving hop holds. When none starts at /// or before it, a start reaches back to the oldest group, and while no group has /// started at all it waits like `Relative(0)`. An end stays open until a group - /// starting after it is known. An untimed track refuses it with - /// [`Error::TimestampMismatch`](crate::Error::TimestampMismatch). + /// starting after it is known, so at the live edge a reader may get one extra group, + /// handed out before its first frame showed it starts past the end. An untimed track + /// refuses it with [`Error::TimestampMismatch`](crate::Error::TimestampMismatch). Time(Timestamp), } From 13a2b33526e250dca7dd042de8feb449a5dd5f2c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 09:59:21 -0700 Subject: [PATCH 06/36] fix(moq-c): refuse a raw track update at the call moq_consume_track_update queued updates for the reader task, which dropped a TimestampMismatch, so a time anchor on an untimed track silently kept the old ranges. Once subscribed, updates now go through the track's Control and fail at the call; one made before SUBSCRIBE_OK applies once it arrives and a refusal then ends the consumer. Co-Authored-By: Claude Opus 5.5 --- rs/moq-c/MIGRATE.md | 1 + rs/moq-c/src/api.rs | 5 ++- rs/moq-c/src/consume.rs | 92 +++++++++++++++++++++-------------------- rs/moq-c/src/test.rs | 90 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 142 insertions(+), 46 deletions(-) diff --git a/rs/moq-c/MIGRATE.md b/rs/moq-c/MIGRATE.md index 74e5dab5b2..1373e7de1f 100644 --- a/rs/moq-c/MIGRATE.md +++ b/rs/moq-c/MIGRATE.md @@ -8,6 +8,7 @@ How to update your code for each breaking change, newest first. A raw track consumer now reads only the groups its `ranges` cover, rather than every group from the lowest start to the last absolute end. `moq_anchor` gains `timestamp_us`, read with the new `MOQ_ANCHOR_TIME` kind, which changes its ABI. +`moq_consume_track_update` fails with a negative code when the track refuses an update, such as a time anchor on an untimed track, leaving the subscription as it was. ### OpenH264 is a build feature (#5771) diff --git a/rs/moq-c/src/api.rs b/rs/moq-c/src/api.rs index 8c6c0e9c01..20b9f4825d 100644 --- a/rs/moq-c/src/api.rs +++ b/rs/moq-c/src/api.rs @@ -3687,7 +3687,10 @@ pub unsafe extern "C" fn moq_consume_track( /// /// Pass NULL for `subscription` to reset to moq-net defaults. /// -/// Returns a zero on success, or a negative code on failure. +/// Returns a zero on success, or a negative code on failure, such as a time anchor on an +/// untimed track, which leaves the subscription as it was. An update made before the +/// subscription is established applies once it is, so a refusal then ends the consumer +/// through its callback instead. /// /// # Safety /// - The caller must ensure that subscription is either NULL or a valid pointer to a [moq_subscription] struct, diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index 5638cc2083..ab941b6e6c 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -4,7 +4,7 @@ use std::{ pin::Pin, task::{Poll, ready}, }; -use tokio::sync::{mpsc, oneshot}; +use tokio::sync::oneshot; use crate::ffi::OnStatus; use crate::{ @@ -106,11 +106,19 @@ struct TaskEntry { /// A raw track task also accepts subscription updates while it is running. struct RawTaskEntry { close: Option>, - update: mpsc::UnboundedSender>, + control: RawControl, callback: OnStatus, } -/// Outcome of polling a raw track source alongside its control channels. +/// Where a raw track's subscription updates go. +enum RawControl { + /// Waiting on SUBSCRIBE_OK, holding the latest update to apply once it arrives. + Pending(Option), + /// Subscribed: updates apply at once, so a refusal reaches the caller. + Live(moq_net::track::Control), +} + +/// Outcome of polling a raw track source alongside its close signal. enum RawStep { /// A delivered value: the next group, or the next frame within a group. Item(T), @@ -653,10 +661,9 @@ impl Consume { let name = name.to_string(); let channel = oneshot::channel(); - let (update, updates) = mpsc::unbounded_channel(); let entry = RawTaskEntry { close: Some(channel.0), - update, + control: RawControl::Pending(None), callback: on_frame, }; let id = self.raw_task.insert(Some(entry))?; @@ -664,9 +671,22 @@ impl Consume { // `subscribe` blocks on SUBSCRIBE_OK, so run it inside the task. tokio::spawn(async move { let res = crate::ffi::run(driver.watch(async move { - let mut track = broadcast.track(&name)?.subscribe(subscription.clone()).await?.ordered(); - Self::apply_raw_subscription(&mut track, subscription); - Self::run_raw(on_frame, track, channel.1, updates).await + let track = broadcast.track(&name)?.subscribe(subscription.clone()).await?.ordered(); + let control = track.control(); + Self::update_raw(&control, Self::raw_subscription(subscription))?; + // Go live under the lock, so no later update lands before the pending one. + if let Some(entry) = State::lock() + .consume + .raw_task + .get_mut(id) + .and_then(|entry| entry.as_mut()) + { + let pending = std::mem::replace(&mut entry.control, RawControl::Live(control.clone())); + if let RawControl::Pending(Some(subscription)) = pending { + Self::update_raw(&control, subscription)?; + } + } + Self::run_raw(on_frame, track, channel.1).await })) .await; @@ -681,25 +701,30 @@ impl Consume { Ok(id) } - fn apply_raw_subscription(track: &mut moq_net::track::Ordered, subscription: Option) { + /// A raw reader's preferences, which join at the latest group unless a range names + /// a floor of its own. + fn raw_subscription(subscription: Option) -> moq_net::track::Subscription { let mut subscription = subscription.unwrap_or_default(); - // The subscription reads only its own ranges, resolved where it is served. A raw - // reader without a floor of its own (the oldest range, the default) joins at the - // latest group instead. for range in &mut subscription.ranges { if range.start == moq_net::track::Range::oldest().start { range.start = moq_net::track::Range::live().start; } } - // A closed track makes the update meaningless; the reader already sees the close. - let _ = track.update(subscription); + subscription + } + + fn update_raw(control: &moq_net::track::Control, subscription: moq_net::track::Subscription) -> Result<(), Error> { + match control.update(subscription) { + // A closed track makes the update meaningless; the reader already sees the close. + Ok(()) | Err(moq_net::Error::Closed) => Ok(()), + Err(err) => Err(err.into()), + } } async fn run_raw( callback: OnStatus, mut track: moq_net::track::Ordered, mut close: oneshot::Receiver<()>, - mut updates: mpsc::UnboundedReceiver>, ) -> Result<(), Error> { // Deliver every frame in sequence order, reading all frames within each // group rather than the one-frame-per-group convenience. This is the @@ -711,7 +736,7 @@ impl Consume { loop { let mut group = match moq_net::kio::wait(|waiter| -> Poll, Error>> { - if Self::poll_raw_control(&mut close, &mut updates, &mut track, waiter) { + if Pin::new(&mut close).poll(&mut waiter.context()).is_ready() { return Poll::Ready(Ok(RawStep::Stop)); } match ready!(track.poll_next_group(waiter))? { @@ -728,7 +753,7 @@ impl Consume { loop { let frame = match moq_net::kio::wait(|waiter| -> Poll, Error>> { - if Self::poll_raw_control(&mut close, &mut updates, &mut track, waiter) { + if Pin::new(&mut close).poll(&mut waiter.context()).is_ready() { return Poll::Ready(Ok(RawStep::Stop)); } match ready!(group.poll_read_frame(waiter))? { @@ -752,33 +777,6 @@ impl Consume { } } - /// Poll the close and update channels, applying any subscription updates inline. - /// - /// Returns `true` when the task must stop: either the consumer was closed - /// (`close` fired) or the update channel was dropped. The caller must then - /// unwind rather than poll `close` again, since a completed oneshot panics if - /// re-polled. Borrows `track` only for the duration of the call so the caller - /// can poll a track/group source afterwards. - fn poll_raw_control( - close: &mut oneshot::Receiver<()>, - updates: &mut mpsc::UnboundedReceiver>, - track: &mut moq_net::track::Ordered, - waiter: &moq_net::kio::Waiter, - ) -> bool { - let mut cx = waiter.context(); - if Pin::new(close).poll(&mut cx).is_ready() { - return true; - } - - loop { - match updates.poll_recv(&mut cx) { - Poll::Ready(Some(subscription)) => Self::apply_raw_subscription(track, subscription), - Poll::Ready(None) => return true, - Poll::Pending => return false, - } - } - } - pub fn raw_track_update( &mut self, track: Id, @@ -789,7 +787,11 @@ impl Consume { .get_mut(track) .and_then(|entry| entry.as_mut()) .ok_or(Error::TrackNotFound)?; - entry.update.send(subscription).map_err(|_| Error::TrackNotFound)?; + let subscription = Self::raw_subscription(subscription); + match &mut entry.control { + RawControl::Pending(pending) => *pending = Some(subscription), + RawControl::Live(control) => Self::update_raw(control, subscription)?, + } Ok(()) } diff --git a/rs/moq-c/src/test.rs b/rs/moq-c/src/test.rs index 40febbd3e0..2ad2d5bf1f 100644 --- a/rs/moq-c/src/test.rs +++ b/rs/moq-c/src/test.rs @@ -1891,6 +1891,96 @@ fn raw_track_time_anchor() { assert_eq!(moq_origin_close(origin), 0); } +/// An untimed track refuses a time anchor update at the call, keeping its ranges. The C +/// API only publishes timed tracks, so the track comes from moq-net. +#[test] +fn raw_track_update_refuses_time_on_untimed() { + let origin = id(moq_origin_create()); + let path = b"raw-track-untimed"; + let broadcast = publish_broadcast(origin, path); + + let track_name = b"data"; + let track = State::lock() + .publish + .track( + Id::try_from(broadcast).unwrap(), + "data", + Some(moq_net::track::Info::default().with_timescale(None)), + ) + .unwrap(); + let write = |payload: &'static [u8]| { + State::lock() + .publish + .track_producer(track) + .unwrap() + .write_frame(None, payload) + .unwrap() + }; + + let consume = request_broadcast(origin, path); + let frame_cb = Callback::new(); + let consumer = id(unsafe { + moq_consume_track( + consume, + track_name.as_ptr() as *const c_char, + track_name.len(), + std::ptr::null(), + Some(channel_callback), + frame_cb.ptr, + ) + }); + + // The first frame proves the subscription is established. + write(b"zero"); + let frame_id = id(frame_cb.recv()); + assert_eq!(moq_consume_track_frame_free(frame_id), 0); + + let ranges = [moq_range { + start: moq_anchor { + kind: moq_anchor_kind::MOQ_ANCHOR_TIME as u32, + group: 0, + frame: 0, + timestamp_us: 0, + }, + end: moq_anchor { + kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, + group: 0, + frame: 0, + timestamp_us: 0, + }, + end_present: false, + }]; + let update = moq_subscription { + priority: 3, + max_delay_us: 1_000_000, + ranges: ranges.as_ptr(), + ranges_len: ranges.len(), + order: 0, + }; + assert!(unsafe { moq_consume_track_update(consumer, &update) } < 0); + + // The consumer keeps reading under its old ranges. + write(b"one"); + let frame_id = id(frame_cb.recv()); + let mut frame = moq_frame { + payload: std::ptr::null(), + payload_size: 0, + timestamp_us: 0, + timestamp_present: false, + keyframe: false, + }; + assert_eq!(unsafe { moq_consume_track_frame(frame_id, &mut frame) }, 0); + let received = unsafe { std::slice::from_raw_parts(frame.payload, frame.payload_size) }; + assert_eq!(received, b"one"); + assert_eq!(moq_consume_track_frame_free(frame_id), 0); + + assert_eq!(moq_consume_track_cancel(consumer), 0); + assert_eq!(frame_cb.recv_terminal(), 0); + assert_eq!(moq_consume_close(consume), 0); + assert_eq!(moq_publish_close(broadcast), 0); + assert_eq!(moq_origin_close(origin), 0); +} + #[test] fn json_snapshot_publish_consume() { let origin = id(moq_origin_create()); From 9dd0ab4ed38b6f6246ef99f5f94ea892856c5ada Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 10:30:03 -0700 Subject: [PATCH 07/36] fix(moq-net): resolve time anchors only from frame-0 starts, refuse datagrams too A copy joined mid-group stamps a later frame than the group's start, so it could convince a time end that the group started past it. A refused cursor also read and discarded datagrams instead of returning its refusal. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 52 +++++++++++++++++++++++++++++------ 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index a916d7308b..cba0a84661 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -564,13 +564,12 @@ impl Slot { } } - /// Where the copy readers get starts in presentation time, if it has started. + /// Where the group starts in presentation time, from a live copy holding frame 0. A + /// copy joined mid-group stamps a later frame, which proves nothing about the start. fn start(&self) -> Option { - let group = self.serving(); - match group.is_aborted() { - true => None, - false => group.timestamp(), - } + self.copies() + .filter(|group| group.live_first_frame() == Some(0)) + .find_map(group::Producer::timestamp) } /// The checked start of whichever copy began at frame 0. @@ -5123,6 +5122,11 @@ impl Cursor { /// group past the cap, nothing holds it for a later raise. fn poll_recv_datagram(&mut self, waiter: &kio::Waiter) -> Poll>> { loop { + // Refuse before reading, so a refused subscription consumes no datagram. + self.sync(); + if let Some(err) = &self.filter.refused { + return Poll::Ready(Err(err.clone())); + } let Some((datagram, found_index)) = ready!(self.poll(waiter, |state| state.poll_recv_datagram(self.datagram_index))?) else { @@ -13661,11 +13665,34 @@ mod test { assert_eq!(drain(&mut live), vec![4]); } + /// A group joined mid-group stamps a later frame than its start, so it never proves a + /// time end falls before it. + #[test] + #[cfg(feature = "async")] + fn a_partial_group_proves_no_start() { + let mut producer = track_producer("test", Info::timed()); + append_at(&mut producer, 0); + append_at(&mut producer, 1000); + let mut partial = producer.append_group().unwrap(); + partial.start_at(1).unwrap(); + partial + .write_frame(Timestamp::from_millis(2500).unwrap(), bytes::Bytes::from_static(b"x")) + .unwrap(); + partial.finish().unwrap(); + + let time = |millis| Anchor::Time(Timestamp::from_millis(millis).unwrap()); + let mut seek = producer.subscribe(replay().with_ranges([Range { + start: time(1500), + end: Some(time(2200)), + }])); + assert_eq!(drain(&mut seek), vec![1, 2]); + } + /// An untimed track refuses a time anchor wherever the ranges reach it. #[test] #[cfg(feature = "async")] fn an_untimed_track_refuses_time_anchors() { - let producer = untimed_producer(); + let mut producer = untimed_producer(); let timed = replay().with_ranges([Range { start: Anchor::Time(Timestamp::from_millis(0).unwrap()), end: None, @@ -13693,12 +13720,21 @@ mod test { refused.recv_group().now_or_never(), Some(Err(Error::TimestampMismatch)) )); + producer.append_datagram(None, &b"x"[..]).unwrap(); + assert!(matches!( + refused.recv_datagram().now_or_never(), + Some(Err(Error::TimestampMismatch)) + )); // Ranges it can serve lift the refusal. refused.update(replay()).unwrap(); let group = producer.append_group().unwrap(); group.finish().unwrap(); let read = refused.recv_group().now_or_never().expect("the group is cached"); - assert_eq!(read.unwrap().map(|group| group.sequence), Some(0)); + assert_eq!( + read.unwrap().map(|group| group.sequence), + Some(1), + "the datagram took sequence 0" + ); } } From c0156905b7d51bc318f68f8ef2b1a52d58cc3dc6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 10:54:16 -0700 Subject: [PATCH 08/36] docs: name the time anchor in the Swift and Dart aliases and the C++ guide Co-Authored-By: Claude Opus 5.5 --- cpp/moq/MIGRATE.md | 6 ++++++ dart/moq/lib/src/aliases.dart | 2 +- swift/Sources/Moq/Aliases.swift | 4 ++-- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/cpp/moq/MIGRATE.md b/cpp/moq/MIGRATE.md index 63ce5456a5..7c476fa23d 100644 --- a/cpp/moq/MIGRATE.md +++ b/cpp/moq/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Ranges filter reads, and anchors can name a time (#5929) + +A subscription now reads only the groups its own `ranges` cover. +`Anchor` gains a `Time` case holding `timestamp_us`, the group containing that presentation time, so exhaustive visits need a case for it. +`TrackConsumer::update` now returns a result, failing with `TimestampMismatch` on a time anchor an untimed track can't serve. + ### Subscriptions carry group ranges (#5732) `Subscription::group_start` and `group_end` are replaced by `ranges`, a list of `Range` whose ends are an `Anchor` (an absolute group and frame, or a count of groups before the latest), and an optional `order`. diff --git a/dart/moq/lib/src/aliases.dart b/dart/moq/lib/src/aliases.dart index a72a77f225..ee9d97750c 100644 --- a/dart/moq/lib/src/aliases.dart +++ b/dart/moq/lib/src/aliases.dart @@ -120,7 +120,7 @@ typedef Subscription = MoqSubscription; /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is null. typedef Range = MoqRange; -/// Where one end of a [Range] falls: absolute or relative to the latest group. +/// Where one end of a [Range] falls: absolute, relative to the latest group, or the group containing a time. typedef Anchor = MoqAnchor; /// Which pending group the publisher sends first. diff --git a/swift/Sources/Moq/Aliases.swift b/swift/Sources/Moq/Aliases.swift index 97c8cce74e..9b9fd71b54 100644 --- a/swift/Sources/Moq/Aliases.swift +++ b/swift/Sources/Moq/Aliases.swift @@ -115,8 +115,8 @@ public typealias ProtocolError = MoqFFI.MoqProtocolError extension Subscription { /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is `nil`. public typealias Range = MoqFFI.MoqRange - /// Where one end of a `Range` falls: `.absolute(group:frame:)` or - /// `.relative(groups:)` back from the latest group. + /// Where one end of a `Range` falls: `.absolute(group:frame:)`, `.relative(groups:)` + /// back from the latest group, or `.time(timestampUs:)` for the group containing it. public typealias Anchor = MoqFFI.MoqAnchor /// Which pending group the publisher sends first: `.ascending` or `.descending`. public typealias Order = MoqFFI.MoqOrder From 01cec52b9fe4acb6e10abf6ffd1cf7c0f8c59dad Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 11:05:35 -0700 Subject: [PATCH 09/36] quest(m1): the time anchor moves to exact timestamps with the bindings Co-Authored-By: Claude Opus 5.5 --- quest/m1/binding-timestamps.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/quest/m1/binding-timestamps.md b/quest/m1/binding-timestamps.md index 20403c6446..1bd34d50a5 100644 --- a/quest/m1/binding-timestamps.md +++ b/quest/m1/binding-timestamps.md @@ -16,8 +16,10 @@ Public API: breaking in every binding. Today every binding speaks microseconds: moq-ffi `timestamp_us: Option` (about 160 sites in 9 files: `MoqFrame`, `MoqDatagram`, `MoqMediaFrame`, audio -frames, json/flate producer `update`/`append`, media `flush`), moq-c -`timestamp_us` plus `timestamp_present` pairs (about 140 sites in 6 files), and +frames, json/flate producer `update`/`append`, media `flush`, the +subscription start `MoqAnchor::Time`), moq-c `timestamp_us` plus +`timestamp_present` pairs (about 140 sites in 6 files, plus +`moq_anchor.timestamp_us` read as `MOQ_ANCHOR_TIME`), and each wrapper's native type (Python `timedelta`, Go `*time.Duration`, Swift `UInt64`, Kotlin `Duration`, Dart `int`, C++ `std::optional`, OBS microseconds). Reads floor through `as_micros()`, so any scale that does not From 52d2c4ffdaff9392760f6a1965fc6eddef43c870 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 12:23:51 -0700 Subject: [PATCH 10/36] fix(moq-net): a lowered floor re-offers the cached groups it newly covers The arrival walk passed groups below the floor without parking them, so widening the ranges (or a SUBSCRIBE_UPDATE lowering Group Start) could never serve a cached group that arrived before the ones already read. A relay could not fold a second subscription into one by lowering the start. The cursor now remembers the highest floor its walk applied. When the floor drops below it, the cached groups in between are looked up by sequence and parked, so each update costs only the groups it re-offers, and the subscriber's delivered set keeps any from going out twice. This reverses #4940's no-rewind rule; its two publisher tests now pin the new contract. Co-Authored-By: Claude Opus 5.5 --- doc/concept/moq-lite.md | 16 ++++--- drafts/draft-lcurley-moq-lite.md | 1 + rs/moq-net/src/lite/publisher.rs | 14 +++--- rs/moq-net/src/model/track.rs | 78 +++++++++++++++++++++++++++++++- 4 files changed, 93 insertions(+), 16 deletions(-) diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index 69770ec533..f6fb4a7c59 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -220,13 +220,15 @@ Each subscription carries the knobs that decide behavior under congestion: Ranges are both a request and a filter. A subscriber reads only the groups its own ranges cover, resolved again whenever it replaces them, even when another -subscriber of the same track asked for more. A relay merges its subscribers' -ranges into one upstream request, which today's wire versions carry as the -single span covering them. A time anchor is resolved from the groups the -serving hop holds; upstream it asks for everything max delay allows, and an -untimed track refuses it. At the live edge, a time end may let through one extra -group that turns out to start past it, since holding each group until its first -frame arrives would cost latency. +subscriber of the same track asked for more. Widening them, including lowering +the start, also serves the cached groups they newly cover, never one already +served. A relay merges its subscribers' ranges into one upstream request, which +today's wire versions carry as the single span covering them. A time anchor is +resolved from the groups the serving hop holds; upstream it asks for everything +max delay allows, and an untimed track refuses it. At the live edge, a time end +lets through every group handed out before its first frame showed it starts past +the end, since holding each group until its first frame arrives would cost +latency. Order only decides what goes first, never what is skipped, so catching up oldest first needs a max delay that covers the backlog. When a relay serves subscribers of diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 941072fbbe..0246921a41 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -1187,6 +1187,7 @@ MUST be 0 when `Group End` is 0, since an unbounded subscription has no end grou A subscriber can modify a subscription with a SUBSCRIBE_UPDATE message. A subscriber MAY send multiple SUBSCRIBE_UPDATE messages to update the subscription. The start and end group can be changed in either direction (growing or shrinking). +Lowering `Group Start` covers the groups below the old start that the publisher still holds, not only those that arrive later; a group already sent is not sent again. ~~~ SUBSCRIBE_UPDATE Message { diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 7a524912ca..21981d380d 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -5253,9 +5253,9 @@ mod serve_group_test { drop(track); run.await.unwrap(); } - // Group 2 arrived before the subscription and the cursor has passed it, so it was - // never served either. - assert_eq!(drops(&mut writer, &log, version).await, [(0, 4), (6, 6)]); + // Group 2 arrived before the subscription, but it is cached, so the lowered start + // serves it and the drops name only what is missing around it. + assert_eq!(drops(&mut writer, &log, version).await, [(0, 1), (3, 4), (6, 6)]); } /// A lowered start restarts the age of the gaps it newly asks for, as the subscriber does, @@ -5597,10 +5597,10 @@ mod serve_group_test { assert_eq!(relay.opened(), 1, "the finished group is the live edge"); } - /// Widening the floor never rewinds past a group already served: group 0 - /// arrived before the served groups, so the cursor has moved past it. + /// Widening the floor serves a cached group below the old one, even though it arrived + /// before the groups already served, and serves those only once. #[moq_clock::test] - async fn a_widening_update_does_not_rewind_past_served_groups() { + async fn a_widening_update_serves_a_cached_group_below_the_old_floor() { let mut track = track::Producer::new( Arc::new(broadcast::Info::default()), "test", @@ -5625,7 +5625,7 @@ mod serve_group_test { end_frame: None, }); relay.settle(); - assert_eq!(relay.opened(), 2, "group 0 was passed, not skipped"); + assert_eq!(relay.opened(), 3, "group 0 is cached and now in range"); } /// A SUBSCRIBE_UPDATE landing while the first group waits on the source's start keeps diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 0b36a398f3..7fd29261ee 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4630,6 +4630,9 @@ struct Cursor { /// are parked here instead of dropped). Keyed by sequence so the lowest is /// re-offered first. parked: BTreeMap, + /// The highest floor `recv_group` has applied. Its walk passes the groups below the + /// floor without parking them, so lowering it parks those still cached. + passed: u64, aborts: Option>, /// [`Self::end_sequence`], shared with the groups this cursor hands out so their /// expiry measures drift against the same servable window after a cap moves. @@ -4802,6 +4805,7 @@ impl Cursor { next_sequence: Some(0), end_sequence: None, parked: BTreeMap::new(), + passed: 0, aborts: None, drift_cap: kio::Producer::new(None), stale: stats::Content::default(), @@ -5025,12 +5029,47 @@ impl Cursor { aborts.register(waiter); } + /// Park the cached groups in range from `floor` up to [`Self::passed`], which the walk + /// passed under a higher floor, so they are re-offered like any widened range. + fn backfill(&mut self, floor: u64, waiter: &kio::Waiter) { + let groups: Vec = { + let state = self.state.read(); + // Nothing is read from the cache now, so leave the floor to lower once it is. + if !state.readable() { + return; + } + self.filter + .window + .runs(floor.max(state.live_floor.unwrap_or(0))) + .take_while(|run| *run.start() < self.passed) + .flat_map(|run| state.lookup.range(run).map(|(_, slot)| slot.serving())) + .take_while(|group| group.sequence < self.passed) + .filter(|group| !group.is_aborted()) + .cloned() + .collect() + }; + self.passed = floor; + if groups.is_empty() { + return; + } + // Consumed with the track guard released, as the walk does. + for group in groups { + self.park(group.consume(), waiter); + } + // In reach at the revision `reoffer` last found nothing at, so look again. + self.filter.reoffer_idle = None; + } + fn poll_recv_group(&mut self, waiter: &kio::Waiter) -> Poll>> { self.sync(); if let Some(err) = &self.filter.refused { return Poll::Ready(Err(err.clone())); } let floor = self.floor(); + match floor < self.passed { + true => self.backfill(floor, waiter), + false => self.passed = floor, + } if let Some(aborts) = &self.aborts { // A sequence may name both a live group and its fetched head. Only the // consumer we parked decides whether this notification invalidates it. @@ -5072,6 +5111,7 @@ impl Cursor { } _ => { let floor = self.floor(); + self.passed = self.passed.max(floor); let Some((producer, found_index)) = ready!(self.poll(waiter, |state| state.poll_recv_group(self.index, floor))?) else { @@ -5093,6 +5133,7 @@ impl Cursor { if !self.filter.window.is_settled() { self.sync(); if consumer.sequence < self.floor() { + self.passed = self.passed.max(self.floor()); continue; } } @@ -5457,7 +5498,11 @@ impl Subscriber { .lookup .get(sequence) .is_some_and(|slot| slot.stamp == *stamp && !slot.is_aborted()) - })) + }) || state + // Passed under a higher floor, so the walk re-parks them; see `Cursor::backfill`. + .lookup + .range(floor..cursor.passed.max(floor)) + .any(|(sequence, slot)| eligible(*sequence) && !slot.is_aborted())) } /// Receive the next group in arrival order. @@ -5712,7 +5757,9 @@ impl Subscriber { /// /// Stored verbatim; the publisher's max age window is applied to the aggregate, not /// here (see [`Producer::subscription`]). Replaced ranges resolve again here, a - /// relative anchor counting back from the latest group now. Returns [`Error::Closed`] + /// relative anchor counting back from the latest group now, and widening them offers + /// the cached groups they newly cover, even ones that arrived before the groups + /// already read. Returns [`Error::Closed`] /// if the track already ended; the update is meaningless at that point and can usually /// be ignored. Refuses a time anchor on an untimed track with /// [`Error::TimestampMismatch`], leaving the preferences as they were. @@ -13620,6 +13667,33 @@ mod test { assert_eq!(drain(&mut late), vec![4]); } + /// Lowering the floor re-offers the cached groups the arrival walk passed below it, + /// as widening any range does. + #[test] + #[cfg(feature = "async")] + fn a_lowered_floor_reoffers_what_arrived_below_it() { + let mut producer = track_producer("test", Info::timed()); + for second in 0..6 { + append_at(&mut producer, second * 1000); + } + let mut subscriber = producer.subscribe(replay().with_groups(5..)); + assert_eq!(drain(&mut subscriber), vec![5]); + + subscriber.update(replay().with_groups(2..)).unwrap(); + assert_eq!(drain(&mut subscriber), vec![2, 3, 4], "passed below the floor, now in range"); + subscriber.update(replay().with_groups(0..)).unwrap(); + assert_eq!(drain(&mut subscriber), vec![0, 1]); + + // Raising and lowering it again re-offers only what is cached in between, and + // nothing already served goes out twice. + subscriber.update(replay().with_groups(4..)).unwrap(); + assert_eq!(drain(&mut subscriber), Vec::::new()); + subscriber.update(replay().with_groups(0..)).unwrap(); + assert_eq!(drain(&mut subscriber), Vec::::new()); + append_at(&mut producer, 6000); + assert_eq!(drain(&mut subscriber), vec![6]); + } + /// Datagrams outside a subscriber's ranges are dropped like the groups. #[test] #[cfg(feature = "async")] From e4f31c20bd72bc20a110c2f93c29bea28fafd6ce Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 12:23:51 -0700 Subject: [PATCH 11/36] docs(moq-net): state the real bound on a time end; quest the one-group cap An unresolved time end covers every group handed out before its first frame, not just one. Say so on Anchor::Time and in the concept doc, and track capping it at one speculative group in quest/m1/subscribe-ranges/time-end-cap.md. Co-Authored-By: Claude Opus 5.5 --- quest/m1/subscribe-ranges/README.md | 1 + quest/m1/subscribe-ranges/time-end-cap.md | 34 +++++++++++++++++++++++ rs/moq-net/src/model/subscription.rs | 2 +- 3 files changed, 36 insertions(+), 1 deletion(-) create mode 100644 quest/m1/subscribe-ranges/time-end-cap.md diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 6897d97560..c30143f95d 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -91,6 +91,7 @@ every sequence delivered or dropped. - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft - [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - range FETCH upstream per missing run, and non-blocking FETCH served downstream - [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity +- [Time end cap](/quest/m1/subscribe-ranges/time-end-cap.md) - at most one group past a time end at the live edge ## Related diff --git a/quest/m1/subscribe-ranges/time-end-cap.md b/quest/m1/subscribe-ranges/time-end-cap.md new file mode 100644 index 0000000000..717e1e359f --- /dev/null +++ b/quest/m1/subscribe-ranges/time-end-cap.md @@ -0,0 +1,34 @@ +# [M] One speculative group past a time end + +## Goal + +A subscriber whose range ends at `Time(pts)` receives at most one group past that +end at the live edge, however many groups the publisher opens before their first +frames land. Today an unresolved time end covers every sequence through `u64::MAX`, +so each group handed out before its first frame can leak past the end; the docs +state that real bound (`track::Anchor::Time`, `doc/concept/moq-lite.md`). + +## Plan + +Found by Codex on #5929 and reproduced there: range `time(3000)..time(3500)` with +three groups opened before any frame delivers all three, not one. + +- `Window` caps an unresolved time end at the newest held group known to start at or + before pts, plus the first unstarted group above it, instead of `u64::MAX`. + `settle` recomputes the cap as groups start. +- Raising the cap must wake readers: register on the speculative group's start + (`group::Producer::poll_started`) from both `Cursor::poll_recv_group` and + `poll_next_group`, so a parked group past the cap is re-offered once the group + before it starts, without waiting on the next arrival. +- A held group that never shows a start (a copy joined mid-group) must not pin the + cap: a later group starting at or before pts lifts it past. +- Datagram-only timed tracks: datagrams are not in `lookup`, so a time end on them + never resolves and the cap would stay at the start group, dropping every later + datagram. Resolve datagram starts too, or leave datagram reads uncapped. + +Regression test: the #5929 repro, plus the wake (a parked group re-offered when its +predecessor's first frame lands) under mock time. + +## Related + +- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - where time anchors are resolved for a `Dynamic` handler diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index 47fffdc0aa..b44a1f8b76 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -166,7 +166,7 @@ pub enum Anchor { /// Resolved from the starts of the groups the serving hop holds. When none starts at /// or before it, a start reaches back to the oldest group, and while no group has /// started at all it waits like `Relative(0)`. An end stays open until a group - /// starting after it is known, so at the live edge a reader may get one extra group, + /// starting after it is known, so at the live edge a reader may get every group /// handed out before its first frame showed it starts past the end. An untimed track /// refuses it with [`Error::TimestampMismatch`](crate::Error::TimestampMismatch). Time(Timestamp), From d79f671bfe9f863ecc6c99f421f11f0a2f2245bd Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 12:47:04 -0700 Subject: [PATCH 12/36] style(moq-net): name the passed-floor check in has_unread_group Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 7fd29261ee..558851b4c7 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -5483,12 +5483,19 @@ impl Subscriber { && cursor.filter.window.contains(sequence) && !unread.delivered.contains(sequence) }; + // Passed under a higher floor, so the walk parks them again; see `Cursor::backfill`. + let passed = || { + state + .lookup + .range(floor..cursor.passed.max(floor)) + .any(|(sequence, slot)| eligible(*sequence) && !slot.is_aborted()) + }; cursor .parked .iter() .any(|(sequence, group)| eligible(*sequence) && !group.is_aborted()) || (state.readable() - && state + && (state .arrival .iter() .skip(cursor.index.saturating_sub(state.offset)) @@ -5498,11 +5505,7 @@ impl Subscriber { .lookup .get(sequence) .is_some_and(|slot| slot.stamp == *stamp && !slot.is_aborted()) - }) || state - // Passed under a higher floor, so the walk re-parks them; see `Cursor::backfill`. - .lookup - .range(floor..cursor.passed.max(floor)) - .any(|(sequence, slot)| eligible(*sequence) && !slot.is_aborted())) + }) || passed())) } /// Receive the next group in arrival order. @@ -13680,7 +13683,11 @@ mod test { assert_eq!(drain(&mut subscriber), vec![5]); subscriber.update(replay().with_groups(2..)).unwrap(); - assert_eq!(drain(&mut subscriber), vec![2, 3, 4], "passed below the floor, now in range"); + assert_eq!( + drain(&mut subscriber), + vec![2, 3, 4], + "passed below the floor, now in range" + ); subscriber.update(replay().with_groups(0..)).unwrap(); assert_eq!(drain(&mut subscriber), vec![0, 1]); From 9b680ece993d0089f14847c96e467f8df9165f2f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 13:26:22 -0700 Subject: [PATCH 13/36] docs(moq-net): quest local frame bounds; fix the stale drift cap comment A local track::Subscriber hands out whole groups even when its range starts or ends mid-group. That predates this branch (reproduced on main 824179a3a), and the wire publishers already position each group per subscription, so it is tracked in quest/m1/subscribe-ranges/frame-bounds.md. The drift_cap comment still said ranges don't filter the subscriber; say why the edge stays uncapped by them instead. Co-Authored-By: Claude Opus 5.5 --- quest/m1/subscribe-ranges/README.md | 1 + quest/m1/subscribe-ranges/frame-bounds.md | 37 +++++++++++++++++++++++ rs/moq-net/src/model/track.rs | 10 +++--- 3 files changed, 42 insertions(+), 6 deletions(-) create mode 100644 quest/m1/subscribe-ranges/frame-bounds.md diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index c30143f95d..299fb91510 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -91,6 +91,7 @@ every sequence delivered or dropped. - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft - [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - range FETCH upstream per missing run, and non-blocking FETCH served downstream - [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity +- [Frame bounds](/quest/m1/subscribe-ranges/frame-bounds.md) - a local subscriber reads only the frames its ranges cover - [Time end cap](/quest/m1/subscribe-ranges/time-end-cap.md) - at most one group past a time end at the live edge ## Related diff --git a/quest/m1/subscribe-ranges/frame-bounds.md b/quest/m1/subscribe-ranges/frame-bounds.md new file mode 100644 index 0000000000..23c940af71 --- /dev/null +++ b/quest/m1/subscribe-ranges/frame-bounds.md @@ -0,0 +1,37 @@ +# [M] A local subscriber reads only its frame bounds + +## Goal + +A `track::Subscriber` whose ranges start or end partway through a group reads only the +frames they cover: a start at group 4 frame 5 hands out group 4 from frame 5, and an +end mid-group stops there, whatever another subscriber widened the aggregate to. Today +the model filters whole groups and hands out every cached frame, so local readers +(moq-ffi, moq-mux, in-process apps) see frames they never asked for. + +## Plan + +Found by Codex on #5929 and reproduced on `main` (824179a3a) as well as on that PR: a +subscription starting at frame 5 of group 0 is handed the group from frame 0. #5929's +`Window` collapses each range to group sequences, but the leak predates it: `main`'s +cursor never positioned a delivered group either. Only the datagram path checks the +start frame (`mid_group` in `Cursor::poll_recv_datagram`). + +The wire is not affected: the lite publisher positions each served group per +subscription (`position_group` in `rs/moq-net/src/lite/publisher.rs`, and the same +shape in `js/net/src/lite/publisher.ts`). + +- Keep each range's absolute frame bounds in the `Window` beside its group runs. +- Position every group the cursor hands out (`group::Consumer::start_at`/`end_at`), on + the arrival, ordered, and re-offer paths, using the bounds of the range that covers it. + A group whose head is gone and that was asked for from frame 0 stays a miss, as + `position_group` does today. +- Then let the lite publisher rely on the model's positioning instead of its own, if + that is simpler. +- Check `js/net`'s `track` subscriber for the same shape and mirror the fix. + +Regression test: a start at frame 5 and an end at frame 3 of a cached group, each read +through `recv_group` and `ordered()`. + +## Related + +- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - the per-subscriber ranges this extends to frames diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 558851b4c7..ebac7edcc7 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4637,12 +4637,10 @@ struct Cursor { /// [`Self::end_sequence`], shared with the groups this cursor hands out so their /// expiry measures drift against the same servable window after a cap moves. /// - /// Deliberately not where [`Subscription::ranges`] end. That is a *request to the publisher*, - /// folded in with every other subscriber's, and it does not filter this handle (see - /// [`Subscriber`]): another unbounded subscriber widens the aggregate and the groups - /// arrive here anyway. Capping the drift edge with it would pin the live edge at the - /// requested end while delivery ran past it, and everything above would then have - /// nothing newer to be late against. + /// Deliberately not where [`Subscription::ranges`] end. They filter what this handle + /// reads, but they are filters, not exemptions: max delay caps every range against the + /// track's live edge. Capping the edge at a range's end would judge a range in the past + /// only against itself, so none of it would ever be late however far the track moved on. drift_cap: kio::Producer>, /// Groups the drift budget skipped since the count was last drained. Accumulated /// here rather than metered in place because the stats scope lives on the owning From bb03fca8c80b35e75dfc0f154986d34d3076d560 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 14:12:33 -0700 Subject: [PATCH 14/36] fix(moq-net): a lowered floor re-offers only groups readers can see The backfill and has_unread_group read the cache by sequence, so they also picked up fetched backfill (never replayed in arrival order) and groups a route still withholds until reveal_group. Filter both to shown slots, as the arrival walk does. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 61 +++++++++++++++++++++++++++++++++-- 1 file changed, 59 insertions(+), 2 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index ebac7edcc7..97b86d638b 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -5040,7 +5040,14 @@ impl Cursor { .window .runs(floor.max(state.live_floor.unwrap_or(0))) .take_while(|run| *run.start() < self.passed) - .flat_map(|run| state.lookup.range(run).map(|(_, slot)| slot.serving())) + // Only what the walk could have read: never a fetched or withheld group. + .flat_map(|run| { + state + .lookup + .range(run) + .filter(|(_, slot)| slot.is_shown()) + .map(|(_, slot)| slot.serving()) + }) .take_while(|group| group.sequence < self.passed) .filter(|group| !group.is_aborted()) .cloned() @@ -5486,7 +5493,7 @@ impl Subscriber { state .lookup .range(floor..cursor.passed.max(floor)) - .any(|(sequence, slot)| eligible(*sequence) && !slot.is_aborted()) + .any(|(sequence, slot)| eligible(*sequence) && slot.is_shown() && !slot.is_aborted()) }; cursor .parked @@ -12840,6 +12847,56 @@ mod test { .unwrap(); } + /// Lowering the floor re-offers only what the arrival walk could have read: a fetched + /// group stays out of arrival order, and a withheld one waits until it is revealed. + #[moq_clock::test] + #[cfg(feature = "async")] + async fn a_lowered_floor_reoffers_neither_fetched_nor_withheld_groups() { + let (producer, _pool) = pooled_producer(1 << 40); + let dynamic = producer.dynamic(); + let consumer = producer.consume(); + producer.create_group(5u64.into()).unwrap().finish().unwrap(); + let mut subscriber = producer.subscribe(replay().with_groups(5..)); + assert_eq!(subscriber.assert_group().sequence, 5); + + let pending = consumer.fetch_group(2, None); + let req = dynamic + .requested_group() + .now_or_never() + .expect("should not block") + .unwrap(); + let mut fetched = req.accept(Info::timed()).unwrap(); + fetched + .write_frame(Timestamp::ZERO, bytes::Bytes::from_static(b"backfill")) + .unwrap(); + fetched.finish().unwrap(); + pending.await.unwrap(); + let mut withheld = producer.receive_group(group::Info { sequence: 3 }).unwrap(); + + subscriber.update(replay().with_groups(0..)).unwrap(); + subscriber.assert_no_group(); + + withheld + .write_frame(Timestamp::from_millis(3).unwrap(), bytes::Bytes::from_static(b"x")) + .unwrap(); + producer.reveal_group(&withheld); + assert_eq!(subscriber.assert_group().sequence, 3, "shown once revealed"); + subscriber.assert_no_group(); + + // A cursor floor lowered before the next read counts only shown groups as unread. + let mut lowered = producer.subscribe(replay()); + lowered.start_at(5); + assert_eq!(lowered.assert_group().sequence, 5); + lowered.start_at(0); + let delivered = super::super::delivered::Delivered::default(); + let unread = Unread { + start: 0, + end: Some(3), + delivered: &delivered, + }; + assert!(!lowered.has_unread_group(&unread), "the fetched group 2 is not unread"); + } + /// A fetched (backfill) group is served by sequence but never replayed to /// arrival-order subscribers. #[moq_clock::test] From dee0478a48ff2858e00310e4309904c92b4d057a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 14:12:43 -0700 Subject: [PATCH 15/36] docs(drafts): log the lowered Group Start behavior in the lite-07 changelog Co-Authored-By: Claude Opus 5.5 --- drafts/draft-lcurley-moq-lite.md | 1 + 1 file changed, 1 insertion(+) diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 0246921a41..6244315291 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -1597,6 +1597,7 @@ The `Message Length` describes the payload size on the wire. - SUBSCRIBE_OK `Group` names where delivery starts when the subscription resolves, which need not be the first group sent. It is not a new floor. A subscriber does not wait for a group between the requested floor and it, but a publisher still delivers one that arrives later within Subscriber Max Age. - Corrected the SUBSCRIBE note that offset `Group Start` by 1. Only `Group End` and `Frame End` are offset so 0 can mean absent. `Group Start` is an absolute sequence, and 0 is group 0. - A subscription's range bounds datagrams like groups, and FETCH never returns a datagram. +- A SUBSCRIBE_UPDATE that lowers `Group Start` covers the groups below the old start that the publisher still holds, not only those that arrive later. A group already sent is not sent again. - A publisher SHOULD order a datagram among the subscription's streams as if it were its Group's stream, ahead of that stream when both exist. It SHOULD drop a datagram rather than block when the transport has no room for it, where it previously sent one only if the congestion controller could transmit it immediately. - Assigned 0x3A NOT_FETCHABLE in the stream error table: a FETCH for a group delivered only as a datagram. - The subscriber FINs its Subscribe Stream after settling its tail; graceful session close waits for that FIN or reset. From cee4bc0c3ea573bc242f6752b741c35f70e0df6f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 14:18:31 -0700 Subject: [PATCH 16/36] fix(moq-net): a failover clamp never widens a range A replacement copy clamped every relative or time start to the reader's newest delivered group, so a later disjoint time range (Time(10s), say) was asked from group 0 and its own filter let through everything in between. Capping a replaced copy did the same to a time end. The Window now reports each range resolved, by index. A replacement copy is clamped from where the serving copy resolved the reader's ranges, read before that copy is capped, and a copy recomputing its own wire uses its own resolution, or resolves replaced ranges against its cache now. Only a range not resolved yet still becomes the bound itself. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 89 ++++++++++++++++++++- rs/moq-net/src/model/subscription.rs | 55 +++++++++---- rs/moq-net/src/model/subscription/window.rs | 42 +++++++++- rs/moq-net/src/model/track.rs | 27 ++++++- 4 files changed, 190 insertions(+), 23 deletions(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 1746945c84..e87af7a558 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -24,7 +24,7 @@ use std::task::{Poll, ready}; use crate::{Error, Result, StreamError, group, stats, track}; use super::delivered::Delivered; -use super::subscription::{Cap, Position, Subscription}; +use super::subscription::{Cap, Position, Resolved, Subscription}; /// What the front says about a logical track: the copy to read now, and how it ended. #[derive(Default)] @@ -269,6 +269,11 @@ struct Copy { sub: Sub, /// Where the subscription was asked to start, so demand updates never ask for less. floor: Option, + /// The reader's ranges [`Self::resolved`] are for. + from: Vec, + /// Each of those ranges resolved, by index, so clamping never widens one: as the + /// serving copy had them when this one was subscribed, until this one answers. + resolved: Vec>, /// Replaced by a newer route: nothing past this group, the newest it had then, is /// taken from it. until: Option>, @@ -293,8 +298,19 @@ enum Sub { impl Copy { /// What this copy is asked for: the reader's own preferences, from no earlier than /// where it was asked to start, and through no later than where it was replaced. - fn wire(&self, reader: &Subscription) -> Subscription { - let mut wire = reader.clamped(self.floor, self.end().map(Position::group)); + fn wire(&mut self, reader: &Subscription) -> Subscription { + match &mut self.sub { + // Its own resolution, which capping its end must not move. + Sub::Ready(sub) if reader.ranges == self.from => self.resolved = sub.resolved_ranges(), + // Replaced ranges resolve against this copy's cache now, as any subscriber's do. + Sub::Ready(sub) => { + self.resolved = sub.resolve_ranges(&reader.ranges); + self.from = reader.ranges.clone(); + } + Sub::Pending(_) if reader.ranges == self.from => {} + Sub::Pending(_) => self.resolved.clear(), + } + let mut wire = reader.clamped(self.floor, self.end().map(Position::group), &self.resolved); // A replaced copy only finishes what it had, so a new group from it never arrives. if self.until.is_some() { wire.new_group = None; @@ -537,6 +553,17 @@ impl Reader { if self.copies.last().is_some_and(|copy| copy.generation == generation) { return; } + // Where the serving copy resolved the reader's ranges: the floor raises each start + // there, never lowering a later range to the floor. Read before capping that copy + // below, which replaces its ranges and so resolves them again as of now. + let resolved = match self.copies.last_mut() { + Some(Copy { + sub: Sub::Ready(sub), + from, + .. + }) if *from == self.mirrored.ranges => sub.resolved_ranges(), + _ => Vec::new(), + }; // The replaced copies finish the groups they had when replaced, and nothing newer. for copy in &mut self.copies { if copy.until.is_none() { @@ -547,12 +574,14 @@ impl Reader { let floor = self.newest.map(Position::group); // The wire request carries the floor from the first message: some sessions read // the demand only once. - let wire = self.mirrored.clamped(floor, None); + let wire = self.mirrored.clamped(floor, None, &resolved); self.copies.push(Copy { generation, sub: Sub::Pending(track.subscribe(wire)), track, floor, + from: self.mirrored.ranges.clone(), + resolved, until: None, done: None, datagrams_done: None, @@ -1713,6 +1742,58 @@ mod test { assert_eq!(recv(&mut sub).sequence, 6); } + /// A replacement copy keeps each range where the serving copy resolved it: the floor + /// raises a later time range's start rather than lowering it to the floor, and capping + /// the replaced copy never widens a time end. + #[test] + #[cfg(feature = "async")] + fn a_replacement_keeps_a_later_time_range_disjoint() { + let routes = Producer::new(); + let logical = logical(&routes); + let a = copy(); + routes.serve(a.consume()); + let ranges = [ + track::Range::groups(0..1), + track::Range { + start: track::Anchor::Time(ts(10_000)), + end: Some(track::Anchor::Time(ts(11_000))), + }, + ]; + let subscription = Subscription::default() + .with_max_delay(Duration::from_secs(30)) + .with_ranges(ranges); + let mut sub = logical + .consume() + .subscribe(subscription) + .now_or_never() + .unwrap() + .unwrap(); + for sequence in 0..13 { + let mut group = a.create_group(group::Info { sequence }).unwrap(); + group.write_frame(ts(sequence * 1000), b"x".as_ref()).unwrap(); + group.finish().unwrap(); + } + assert_eq!(recv(&mut sub).sequence, 0); + + let b = copy(); + routes.serve(b.consume()); + // The read subscribes the replacement while the replaced copy still serves. + let mut rest = Vec::new(); + while let Some(Ok(Some(group))) = sub.recv_group().now_or_never() { + rest.push(group.sequence); + } + for sequence in 0..13 { + let mut group = b.create_group(group::Info { sequence }).unwrap(); + group.write_frame(ts(sequence * 1000), b"x".as_ref()).unwrap(); + group.finish().unwrap(); + } + while let Some(Ok(Some(group))) = sub.recv_group().now_or_never() { + rest.push(group.sequence); + } + rest.sort(); + assert_eq!(rest, vec![10, 11]); + } + #[test] fn a_replaced_copy_at_the_last_sequence_has_no_overflow_cap() { for (last, end) in [(Some(u64::MAX), None), (None, Some(Position::default()))] { diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index b44a1f8b76..57ac9a122e 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -6,7 +6,7 @@ use std::{ use crate::Timestamp; mod window; -pub(super) use window::{Containing, Holdings, Window}; +pub(super) use window::{Containing, Holdings, Resolved, Window}; /// Subscriber-side preferences for receiving a track. /// @@ -402,24 +402,47 @@ impl Subscription { /// These preferences with every range raised to start no earlier than `floor` and /// stop no later than `end`, the exclusive end. /// - /// A relative or time anchor becomes the bound itself, since which of the two is tighter - /// depends on groups this hop may not know. That asks for no less than the - /// range would. - pub(super) fn clamped(&self, floor: Option, end: Option) -> Self { + /// `resolved` names where each range resolved, by index, so a relative or time anchor + /// becomes its own group, raised to `floor` or capped at `end`, rather than the bound + /// itself: clamping never widens a range. One not resolved becomes the bound, which + /// asks for no less than the range would. + pub(super) fn clamped( + &self, + floor: Option, + end: Option, + resolved: &[Option], + ) -> Self { + let clamping = floor.is_some() || end.is_some(); let ranges = self .ranges .iter() - .map(|range| Range { - start: match (range.start, floor) { - (start, None) => start, - (Anchor::Absolute(start), Some(floor)) => Anchor::Absolute(start.max(floor)), - (Anchor::Relative(_) | Anchor::Time(_), Some(floor)) => Anchor::Absolute(floor), - }, - end: match (range.end, end) { - (requested, None) => requested, - (Some(Anchor::Absolute(requested)), Some(end)) => Some(Anchor::Absolute(requested.min(end))), - (_, Some(end)) => Some(Anchor::Absolute(end)), - }, + .enumerate() + .map(|(index, range)| { + let resolved = resolved.get(index).copied().flatten().filter(|_| clamping); + let start = match (range.start, resolved) { + (Anchor::Relative(_) | Anchor::Time(_), Some(resolved)) => { + Anchor::Absolute(Position::group(resolved.start)) + } + (start, _) => start, + }; + let requested = match (range.end, resolved.and_then(|resolved| resolved.end)) { + (Some(Anchor::Relative(_) | Anchor::Time(_)), Some(resolved)) => { + Some(Anchor::Absolute(Position::group(resolved))) + } + (requested, _) => requested, + }; + Range { + start: match (start, floor) { + (start, None) => start, + (Anchor::Absolute(start), Some(floor)) => Anchor::Absolute(start.max(floor)), + (Anchor::Relative(_) | Anchor::Time(_), Some(floor)) => Anchor::Absolute(floor), + }, + end: match (requested, end) { + (requested, None) => requested, + (Some(Anchor::Absolute(requested)), Some(end)) => Some(Anchor::Absolute(requested.min(end))), + (_, Some(end)) => Some(Anchor::Absolute(end)), + }, + } }) .collect(); Self { ranges, ..self.clone() } diff --git a/rs/moq-net/src/model/subscription/window.rs b/rs/moq-net/src/model/subscription/window.rs index aa9052323c..4540e42701 100644 --- a/rs/moq-net/src/model/subscription/window.rs +++ b/rs/moq-net/src/model/subscription/window.rs @@ -11,6 +11,14 @@ use rangemap::RangeInclusiveSet; use super::{Anchor, Range}; use crate::{Error, Result, Timestamp}; +/// One range resolved to groups: from `start` up to the exclusive `end`, `None` while it +/// stays open. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(in crate::model) struct Resolved { + pub start: u64, + pub end: Option, +} + /// When a serving hop's groups start, for resolving time anchors. pub(in crate::model) trait Holdings { /// Whether the track carries timestamps at all. @@ -52,6 +60,8 @@ pub(in crate::model) struct Window { /// One non-empty range, resolved. #[derive(Clone, Copy, Debug)] struct Span { + /// Which of the ranges it came from. + index: usize, start: u64, end: End, /// Whether the start names a group, rather than reaching back to the oldest one. @@ -143,6 +153,22 @@ impl Window { self.spans.iter().map(|span| span.start).min() } + /// Each range resolved, in the order given: `None` for one that is empty or still + /// waits on a first group. + pub fn resolved(&self) -> Vec> { + let mut resolved = vec![None; self.ranges.len()]; + for span in &self.spans { + resolved[span.index] = Some(Resolved { + start: span.start, + end: match span.end { + End::Before(end) => Some(end), + End::Open | End::Time(_) => None, + }, + }); + } + resolved + } + /// The exclusive end past every group in range, `None` when one stays open. pub fn end(&self) -> Option { if self.pending { @@ -191,7 +217,7 @@ impl Window { } self.spans.clear(); self.pending = false; - for range in self.ranges.iter().filter(|range| !range.is_empty()) { + for (index, range) in self.ranges.iter().enumerate().filter(|(_, range)| !range.is_empty()) { let (Some((start, floored)), Some(end)) = (start(range.start, latest, holdings), end(range.end, latest)) else { self.pending = true; @@ -201,7 +227,12 @@ impl Window { End::Time(pts) => holdings.after(pts, start).map_or(end, End::Before), end => end, }; - self.spans.push(Span { start, end, floored }); + self.spans.push(Span { + index, + start, + end, + floored, + }); } self.rebuild(); Ok(()) @@ -331,6 +362,13 @@ mod tests { assert_eq!(window.clip(0..10), Some(2..4)); assert_eq!(window.clip(4..10), Some(7..10)); assert_eq!(window.clip(4..7), None); + assert_eq!( + window.resolved(), + vec![ + Some(Resolved { start: 2, end: Some(4) }), + Some(Resolved { start: 7, end: None }), + ] + ); // A mid-group end includes its group, a head-of-group end excludes it. let window = Window::new( diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 97b86d638b..d7e7575dd5 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -22,7 +22,7 @@ use super::{Datagram, Requests}; use super::Cap; pub use super::subscription::{Anchor, Order, Position, Range, Subscription}; -use super::subscription::{Containing, Holdings, NewGroup, Window}; +use super::subscription::{Containing, Holdings, NewGroup, Resolved, Window}; use rangemap::RangeSet; mod demand; @@ -5637,6 +5637,31 @@ impl Subscriber { } } + /// This subscriber's own ranges, by index, as resolved here: `None` for one empty or + /// still waiting, and for every range of a front's reader. + pub(super) fn resolved_ranges(&mut self) -> Vec> { + match &mut self.inner { + Inner::Plain(cursor) => { + cursor.sync(); + cursor.filter.window.resolved() + } + Inner::Resume(..) => Vec::new(), + } + } + + /// `ranges` resolved against this track's cache now, by index. + pub(super) fn resolve_ranges(&self, ranges: &[Range]) -> Vec> { + match &self.inner { + Inner::Plain(cursor) => { + let state = cursor.state.read(); + Window::new(ranges, state.newest_produced, &*state) + .map(|window| window.resolved()) + .unwrap_or_default() + } + Inner::Resume(..) => Vec::new(), + } + } + /// Poll for sequences this subscription gave up without serving. /// /// Stale skips are recorded only once this has been polled, so a cursor no wire From a24bff1e1d55687f45ab5c2c52d627b96ba73463 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 14:59:23 -0700 Subject: [PATCH 17/36] fix(moq-net): an unresolved start never widens to the failover floor A relative or time start a replacement copy cannot resolve yet, such as one updated while the copy is still pending, keeps its anchor instead of becoming the floor, which would read every group from there. A pending copy replaced in turn passes on the resolution it was handed. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 30 ++++++++++++++++++++++++++++ rs/moq-net/src/model/subscription.rs | 11 +++++----- 2 files changed, 35 insertions(+), 6 deletions(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index e0e21667be..4c20b037b7 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -562,6 +562,13 @@ impl Reader { from, .. }) if *from == self.mirrored.ranges => sub.resolved_ranges(), + // One still unanswered passes on what it was handed. + Some(Copy { + sub: Sub::Pending(_), + from, + resolved, + .. + }) if *from == self.mirrored.ranges => resolved.clone(), _ => Vec::new(), }; // The replaced copies finish the groups they had when replaced, and nothing newer. @@ -1952,6 +1959,29 @@ mod test { assert_eq!(rest, vec![10, 11]); } + /// A replacement updated to a time range it cannot resolve yet asks for that range, + /// not everything from the floor: before it answers, and once it answers with nothing + /// to resolve it against. + #[test] + fn a_replacement_keeps_an_updated_time_start() { + let routes = Producer::new(); + let a = copy(); + routes.serve(a.consume()); + let sub = routes.consume().subscribe(kio::Producer::new(Subscription::default()).into()); + let mut reader = sub.reader.lock().unwrap(); + let copy = &mut reader.copies[0]; + copy.floor = Some(Position::group(0)); + let ranges = vec![track::Range { + start: track::Anchor::Time(ts(10_000)), + end: None, + }]; + let updated = Subscription::default().with_ranges(ranges.clone()); + assert!(matches!(copy.sub, Sub::Pending(_))); + assert_eq!(copy.wire(&updated).ranges, ranges); + assert!(copy.poll_ready((0, Cap::from(..)), &kio::Waiter::noop()).is_ready()); + assert_eq!(copy.wire(&updated).ranges, ranges); + } + #[test] fn a_replaced_copy_at_the_last_sequence_has_no_overflow_cap() { for (last, end) in [(Some(u64::MAX), None), (None, Some(Position::default()))] { diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index 57ac9a122e..289328679e 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -400,12 +400,12 @@ impl Subscription { } /// These preferences with every range raised to start no earlier than `floor` and - /// stop no later than `end`, the exclusive end. + /// stop no later than `end`, the exclusive end, without ever widening one. /// /// `resolved` names where each range resolved, by index, so a relative or time anchor - /// becomes its own group, raised to `floor` or capped at `end`, rather than the bound - /// itself: clamping never widens a range. One not resolved becomes the bound, which - /// asks for no less than the range would. + /// becomes its own group, raised to `floor` or capped at `end`. A start not resolved + /// keeps its anchor, since the floor may lie before it; an end not resolved becomes + /// the bound. pub(super) fn clamped( &self, floor: Option, @@ -433,9 +433,8 @@ impl Subscription { }; Range { start: match (start, floor) { - (start, None) => start, (Anchor::Absolute(start), Some(floor)) => Anchor::Absolute(start.max(floor)), - (Anchor::Relative(_) | Anchor::Time(_), Some(floor)) => Anchor::Absolute(floor), + (start, _) => start, }, end: match (requested, end) { (requested, None) => requested, From d788a309123ca36d28e1892bdac25104cead7121 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 15:02:04 -0700 Subject: [PATCH 18/36] style(moq-net): format the pending-copy regression test Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 4c20b037b7..f4e5a17caf 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -1967,7 +1967,9 @@ mod test { let routes = Producer::new(); let a = copy(); routes.serve(a.consume()); - let sub = routes.consume().subscribe(kio::Producer::new(Subscription::default()).into()); + let sub = routes + .consume() + .subscribe(kio::Producer::new(Subscription::default()).into()); let mut reader = sub.reader.lock().unwrap(); let copy = &mut reader.copies[0]; copy.floor = Some(Position::group(0)); From 7060b2673728c014e1f3c7de1220ec871d7e96b1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 15:40:10 -0700 Subject: [PATCH 19/36] fix(moq-net): a replacement copy never reads below where the reader got to A start the wire kept as an anchor can resolve below the copy's floor once it answers, and the reader never delivered the groups it skipped there, so the delivered set did not stop them. Raise the copy's local start to its floor when it answers, as start_at already does. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 41 +++++++++++++++++++++++++++- rs/moq-net/src/model/subscription.rs | 4 +-- 2 files changed, 42 insertions(+), 3 deletions(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index f4e5a17caf..02dd720419 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -337,7 +337,8 @@ impl Copy { fn poll_ready(&mut self, groups: (u64, Cap), waiter: &kio::Waiter) -> Poll> { if let Sub::Pending(pending) = &self.sub { let mut sub = ready!(pending.poll_ok(waiter))?; - sub.raise_start_to(groups.0); + // An anchor the wire kept may resolve below where the reader got to. + sub.raise_start_to(self.floor.map_or(groups.0, |floor| groups.0.max(floor.group))); sub.end_at(groups.1); self.sub = Sub::Ready(Box::new(sub)); } @@ -1984,6 +1985,44 @@ mod test { assert_eq!(copy.wire(&updated).ranges, ranges); } + /// A replacement asked for a range reaching back past where the reader got to, while + /// still unanswered, never hands out a group the reader skipped below that point. + #[test] + #[cfg(feature = "async")] + fn a_pending_replacement_never_reads_below_the_floor() { + let routes = Producer::new(); + let logical = logical(&routes); + let a = copy(); + routes.serve(a.consume()); + let mut sub = subscribe(&logical, Duration::from_secs(30)); + let mut five = a.create_group(group::Info { sequence: 5 }).unwrap(); + five.write_frame(ts(5000), b"x".as_ref()).unwrap(); + assert_eq!(recv(&mut sub).sequence, 5); + + let b = track::Request::new(Arc::new(broadcast::Info::default()), "video"); + routes.serve(b.consume()); + // The read subscribes the replacement, which has not answered yet. + assert!(sub.recv_group().now_or_never().is_none()); + let back = sub.subscription().with_ranges([track::Range { + start: track::Anchor::Relative(100), + end: None, + }]); + sub.update(back).unwrap(); + assert!(sub.recv_group().now_or_never().is_none()); + + let b = b.accept(crate::track::Info::timed()); + for sequence in 0..7 { + let mut group = b.create_group(group::Info { sequence }).unwrap(); + group.write_frame(ts(sequence * 1000), b"x".as_ref()).unwrap(); + group.finish().unwrap(); + } + let mut rest = Vec::new(); + while let Some(Ok(Some(group))) = sub.recv_group().now_or_never() { + rest.push(group.sequence); + } + assert_eq!(rest, vec![6]); + } + #[test] fn a_replaced_copy_at_the_last_sequence_has_no_overflow_cap() { for (last, end) in [(Some(u64::MAX), None), (None, Some(Position::default()))] { diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index 289328679e..13d849835c 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -404,8 +404,8 @@ impl Subscription { /// /// `resolved` names where each range resolved, by index, so a relative or time anchor /// becomes its own group, raised to `floor` or capped at `end`. A start not resolved - /// keeps its anchor, since the floor may lie before it; an end not resolved becomes - /// the bound. + /// keeps its anchor, since the floor may lie before it, so the caller drops what it + /// resolves to below `floor`; an end not resolved becomes the bound. pub(super) fn clamped( &self, floor: Option, From a4eea28099f0c2a16e2e42ef7f32106e1fe2b74b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 15:55:28 -0700 Subject: [PATCH 20/36] fix(moq-mux): read the floor through the optional subscription main introduced Co-Authored-By: Claude Opus 5.5 --- rs/moq-mux/src/container/consumer.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 31fb4b5296..85ee5cc90f 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -311,7 +311,7 @@ impl Consumer { // A relative start names its floor once the first group to arrive resolves it. if self.startup && !self.newest - && let Some(start) = self.track.floor() + && let Some(start) = self.subscriber().floor() { self.pending.retain(|group| group.sequence >= start); self.current = start; From 80363284668b80c488416daa59364474466f2b02 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 17:09:39 -0700 Subject: [PATCH 21/36] fix(moq-net): a cursor holds only its range, and widening re-scans the cache A reader no longer parks a consumer and an abort watch for every group outside its ranges or past its cap, which a narrow reader under an open aggregate accumulated per produced group. The walk passes such a group holding nothing. A move of the floor, ranges, or cap owes the spans it newly admits, and the walk re-offers what the cache still holds there through an ordered lookup, once per widening: O(log n + k) for the k groups in the span. This subsumes the lowered-floor backfill. A finished track now ends a reader with nothing left in range instead of holding it open for a cap that may rise. Drift is measured under where the ranges end as well as the local cap, so a range behind the live edge keeps its own newest group. track_range_widening replaces track_parked_abort. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/benches/track.rs | 96 ++++--- rs/moq-net/src/model/aborts.rs | 85 +----- rs/moq-net/src/model/cache.rs | 10 +- rs/moq-net/src/model/group.rs | 6 - rs/moq-net/src/model/track.rs | 505 +++++++++++++++------------------ 5 files changed, 292 insertions(+), 410 deletions(-) diff --git a/rs/moq-net/benches/track.rs b/rs/moq-net/benches/track.rs index f0887e1e6a..5d34818d40 100644 --- a/rs/moq-net/benches/track.rs +++ b/rs/moq-net/benches/track.rs @@ -60,6 +60,11 @@ //! many ranges and how many cached groups each spans: each group handed out must cost the //! same, so a walk over the ranges or the groups between them shows up as a slope. //! +//! `track_range_widening` widens one reader's ranges by a fixed span of cached groups and +//! back, swept over the groups cached and the readers beside it: a reader holds nothing +//! for the groups outside its ranges, so a widening scans only the span it newly covers, +//! and a scan of the cache or the audience shows up as a slope. +//! //! Run with `cargo bench -p moq-net --bench track`. // Custom timings measure the real time the work took, not the ambient clock. @@ -115,6 +120,13 @@ const SPAN: [u64; 3] = [1, 64, 4096]; const SERVED_RANGES: [u64; 3] = [1, 16, 256]; const SERVED_SPAN: [u64; 3] = [1, 8, 64]; +/// Readers beside the one widening, and the groups cached around the span it widens by. +const WIDENING_READERS: [usize; 3] = [1, 64, 1024]; +const WIDENING_CACHED: [u64; 3] = [64, 1024, 16_384]; + +/// Cached groups one widening newly covers. +const WIDENED: u64 = 8; + /// Concurrent publishers sharing one relay-style cache pool. const WRITERS: [usize; 4] = [1, 2, 4, 8]; @@ -1202,47 +1214,47 @@ fn bench_empty_run(c: &mut Criterion) { group.finish(); } -/// Abort one non-prefix parked group, swept over backlog and capped subscribers. -fn bench_parked_abort(c: &mut Criterion) { - let mut bench = c.benchmark_group("track_parked_abort"); - bench.sample_size(10); - for readers in [1, 8, 64] { - for retained in [1u64, 64, 1024] { - bench.bench_function(BenchmarkId::new(readers.to_string(), retained), |b| { - b.iter_batched_ref( - || { - let broadcast = broadcast::Producer::new(Default::default()); - let track = broadcast.create_track("parked", timed()).unwrap(); - let mut subscribers: Vec<_> = (0..readers) - .map(|_| { - let mut subscriber = - track.subscribe(track::Subscription::default().with_max_delay(AGED_BUDGET)); - subscriber.set_groups(..0); - subscriber - }) - .collect(); - let groups: Vec<_> = (0..retained) - .map(|sequence| { - let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); - group.write_frame(Timestamp::ZERO, b"f".as_ref()).unwrap(); - group.clone().finish().unwrap(); - group - }) - .collect(); - for subscriber in &mut subscribers { - assert!(subscriber.poll_recv_group(&kio::Waiter::noop()).is_pending()); - } - (broadcast, track, subscribers, groups) - }, - |(_, _, subscribers, groups)| { - groups[retained as usize / 2].clone().abort(Error::Old).unwrap(); - for subscriber in subscribers { - let res = subscriber.poll_recv_group(&kio::Waiter::noop()); - assert!(res.is_pending()); - } - }, - BatchSize::PerIteration, - ); +/// One reader widening its ranges by [`WIDENED`] cached groups mid-cache and back, beside +/// idle narrow readers. Every widening after the first finds groups already handed out, +/// which the reader skips, so each one costs the same scan of the span. +fn bench_range_widening(c: &mut Criterion) { + let mut bench = c.benchmark_group("track_range_widening"); + bench.throughput(Throughput::Elements(1)); + for readers in WIDENING_READERS { + for cached in WIDENING_CACHED { + bench.bench_function(BenchmarkId::new(format!("{readers}r"), cached), |b| { + let broadcast = broadcast::Info::default().produce(); + let track = broadcast.create_track("bench", None).unwrap(); + for _ in 0..cached { + let mut group = track.append_group().unwrap(); + group.write_frame(None, Bytes::from_static(b"x")).unwrap(); + group.finish().unwrap(); + } + let narrow = track::Subscription::default() + .with_max_delay(AGED_BUDGET) + .with_groups(0..1); + let mid = cached / 2; + let wide = narrow + .clone() + .with_ranges([track::Range::groups(0..1), track::Range::groups(mid..mid + WIDENED)]); + let waiter = kio::Waiter::noop(); + let mut readers: Vec<_> = (0..readers) + .map(|_| { + let mut reader = track.subscribe(narrow.clone()); + while let Poll::Ready(Ok(Some(_))) = reader.poll_recv_group(&waiter) {} + reader + }) + .collect(); + let reader = &mut readers[0]; + b.iter(|| { + reader.update(wide.clone()).unwrap(); + while let Poll::Ready(Ok(Some(group))) = reader.poll_recv_group(&waiter) { + black_box(group); + } + reader.update(narrow.clone()).unwrap(); + assert!(reader.poll_recv_group(&waiter).is_pending()); + }); + drop((readers, track, broadcast)); }); } } @@ -1251,7 +1263,7 @@ fn bench_parked_abort(c: &mut Criterion) { criterion_group!( benches, - bench_parked_abort, + bench_range_widening, bench_fanout, bench_fanout_frame, bench_fetch_expiry, diff --git a/rs/moq-net/src/model/aborts.rs b/rs/moq-net/src/model/aborts.rs index 162ddedb3e..aa9995f7e2 100644 --- a/rs/moq-net/src/model/aborts.rs +++ b/rs/moq-net/src/model/aborts.rs @@ -1,40 +1,25 @@ -//! Abort notifications for the groups each capped subscriber parks, and for every -//! producer cut a wire publisher forwards as a drop. +//! Notifications of every producer cut a wire publisher forwards as a drop. use std::collections::{BTreeMap, BTreeSet}; use std::sync::Arc; -/// One registration per subscriber, with pending sequences bounded by its parked set, -/// or by the cuts between two polls. +/// One registration per wire publisher, with pending sequences bounded by the cuts +/// between two polls. #[derive(Default)] pub(super) struct Aborts { - // Visit only active cursors, even after a larger fanout has disconnected. + // Visit only active watchers, even after a larger fanout has disconnected. readers: kio::Lock>>, } #[derive(Default)] struct State { - // Notified of every cut, whatever `watched` holds. - cuts: bool, - watched: BTreeSet, pending: BTreeSet, waiters: kio::WaiterList, } impl Aborts { - pub(super) fn watch(self: &Arc) -> Watch { - self.register(State::default()) - } - pub(super) fn watch_cuts(self: &Arc) -> Watch { - self.register(State { - cuts: true, - ..State::default() - }) - } - - fn register(self: &Arc, state: State) -> Watch { - let state = kio::Lock::new(state); + let state = kio::Lock::new(State::default()); let mut readers = self.readers.lock(); let key = readers .last_key_value() @@ -47,13 +32,14 @@ impl Aborts { } } - pub(super) fn notify(&self, sequence: u64, cut: bool) { + /// Tell every watcher that the publisher cut group `sequence`. + pub(super) fn notify(&self, sequence: u64) { let mut wakes = Vec::new(); { let readers = self.readers.lock(); for reader in readers.values() { let mut state = reader.lock(); - if ((cut && state.cuts) || state.watched.contains(&sequence)) && state.pending.insert(sequence) { + if state.pending.insert(sequence) { wakes.push(state.waiters.take()); } } @@ -72,26 +58,6 @@ pub(super) struct Watch { } impl Watch { - pub(super) fn insert(&self, sequence: u64) { - self.state.lock().watched.insert(sequence); - } - - pub(super) fn remove(&self, sequence: u64) { - let mut state = self.state.lock(); - state.watched.remove(&sequence); - state.pending.remove(&sequence); - } - - pub(super) fn register(&self, waiter: &kio::Waiter) { - let mut state = self.state.lock(); - state.waiters.register(waiter); - let pending = !state.pending.is_empty(); - drop(state); - if pending { - waiter.waker().wake_by_ref(); - } - } - pub(super) fn poll(&self, waiter: &kio::Waiter) -> BTreeSet { let mut state = self.state.lock(); state.waiters.register(waiter); @@ -120,39 +86,16 @@ mod tests { } #[test] - fn registering_after_an_abort_still_wakes() { + fn a_cut_wakes_the_watcher_once_per_poll() { let registry = Arc::new(Aborts::default()); - let watch = registry.watch(); - watch.insert(7); - registry.notify(7, false); + let watch = registry.watch_cuts(); let counter = Arc::new(Counter::default()); let waiter = kio::Waiter::new(Waker::from(counter.clone())); - watch.register(&waiter); + assert!(watch.poll(&waiter).is_empty()); + registry.notify(2); + registry.notify(3); assert_eq!(counter.0.load(Ordering::SeqCst), 1); - assert_eq!(watch.poll(&waiter), BTreeSet::from([7])); - } - - #[test] - fn a_cuts_watch_sees_every_cut_and_nothing_else() { - let registry = Arc::new(Aborts::default()); - let watch = registry.watch_cuts(); - registry.notify(1, false); - registry.notify(2, true); - assert_eq!(watch.poll(&kio::Waiter::noop()), BTreeSet::from([2])); - } - - #[test] - fn notifications_remain_bounded_by_watched_groups() { - let registry = Arc::new(Aborts::default()); - let watch = registry.watch(); - watch.insert(3); - for sequence in 0..1024 { - registry.notify(sequence, false); - } - assert_eq!(watch.poll(&kio::Waiter::noop()), BTreeSet::from([3])); - registry.notify(3, false); - watch.remove(3); - assert!(watch.poll(&kio::Waiter::noop()).is_empty()); + assert_eq!(watch.poll(&waiter), BTreeSet::from([2, 3])); drop(watch); assert!(registry.readers.lock().is_empty()); } diff --git a/rs/moq-net/src/model/cache.rs b/rs/moq-net/src/model/cache.rs index 640fedfa7b..de7a2a20f6 100644 --- a/rs/moq-net/src/model/cache.rs +++ b/rs/moq-net/src/model/cache.rs @@ -814,19 +814,15 @@ impl Track { &self.wakes } - pub(super) fn watch_aborts(&self) -> aborts::Watch { - self.aborts.get_or_init(Arc::default).watch() - } - /// Watch every group the publisher cuts, rather than chosen sequences. pub(super) fn watch_cuts(&self) -> aborts::Watch { self.aborts.get_or_init(Arc::default).watch_cuts() } - /// Notify watchers that group `sequence` aborted, and whether the publisher cut it. + /// Notify watchers that group `sequence` aborted, if the publisher cut it. pub(super) fn aborted(&self, sequence: u64, cut: bool) { - if let Some(aborts) = self.aborts.get() { - aborts.notify(sequence, cut); + if cut && let Some(aborts) = self.aborts.get() { + aborts.notify(sequence); } } diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index 0090be9773..f145cb9ee6 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -2139,12 +2139,6 @@ impl Consumer { self.cursor.expiry_pending() } - /// Whether the group has been aborted (including pool eviction); the abort - /// dropped the cached frames, so a held consumer has nothing left to read. - pub(super) fn is_aborted(&self) -> bool { - self.cursor.state.read().abort.is_some() - } - /// Mark the group as still being read, so a slow batch drain does not expire it. pub fn keep_alive(&self) { self.cursor.state.read().charge.refresh(); diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index dd052d54c7..778093597a 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -23,7 +23,7 @@ use super::{Datagram, Requests}; use super::Cap; pub use super::subscription::{Anchor, Order, Position, Range, Subscription}; use super::subscription::{Containing, Holdings, NewGroup, Resolved, Window}; -use rangemap::RangeSet; +use rangemap::{RangeInclusiveSet, RangeSet}; mod demand; pub(super) use demand::Preferences; @@ -516,6 +516,12 @@ impl Slot { self.is_live() && !self.pending } + /// Whether the arrival walk hands this slot out: the live feed's, not withheld, and + /// still serving, a fetched head counting for an aborted live group. + fn is_walked(&self) -> bool { + self.visible && !self.pending && !self.is_aborted() + } + /// The copy to hand readers: the fetched head while it lasts, else the live group. fn serving(&self) -> &group::Producer { match &self.head { @@ -4647,22 +4653,8 @@ struct Cursor { /// the producer's cache and become eligible again when the cap rises (or is removed). /// `Some(0)` is the empty range. end_sequence: Option, - /// Groups received beyond the [`Self::end_sequence`] cap, held for `recv_group` - /// until the cap rises (arrival-order reads consume the shared cursor, so they - /// are parked here instead of dropped). Keyed by sequence so the lowest is - /// re-offered first. - parked: BTreeMap, - /// The highest floor `recv_group` has applied. Its walk passes the groups below the - /// floor without parking them, so lowering it parks those still cached. - passed: u64, - aborts: Option>, - /// [`Self::end_sequence`], shared with the groups this cursor hands out so their - /// expiry measures drift against the same servable window after a cap moves. - /// - /// Deliberately not where [`Subscription::ranges`] end. They filter what this handle - /// reads, but they are filters, not exemptions: max delay caps every range against the - /// track's live edge. Capping the edge at a range's end would judge a range in the past - /// only against itself, so none of it would ever be late however far the track moved on. + /// [`Self::edge_cap`], shared with the groups this cursor hands out so their expiry + /// measures drift against the same servable window after the cap or ranges move. drift_cap: kio::Producer>, /// Groups the drift budget skipped since the count was last drained. Accumulated /// here rather than metered in place because the stats scope lives on the owning @@ -4772,10 +4764,14 @@ struct Filter { revision: u64, /// Why the ranges can't be served here: a time anchor on an untimed track. refused: Option, - /// The revision and cap at which no parked group was in reach. A group is only parked - /// out of reach, so the walk waits for either to move rather than running once per - /// hand-out over every range. - reoffer_idle: Option<(u64, Option)>, + /// What the arrival walk reads: the window from the floor, below the cap. The walk + /// passes a group outside it holding nothing, so widening it re-scans the cache. + admitted: RangeInclusiveSet, + /// The floor, [`Self::revision`], and cap [`Self::admitted`] was taken at. + admitted_at: Option<(u64, u64, Option)>, + /// The spans a widening newly admitted, scanned in the cache lowest first and handed + /// out ahead of the walk: each cached group in them is visited once. + owed: RangeInclusiveSet, } impl Cursor { @@ -4811,7 +4807,9 @@ impl Cursor { resolved: ranges_revision, revision: 0, refused, - reoffer_idle: None, + admitted: RangeInclusiveSet::new(), + admitted_at: None, + owed: RangeInclusiveSet::new(), }; (filter, state.datagram_offset + start) }; @@ -4824,9 +4822,6 @@ impl Cursor { datagram_index, next_sequence: Some(0), end_sequence: None, - parked: BTreeMap::new(), - passed: 0, - aborts: None, drift_cap: kio::Producer::new(None), stale: stats::Content::default(), seek_pending: BTreeMap::new(), @@ -4860,13 +4855,24 @@ impl Cursor { } } - /// Publish [`Self::end_sequence`] to the groups already handed out. + /// The exclusive end drift is measured under: the cap, lowered to where the ranges + /// end, so a range behind the live edge is judged against its own newest group rather + /// than skipped for groups it never reads. + fn edge_cap(&self) -> Option { + match (self.end_sequence, self.filter.window.end()) { + (Some(cap), Some(end)) => Some(cap.min(end)), + (cap, end) => cap.or(end), + } + } + + /// Publish [`Self::edge_cap`] to the groups already handed out. fn update_drift_cap(&mut self) { + let cap = self.edge_cap(); // Skip a no-op write: every handed-out group's expiry watches this channel. - if *self.drift_cap.read() != self.end_sequence + if *self.drift_cap.read() != cap && let Ok(mut current) = self.drift_cap.write() { - *current = self.end_sequence; + *current = cap; } } @@ -4901,6 +4907,7 @@ impl Cursor { drop(state); if moved { self.filter.revision += 1; + self.update_drift_cap(); } } @@ -4915,20 +4922,90 @@ impl Cursor { super::subscription::before_end(sequence, self.end_sequence) && self.filter.window.contains(sequence) } - /// The lowest parked group this cursor reads now. - fn reoffer(&mut self) -> Option { - let at = (self.filter.revision, self.end_sequence); - if self.parked.is_empty() || self.filter.reoffer_idle == Some(at) { + /// The groups the arrival walk reads under the exclusive `cap`: the window from the + /// floor, below it. + fn admitted(&self, cap: Option) -> RangeInclusiveSet { + let mut admitted = RangeInclusiveSet::new(); + let last = match cap { + Some(end) => match end.checked_sub(1) { + Some(last) => last, + None => return admitted, + }, + None => u64::MAX, + }; + for run in self.filter.window.runs(self.floor()) { + if *run.start() > last { + break; + } + admitted.insert(*run.start()..=(*run.end()).min(last)); + } + admitted + } + + /// [`Filter::owed`] as of now: what was owed and what the walk newly reads since + /// [`Filter::admitted`] was taken, both cut to `admitted`. + fn owed(&self, admitted: &RangeInclusiveSet) -> RangeInclusiveSet { + // Nothing is owed before the walk first reads: it starts at the oldest arrival. + if self.filter.admitted_at.is_none() { + return RangeInclusiveSet::new(); + } + let mut owed = RangeInclusiveSet::new(); + for run in admitted.iter() { + for gap in self.filter.admitted.gaps(run) { + owed.insert(gap); + } + for span in self.filter.owed.overlapping(run) { + owed.insert((*span.start()).max(*run.start())..=(*span.end()).min(*run.end())); + } + } + owed + } + + /// Take in a move of the floor, the ranges, or the cap: whatever it newly admits is + /// owed. Costs the ranges, never the cache, and only when one moved. + fn admit(&mut self) { + let at = (self.floor(), self.filter.revision, self.end_sequence); + if self.filter.admitted_at == Some(at) { + return; + } + let admitted = self.admitted(self.end_sequence); + self.filter.owed = self.owed(&admitted); + self.filter.admitted = admitted; + self.filter.admitted_at = Some(at); + } + + /// The lowest cached group still owed, consuming the spans scanned to find it: only + /// one the walk could have read, never fetched backfill or a withheld group. + fn next_owed(&mut self) -> Option { + if self.filter.owed.is_empty() { return None; } - for run in self.filter.window.runs(self.floor()) { - if let Some((&sequence, _)) = self.parked.range(run).next() - && super::subscription::before_end(sequence, self.end_sequence) - { - return Some(sequence); + let state = self.state.read(); + // Nothing is read from the cache now, so leave what is owed until it is. + if !state.readable() { + return None; + } + let low = state.live_floor.unwrap_or(0); + while let Some(run) = self.filter.owed.iter().next().cloned() { + let from = (*run.start()).max(low); + let found = (from <= *run.end()) + .then(|| { + state + .lookup + .range(from..=*run.end()) + .map(|(_, slot)| slot) + .find(|slot| slot.is_walked()) + .map(Slot::serving) + }) + .flatten(); + match found { + Some(group) => { + self.filter.owed.remove(*run.start()..=group.sequence); + return Some(group.clone()); + } + None => self.filter.owed.remove(run), } } - self.filter.reoffer_idle = Some(at); None } @@ -4992,7 +5069,7 @@ impl Cursor { let max_delay = self.subscription.read().max_delay; let state = self.state.read(); let budget = clamp_max_delay(max_delay, state.max_age_bound()); - let edge = state.drift_edge(self.end_sequence); + let edge = state.drift_edge(self.edge_cap()); // Every group below the last successor found shares it, so a run of cached groups // that cannot serve is walked once per pass, not once per gap. let mut arrival: Option<&Slot> = None; @@ -5031,123 +5108,33 @@ impl Cursor { })) } - /// Hold a capped group without losing another copy's abort notification. - fn park(&mut self, consumer: group::Consumer, waiter: &kio::Waiter) { - let aborts = self - .aborts - .get_or_insert_with(|| Box::new(self.state.read().cache.watch_aborts())); - // Register before checking: an abort on either side of the check is - // observed now or queued for the next poll. - aborts.insert(consumer.sequence); - if consumer.is_aborted() { - if !self.parked.contains_key(&consumer.sequence) { - aborts.remove(consumer.sequence); - } - } else { - self.parked.insert(consumer.sequence, consumer); - } - aborts.register(waiter); - } - - /// Park the cached groups in range from `floor` up to [`Self::passed`], which the walk - /// passed under a higher floor, so they are re-offered like any widened range. - fn backfill(&mut self, floor: u64, waiter: &kio::Waiter) { - let groups: Vec = { - let state = self.state.read(); - // Nothing is read from the cache now, so leave the floor to lower once it is. - if !state.readable() { - return; - } - self.filter - .window - .runs(floor.max(state.live_floor.unwrap_or(0))) - .take_while(|run| *run.start() < self.passed) - // Only what the walk could have read: never a fetched or withheld group. - .flat_map(|run| { - state - .lookup - .range(run) - .filter(|(_, slot)| slot.is_shown()) - .map(|(_, slot)| slot.serving()) - }) - .take_while(|group| group.sequence < self.passed) - .filter(|group| !group.is_aborted()) - .cloned() - .collect() - }; - self.passed = floor; - if groups.is_empty() { - return; - } - // Consumed with the track guard released, as the walk does. - for group in groups { - self.park(group.consume(), waiter); - } - // In reach at the revision `reoffer` last found nothing at, so look again. - self.filter.reoffer_idle = None; - } - fn poll_recv_group(&mut self, waiter: &kio::Waiter) -> Poll>> { self.sync(); if let Some(err) = &self.filter.refused { return Poll::Ready(Err(err.clone())); } - let floor = self.floor(); - match floor < self.passed { - true => self.backfill(floor, waiter), - false => self.passed = floor, - } - if let Some(aborts) = &self.aborts { - // A sequence may name both a live group and its fetched head. Only the - // consumer we parked decides whether this notification invalidates it. - for sequence in aborts.poll(waiter) { - if self.parked.get(&sequence).is_some_and(group::Consumer::is_aborted) { - self.parked.remove(&sequence); - aborts.remove(sequence); - } - } - // Each overtaken entry is removed once, rather than scanning every entry - // again for every delivered group. - while let Some((&sequence, _)) = self.parked.first_key_value() - && sequence < floor - { - self.parked.pop_first(); - aborts.remove(sequence); - } - if self.parked.is_empty() { - self.aborts = None; - } - } + self.admit(); // One scan for the whole poll, so walking a backlog off stays linear in its size. - let drift = ready!(self.poll_drift(self.end_sequence, waiter))?; + let drift = ready!(self.poll_drift(self.edge_cap(), waiter))?; loop { - // Re-offer the lowest parked group back in range once the cap rises or the ranges - // widen, ahead of the arrival cursor: it is the oldest thing still owed. - let consumer = match self.reoffer() { - Some(sequence) => { - let group = self.parked.remove(&sequence).expect("just looked it up"); - self.aborts - .as_ref() - .expect("parked groups are watched") - .remove(sequence); + // Re-offer what a widening newly covers ahead of the arrival walk, lowest first: + // it is the oldest thing still owed. A group the walk has yet to reach comes + // again from it, and the subscriber's delivered set drops the repeat. + let consumer = match self.next_owed() { + Some(producer) => { + let consumer = producer.consume(); // A re-offer is a delivery: stamp it like a fresh hand-out. - group.cache_refresh(); - group + consumer.cache_refresh(); + consumer } - _ => { + None => { let floor = self.floor(); - self.passed = self.passed.max(floor); let Some((producer, found_index)) = ready!(self.poll(waiter, |state| state.poll_recv_group(self.index, floor))?) else { - // Parked groups survive a finished track: they become deliverable - // again if the cap rises, so the stream isn't over while any are held. - if self.parked.is_empty() { - return Poll::Ready(Ok(None)); - } - return Poll::Pending; + return Poll::Ready(Ok(None)); }; let consumer = producer.consume(); // Stamp with the track guard released, so delivery never nests the @@ -5159,17 +5146,16 @@ impl Cursor { // above it, and a later one may settle where a time end falls. if !self.filter.window.is_settled() { self.sync(); + self.admit(); if consumer.sequence < self.floor() { - self.passed = self.passed.max(self.floor()); continue; } } - // Park a group beyond the cap or outside the ranges instead of dropping it, - // and keep scanning so an in-range group that arrived behind it still flows. + // Pass a group beyond the cap or outside the ranges, holding nothing: a + // widening finds it in the cache again. Keep scanning so an in-range group + // that arrived behind it still flows. if !self.admits(consumer.sequence) { - self.park(consumer, waiter); - continue; } consumer @@ -5236,7 +5222,7 @@ impl Cursor { let mut floor = next.max(self.floor()); let end = self.end_sequence; // One scan for the whole poll, so walking a backlog off stays linear in its size. - let drift = ready!(self.poll_drift(end, waiter))?; + let drift = ready!(self.poll_drift(self.edge_cap(), waiter))?; let group = loop { let window = &self.filter.window; @@ -5414,11 +5400,11 @@ impl Subscriber { /// The budget remains attached to a returned group: if it stalls while newer data /// advances, its pending frame read ends with [`Error::Old`]. /// - /// Honors the group range set by [`Self::set_groups`]: - /// a group beyond the cap is parked (not dropped) and re-offered once the cap rises - /// (lowest sequence first), without blocking in-range groups that arrive behind it. - /// A parked group that the producer evicts or expires in the meantime is dropped, - /// so parking never outlives the track's cache policy. + /// Honors the group range set by [`Self::set_groups`]: a group beyond the cap is + /// passed without blocking in-range groups that arrive behind it, and holds nothing + /// here. Raising the cap, or widening the ranges, offers the groups it newly covers + /// that the cache still holds, lowest sequence first. Once the track finishes, a + /// reader with nothing left in range ends. /// /// Returns `Poll::Ready(Ok(Some(group)))` when a group is available, /// `Poll::Ready(Ok(None))` when the track is finished, @@ -5510,29 +5496,36 @@ impl Subscriber { && cursor.filter.window.contains(sequence) && !unread.delivered.contains(sequence) }; - // Passed under a higher floor, so the walk parks them again; see `Cursor::backfill`. - let passed = || { - state - .lookup - .range(floor..cursor.passed.max(floor)) - .any(|(sequence, slot)| eligible(*sequence) && slot.is_shown() && !slot.is_aborted()) + // What a widening owes, the walk has yet to take in, or lifting the local cap would: + // a group past it is unread, not gone. + let owed = cursor.owed(&cursor.admitted(None)); + let last = unread.end.map(|end| end.checked_sub(1)); + let owed = || { + owed.iter().any(|run| { + let end = match last { + Some(Some(last)) => (*run.end()).min(last), + Some(None) => return false, + None => *run.end(), + }; + *run.start() <= end + && state + .lookup + .range(*run.start()..=end) + .any(|(sequence, slot)| eligible(*sequence) && slot.is_walked()) + }) }; - cursor - .parked - .iter() - .any(|(sequence, group)| eligible(*sequence) && !group.is_aborted()) - || (state.readable() - && (state - .arrival - .iter() - .skip(cursor.index.saturating_sub(state.offset)) - .any(|(sequence, stamp)| { - eligible(*sequence) - && state - .lookup - .get(sequence) - .is_some_and(|slot| slot.stamp == *stamp && !slot.is_aborted()) - }) || passed())) + state.readable() + && (state + .arrival + .iter() + .skip(cursor.index.saturating_sub(state.offset)) + .any(|(sequence, stamp)| { + eligible(*sequence) + && state + .lookup + .get(sequence) + .is_some_and(|slot| slot.stamp == *stamp && !slot.is_aborted()) + }) || owed()) } /// Receive the next group in arrival order. @@ -5743,7 +5736,7 @@ impl Subscriber { /// /// `2..=5` includes groups 2 through 5; `2..5` excludes group 5. An omitted /// start preserves the current floor, and an omitted end removes the cap. - /// Groups above the end remain buffered and can be read after raising the cap. + /// Groups above the end stay cached and can be read after raising the cap. /// This narrows only local delivery, within [`Subscription::ranges`]; use those and /// [`Self::update`] to change the requested groups. pub fn set_groups(&mut self, groups: impl RangeBounds) { @@ -5788,10 +5781,10 @@ impl Subscriber { /// A local filter, not a request; [`Subscription::ranges`] are the wire-level /// counterpart. See [Local cursor vs wire preference](Self#local-cursor-vs-wire-preference). /// - /// Groups beyond the cap are held rather than skipped past, so a later call to - /// [`Self::set_groups`] with a higher bound (or unbounded) makes them available again. - /// Lowering the cap below the consumer's current cursor parks the consumer until the - /// cap is raised. + /// Groups beyond the cap stay in the cache, so a later call to [`Self::set_groups`] + /// with a higher bound (or unbounded) offers the ones still cached. Lowering the cap + /// below the consumer's current cursor waits until the cap is raised, or the track + /// finishes. // pending: /quest/m1/public-model/serve.md pub(crate) fn end_at(&mut self, end: impl Into) { match &mut self.inner { @@ -9496,16 +9489,19 @@ mod test { #[test] #[cfg(feature = "async")] - fn a_requested_end_does_not_cap_the_live_edge() { + fn a_range_behind_the_live_edge_keeps_its_newest_group() { let mut producer = track_producer("test", Info::timed()); - for second in 0..4 { + for second in 0..=10 { append_at(&mut producer, second * 1000); } - // The budget caps every range: a range ending below the live edge is judged against - // that edge, not its own end, so with no budget nothing in it is fresh. - let mut subscriber = producer.subscribe(Subscription::default().with_end(Position::after_group(1))); - assert_eq!(drain(&mut subscriber), Vec::::new()); + // Drift is measured under where the range ends, so with no budget its own newest + // group is still fresh: the groups past it are not ones it reads. + let mut subscriber = producer.subscribe(Subscription::default().with_groups(0..2)); + assert_eq!(drain(&mut subscriber), vec![1]); + let mut subscriber = producer.subscribe(Subscription::default().with_groups(0..2)).ordered(); + let next = subscriber.next_group().now_or_never().unwrap().unwrap().unwrap(); + assert_eq!(next.sequence, 1); // A budget that covers it delivers the range, and nothing above it. let mut subscriber = producer.subscribe(replay().with_end(Position::after_group(1))); @@ -10891,12 +10887,12 @@ mod test { ); } - /// `recv_group` (arrival order) honors the `end_at` cap by parking, like - /// `next_group`: beyond-cap groups are held, not dropped, and a raised cap - /// re-offers them, even after the track finishes. + /// `recv_group` (arrival order) honors the `end_at` cap like `next_group`: a group + /// beyond it is passed, holding nothing, and a raised cap finds it in the cache again. + /// Once the track finishes, a capped reader ends with what it could read. #[test] #[cfg(feature = "async")] - fn end_at_parks_recv_group() { + fn end_at_caps_recv_group() { let producer = track_producer("test", Info::timed()); let mut consumer = producer.subscribe(replay()); @@ -10915,21 +10911,19 @@ mod test { ); assert!(consumer.recv_group().now_or_never().is_none(), "capped at 2"); - // A finished track keeps the parked group claimable: the cap may rise. - producer.finish().unwrap(); - assert!( - consumer.recv_group().now_or_never().is_none(), - "still parked after finish" - ); - consumer.set_groups(..); assert_eq!( consumer.recv_group().now_or_never().unwrap().unwrap().unwrap().sequence, 2 ); + + consumer.set_groups(..3); + producer.create_group(group::Info { sequence: 3 }).unwrap(); + assert!(consumer.recv_group().now_or_never().is_none(), "capped at 3"); + producer.finish().unwrap(); assert!( matches!(consumer.recv_group().now_or_never(), Some(Ok(None))), - "finished once the parked group drains" + "a finished track ends a capped reader" ); } @@ -11041,52 +11035,6 @@ mod test { ); } - /// Eviction aborts a parked group behind a sleeping subscriber's back. Nothing - /// else will poll it (the track already finished), so the entry has to carry a - /// waiter or the subscription sleeps forever holding its stream open. - #[test] - #[cfg(feature = "async")] - fn evicted_parked_group_wakes_the_clean_end() { - use std::sync::atomic::{AtomicUsize, Ordering}; - use std::task::{Context, Wake}; - - /// A waker that counts its wakes, for asserting a pending poll left a live - /// registration behind. - struct CountWaker(AtomicUsize); - impl Wake for CountWaker { - fn wake(self: std::sync::Arc) { - self.0.fetch_add(1, Ordering::SeqCst); - } - } - - let producer = track_producer("test", Info::timed()); - let mut consumer = producer.subscribe(None); - - producer.create_group(group::Info { sequence: 0 }).unwrap(); - assert_eq!( - consumer.recv_group().now_or_never().unwrap().unwrap().unwrap().sequence, - 0 - ); - - consumer.set_groups(..1); - let straggler = producer.create_group(group::Info { sequence: 1 }).unwrap(); - assert!(consumer.recv_group().now_or_never().is_none(), "parked at the cap"); - producer.finish().unwrap(); - - let counter = std::sync::Arc::new(CountWaker(AtomicUsize::new(0))); - let waker = std::task::Waker::from(counter.clone()); - let mut cx = Context::from_waker(&waker); - let mut fut = std::pin::pin!(consumer.recv_group()); - assert!( - fut.as_mut().poll(&mut cx).is_pending(), - "the parked group holds it open" - ); - - straggler.abort(Error::Old).unwrap(); - assert!(counter.0.load(Ordering::SeqCst) > 0, "the eviction wakeup was lost"); - assert!(matches!(fut.as_mut().poll(&mut cx), Poll::Ready(Ok(None)))); - } - /// An exclusive cap at 0 is the empty range: no group is delivered, even group 0. #[test] #[cfg(feature = "async")] @@ -11783,47 +11731,6 @@ mod test { ); } - #[test] - fn an_aborted_candidate_keeps_the_parked_copy_watched() { - let producer = track_producer("test", Info::timed()); - let live = producer.create_group(group::Info { sequence: 0 }).unwrap(); - let mut subscriber = producer.subscribe(replay()); - subscriber.set_groups(..0); - let waiter = kio::Waiter::noop(); - assert!(subscriber.poll_recv_group(&waiter).is_pending()); - - // A second copy can abort after lookup but before the cursor parks it. - let candidate = group::Info { sequence: 0 }.produce(); - let aborted = candidate.consume(); - candidate.abort(Error::Old).unwrap(); - let Inner::Plain(cursor) = &mut subscriber.inner else { - unreachable!() - }; - cursor.park(aborted, &waiter); - - live.abort(Error::Old).unwrap(); - producer.finish().unwrap(); - assert!(matches!(subscriber.poll_recv_group(&waiter), Poll::Ready(Ok(None)))); - } - - #[test] - fn an_abort_removes_a_non_prefix_parked_entry() { - let producer = track_producer("test", Info::timed()); - let groups: Vec<_> = (0..3) - .map(|sequence| producer.create_group(group::Info { sequence }).unwrap()) - .collect(); - let mut subscriber = producer.subscribe(replay()); - subscriber.set_groups(..0); - let waiter = kio::Waiter::noop(); - assert!(subscriber.poll_recv_group(&waiter).is_pending()); - groups[1].clone().abort(Error::Old).unwrap(); - assert!(subscriber.poll_recv_group(&waiter).is_pending()); - let Inner::Plain(cursor) = &subscriber.inner else { - unreachable!() - }; - assert_eq!(cursor.parked.keys().copied().collect::>(), vec![0, 2]); - } - /// The live copy ending early (its subscription went away) leaves a fetched head /// that still holds the whole group servable, instead of reclaiming the slot. #[moq_clock::test] @@ -13804,6 +13711,36 @@ mod test { assert_eq!(drain(&mut late), vec![4]); } + /// A narrow reader under an open aggregate passes the groups outside its range holding + /// nothing for them, however many arrive, and widening its ranges still serves the + /// ones the cache holds in the new span, lowest first and once each. + #[test] + #[cfg(feature = "async")] + fn a_narrow_reader_holds_nothing_for_excluded_groups() { + let producer = track_producer("test", Info::timed()); + let _wide = producer.subscribe(replay()); + let mut narrow = producer.subscribe(replay().with_groups(0..2)); + for sequence in 0..1000 { + producer.create_group(group::Info { sequence }).unwrap(); + } + assert_eq!(drain(&mut narrow), vec![0, 1]); + let Inner::Plain(cursor) = &narrow.inner else { + unreachable!() + }; + assert!(cursor.filter.owed.is_empty(), "nothing owed for excluded groups"); + assert_eq!(cursor.filter.admitted.iter().count(), 1, "only the range is kept"); + + narrow + .update(replay().with_ranges([Range::groups(0..2), Range::groups(5..8)])) + .unwrap(); + assert_eq!(drain(&mut narrow), vec![5, 6, 7]); + narrow.set_groups(..9); + narrow.update(replay().with_groups(0..)).unwrap(); + assert_eq!(drain(&mut narrow), vec![2, 3, 4, 8]); + narrow.set_groups(..); + assert_eq!(drain(&mut narrow), (9..1000).collect::>()); + } + /// Lowering the floor re-offers the cached groups the arrival walk passed below it, /// as widening any range does. #[test] From 5f53bdd6168a455b14949efa087e9c766cd6eda3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 17:09:39 -0700 Subject: [PATCH 22/36] fix(moq-net): a copy updated while pending names the ranges it will resolve Clearing a pending copy's resolution on a range update left `from` naming the old ranges, so once it answered and was replaced, the replacement could not take its resolution and resolved anchors from its own cache instead. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 50 +++++++++++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 2924493057..1bf4443bb8 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -314,7 +314,11 @@ impl Copy { self.from = reader.ranges.clone(); } Sub::Pending(_) if reader.ranges == self.from => {} - Sub::Pending(_) => self.resolved.clear(), + // Its own resolution, once it answers, is for these. + Sub::Pending(_) => { + self.resolved.clear(); + self.from = reader.ranges.clone(); + } } let mut wire = reader.clamped(self.floor, self.end().map(Position::group), &self.resolved); // A bounded copy only finishes what it had, so a new group from it never arrives. @@ -2283,6 +2287,50 @@ mod test { assert_eq!(rest, vec![6]); } + /// A copy updated while pending, then answering, hands its own resolution of the + /// updated ranges to the copy replacing it, rather than leaving the replacement to + /// resolve them from whatever it caches. + #[test] + #[cfg(feature = "async")] + fn a_copy_updated_while_pending_passes_on_its_resolution() { + let routes = Producer::new(); + let logical = logical(&routes); + let a = copy(); + routes.serve(a.consume()); + let mut sub = subscribe(&logical, Duration::from_secs(30)); + let mut zero = a.create_group(group::Info { sequence: 0 }).unwrap(); + zero.write_frame(ts(0), b"x".as_ref()).unwrap(); + assert_eq!(recv(&mut sub).sequence, 0); + + let b = track::Request::new(Arc::new(broadcast::Info::default()), "video"); + routes.serve(b.consume()); + // The read subscribes the replacement, which has not answered yet. + assert!(sub.recv_group().now_or_never().is_none()); + let ranges = [track::Range { + start: track::Anchor::Time(ts(10_000)), + end: None, + }]; + sub.update(sub.subscription().with_ranges(ranges)).unwrap(); + assert!(sub.recv_group().now_or_never().is_none()); + + let b = b.accept(crate::track::Info::timed()); + for sequence in 0..13 { + let mut group = b.create_group(group::Info { sequence }).unwrap(); + group.write_frame(ts(sequence * 1000), b"x".as_ref()).unwrap(); + group.finish().unwrap(); + } + while let Some(Ok(Some(_))) = sub.recv_group().now_or_never() {} + + let c = copy(); + routes.serve(c.consume()); + assert!(sub.recv_group().now_or_never().is_none()); + let start = c.subscription().expect("subscribed").ranges[0].start; + assert!( + matches!(start, track::Anchor::Absolute(position) if position.group >= 10), + "the replacement asks from where b resolved the time: {start:?}" + ); + } + #[test] fn a_replaced_copy_at_the_last_sequence_has_no_overflow_cap() { for (last, end) in [(Some(u64::MAX), None), (None, Some(Position::default()))] { From 6c77716113d2a661fe65d941b5f80090b7866431 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 17:10:15 -0700 Subject: [PATCH 23/36] docs(moq-net): a finished track ends a capped subscriber Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/MIGRATE.md | 1 + 1 file changed, 1 insertion(+) diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index 0f109e1f00..72d340397b 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -10,6 +10,7 @@ A `track::Subscriber` now reads only the groups its own `Subscription::ranges` c A reader that relied on groups another subscriber asked for must widen its own ranges. `track::Anchor` gains `Time(Timestamp)`, the group containing a presentation time, so exhaustive matches need an arm for it. Subscribing or updating with one on an untimed track fails with `Error::TimestampMismatch`. +Once a track finishes, a subscriber with nothing left under its `set_groups` cap ends rather than waiting for the cap to rise; raise it before the track finishes to read past it. ### An expired group fails with `Old` even when drained (#5944) From 6db1f342bfa5e7c7e4e2f9cd780f0c336c44ff2e Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 17:17:19 -0700 Subject: [PATCH 24/36] test(moq-net): apply the pending copy's update as the reader does Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 1bf4443bb8..02b5512622 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -2244,6 +2244,7 @@ mod test { }]; let updated = Subscription::default().with_ranges(ranges.clone()); assert!(matches!(copy.sub, Sub::Pending(_))); + copy.update(&updated); assert_eq!(copy.wire(&updated).ranges, ranges); assert!(copy.poll_ready((0, Cap::from(..)), &kio::Waiter::noop()).is_ready()); assert_eq!(copy.wire(&updated).ranges, ranges); From 30cb4f16cf6b3927ac615613ee7ee8c74cbf80c9 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 18:01:54 -0700 Subject: [PATCH 25/36] fix(moq-net): a time end stays open across a group yet to start A later group starting past a time end settled it even with a cached group below that had not presented frame 0, so once that group started inside the end it was still delivered. A group that may still start now keeps the end unresolved, and the window settles it on a later poll as before; one that never will (finished empty, aborted, or joined mid-group) proves nothing. No per-group state is kept. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/group.rs | 7 +++++ rs/moq-net/src/model/subscription.rs | 5 ++-- rs/moq-net/src/model/subscription/window.rs | 6 ++-- rs/moq-net/src/model/track.rs | 31 ++++++++++++++++++++- 4 files changed, 44 insertions(+), 5 deletions(-) diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index f145cb9ee6..2ebe058c27 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -1478,6 +1478,13 @@ impl Producer { self.state.read().started() } + /// Whether this copy may yet present the group's frame 0: it holds it, and nothing + /// opened or ended it. + pub(super) fn may_start(&self) -> bool { + let state = self.state.read(); + state.offset == 0 && !state.started() + } + /// Where the group starts in presentation time: its first frame's timestamp, or /// `None` while no frame has been opened or when that frame is untimed. /// diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index 13d849835c..ed443d27c1 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -166,8 +166,9 @@ pub enum Anchor { /// Resolved from the starts of the groups the serving hop holds. When none starts at /// or before it, a start reaches back to the oldest group, and while no group has /// started at all it waits like `Relative(0)`. An end stays open until a group - /// starting after it is known, so at the live edge a reader may get every group - /// handed out before its first frame showed it starts past the end. An untimed track + /// starting after it is known, with every group below that one started or never to + /// start, so at the live edge a reader may get every group handed out before its + /// first frame showed it starts past the end. An untimed track /// refuses it with [`Error::TimestampMismatch`](crate::Error::TimestampMismatch). Time(Timestamp), } diff --git a/rs/moq-net/src/model/subscription/window.rs b/rs/moq-net/src/model/subscription/window.rs index 4540e42701..96f401c1e0 100644 --- a/rs/moq-net/src/model/subscription/window.rs +++ b/rs/moq-net/src/model/subscription/window.rs @@ -27,7 +27,8 @@ pub(in crate::model) trait Holdings { /// Where `pts` falls among the groups with a known start. fn containing(&self, pts: Timestamp) -> Containing; - /// The lowest group at or above `from` known to start after `pts`. + /// The lowest group at or above `from` known to start after `pts`, `None` while a + /// group below it may yet start at or before `pts`. fn after(&self, pts: Timestamp, from: u64) -> Option; } @@ -74,7 +75,8 @@ enum End { Before(u64), /// Past every group. Open, - /// Before the first group starting after this time, once one is known. + /// Before the first group starting after this time, once one is known and no group + /// below it may yet start before the time. Time(Timestamp), } diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 778093597a..19fe2a61e4 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -587,6 +587,12 @@ impl Slot { .find_map(group::Producer::timestamp) } + /// Whether the group has no start yet but may still get one: a copy holding frame 0 + /// has not presented it. + fn may_start(&self) -> bool { + self.start().is_none() && self.copies().any(group::Producer::may_start) + } + /// The checked start of whichever copy began at frame 0. fn bound(&self) -> Option { self.copies().find_map(group::Producer::bound) @@ -684,7 +690,9 @@ impl Holdings for TrackState { match slot.start() { Some(start) if later(start, pts) => after = Some(sequence), Some(_) => break, - // Not started, so it proves nothing either way. + // It may still start before `pts`, so nothing above it settles the end yet. + None if slot.may_start() => after = None, + // It never will, so it proves nothing either way. None => {} } } @@ -13864,6 +13872,27 @@ mod test { assert_eq!(drain(&mut seek), vec![1, 2]); } + /// A time end stays open across a group that has yet to start: it may start before the + /// end, so a later group starting after it settles nothing. + #[test] + #[cfg(feature = "async")] + fn a_time_end_waits_on_an_unstarted_group_below_it() { + let producer = track_producer("test", Info::timed()); + let at = |millis| Timestamp::from_millis(millis).unwrap(); + let mut zero = producer.create_group(group::Info { sequence: 0 }).unwrap(); + zero.write_frame(at(0), bytes::Bytes::from_static(b"x")).unwrap(); + let mut hole = producer.create_group(group::Info { sequence: 1 }).unwrap(); + let mut two = producer.create_group(group::Info { sequence: 2 }).unwrap(); + two.write_frame(at(2000), bytes::Bytes::from_static(b"x")).unwrap(); + + let mut subscriber = producer.subscribe(replay().with_ranges([Range { + start: Anchor::Absolute(Position::group(0)), + end: Some(Anchor::Time(at(500))), + }])); + hole.write_frame(at(1000), bytes::Bytes::from_static(b"x")).unwrap(); + assert_eq!(drain(&mut subscriber), vec![0]); + } + /// An untimed track refuses a time anchor wherever the ranges reach it. #[test] #[cfg(feature = "async")] From 03611bc579297b94a9f899c9bef3ae1452c8d4b4 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 18:23:37 -0700 Subject: [PATCH 26/36] fix(moq-net): a time end settles only once every sequence below it is accounted for A later group starting past a time end settled it across a sequence that had not arrived, so when that group then arrived starting inside the end it was still delivered. Timestamps never rewind, so the end now settles before the lowest group starting past it only once every sequence between it and a group starting at or before the time is accounted for: arrived and started, unable to start, dropped, below where the feed starts, or given up by the arrivals' grace. Checked as ranges against the drops and arrivals the track already keeps, so nothing is held per group and a wide gap costs no more than a narrow one. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/subscription.rs | 8 ++- rs/moq-net/src/model/subscription/window.rs | 4 +- rs/moq-net/src/model/track.rs | 74 +++++++++++++++++++-- 3 files changed, 74 insertions(+), 12 deletions(-) diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index ed443d27c1..97c7b46f62 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -166,9 +166,11 @@ pub enum Anchor { /// Resolved from the starts of the groups the serving hop holds. When none starts at /// or before it, a start reaches back to the oldest group, and while no group has /// started at all it waits like `Relative(0)`. An end stays open until a group - /// starting after it is known, with every group below that one started or never to - /// start, so at the live edge a reader may get every group handed out before its - /// first frame showed it starts past the end. An untimed track + /// starting after it is known and every sequence below that one is accounted for + /// (started, dropped, or never to start), so at the live edge a reader may get every + /// group handed out before its first frame showed it starts past the end, and a feed + /// that reports no drops leaves it open across a missing group until the wait for that + /// group runs out. An untimed track /// refuses it with [`Error::TimestampMismatch`](crate::Error::TimestampMismatch). Time(Timestamp), } diff --git a/rs/moq-net/src/model/subscription/window.rs b/rs/moq-net/src/model/subscription/window.rs index 96f401c1e0..d7360c6afa 100644 --- a/rs/moq-net/src/model/subscription/window.rs +++ b/rs/moq-net/src/model/subscription/window.rs @@ -28,7 +28,7 @@ pub(in crate::model) trait Holdings { fn containing(&self, pts: Timestamp) -> Containing; /// The lowest group at or above `from` known to start after `pts`, `None` while a - /// group below it may yet start at or before `pts`. + /// sequence below it, cached or not, may yet start at or before `pts`. fn after(&self, pts: Timestamp, from: u64) -> Option; } @@ -75,7 +75,7 @@ enum End { Before(u64), /// Past every group. Open, - /// Before the first group starting after this time, once one is known and no group + /// Before the first group starting after this time, once one is known and no sequence /// below it may yet start before the time. Time(Timestamp), } diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 19fe2a61e4..de578e5bec 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -685,22 +685,42 @@ impl Holdings for TrackState { } fn after(&self, pts: Timestamp, from: u64) -> Option { + // Timestamps never rewind, so the end falls before the lowest group starting past + // `pts`, once every sequence below it, down to one starting at or before `pts`, is + // accounted for: none may still start before `pts`. let mut after = None; + // The lowest sequence walked, and whether one below `after` may still start. + let mut low = None; + let mut open = false; for (&sequence, slot) in self.lookup.range(from..).rev() { + let gap = sequence.saturating_add(1)..low.unwrap_or(sequence.saturating_add(1)); + low = Some(sequence); match slot.start() { - Some(start) if later(start, pts) => after = Some(sequence), - Some(_) => break, - // It may still start before `pts`, so nothing above it settles the end yet. - None if slot.may_start() => after = None, - // It never will, so it proves nothing either way. - None => {} + Some(start) if later(start, pts) => { + after = Some(sequence); + open = false; + } + Some(_) => return after.filter(|_| !open && self.accounted(gap)), + // Not started: one that still may blocks the end, one that never will + // proves nothing either way. + None => open |= after.is_some() && (slot.may_start() || !self.accounted(gap)), } } - after + after.filter(|_| !open && self.accounted(from..low.unwrap_or(from))) } } impl TrackState { + /// Whether no sequence in `groups` can still arrive: each arrived, was dropped, lies + /// below where the live feed starts, or was given up once the arrivals' grace ran out. + /// Ranges throughout, so a wide gap costs no more than a narrow one. + fn accounted(&self, groups: ops::Range) -> bool { + // Without a declared start, the first arrival stands in for it, as the tail's does. + let floor = self.start_sequence.or(self.arrivals.first()).unwrap_or(0); + let groups = groups.start.max(floor)..groups.end; + groups.is_empty() || self.drops.gaps(&groups).all(|gap| self.arrivals.covers(gap)) + } + /// The starts a first frame of group `sequence` must fall between: the nearest checked /// starts cached on either side, since groups land out of sequence order, and the /// eviction watermark. A group at or below the watermark is late: its floor may be gone, @@ -13893,6 +13913,46 @@ mod test { assert_eq!(drain(&mut subscriber), vec![0]); } + /// A time end stays open across a sequence yet to arrive: it may start before the end, + /// so a later group starting after it settles nothing. Once that sequence is reported + /// dropped, nothing below the later group can start before the end, so it settles. + #[test] + #[cfg(feature = "async")] + fn a_time_end_waits_on_an_absent_group_below_it() { + let at = |millis| Timestamp::from_millis(millis).unwrap(); + let ranges = [Range { + start: Anchor::Absolute(Position::group(0)), + end: Some(Anchor::Time(at(500))), + }]; + for dropped in [false, true] { + let producer = track_producer("test", Info::timed()); + let mut zero = producer.create_group(group::Info { sequence: 0 }).unwrap(); + zero.write_frame(at(0), bytes::Bytes::from_static(b"x")).unwrap(); + let mut two = producer.create_group(group::Info { sequence: 2 }).unwrap(); + two.write_frame(at(2000), bytes::Bytes::from_static(b"x")).unwrap(); + + let mut subscriber = producer.subscribe(replay().with_ranges(ranges)); + assert_eq!( + subscriber.resolved_ranges(), + vec![Some(Resolved { start: 0, end: None })], + "group 1 may still start before the end" + ); + if dropped { + producer.drop_groups(1..2); + assert_eq!(drain(&mut subscriber), vec![0]); + assert_eq!( + subscriber.resolved_ranges(), + vec![Some(Resolved { start: 0, end: Some(2) })], + "the drop settles the end" + ); + continue; + } + let mut one = producer.create_group(group::Info { sequence: 1 }).unwrap(); + one.write_frame(at(1000), bytes::Bytes::from_static(b"x")).unwrap(); + assert_eq!(drain(&mut subscriber), vec![0]); + } + } + /// An untimed track refuses a time anchor wherever the ranges reach it. #[test] #[cfg(feature = "async")] From 330ae11ef52e94fc6d54756c8c9641a2ce9649e4 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 18:48:38 -0700 Subject: [PATCH 27/36] test(moq-c): set refresh in the time anchor test's track info Co-Authored-By: Claude Opus 5.5 --- rs/moq-c/src/test.rs | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rs/moq-c/src/test.rs b/rs/moq-c/src/test.rs index 97c934bedb..c958ea50bd 100644 --- a/rs/moq-c/src/test.rs +++ b/rs/moq-c/src/test.rs @@ -1826,6 +1826,8 @@ fn raw_track_time_anchor() { max_age_present: true, timescale: 1_000_000, timescale_present: true, + refresh: 0, + refresh_present: false, }; let track = id(unsafe { moq_publish_track(broadcast, track_name.as_ptr() as *const c_char, track_name.len(), &info) }); From 9f0ec25d8a139daeee213f0b05477a2da3d4631c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 19:29:24 -0700 Subject: [PATCH 28/36] fix(moq-net): measure drift only within the groups a subscriber's ranges cover A group in the gap between two ranges is never read, so it neither sets the drift edge nor supersedes a group below it. The cursor's scalar drift cap becomes the servable window below its cap, shared with handed-out groups. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 252 ++++++++++++++++++++-------------- 1 file changed, 149 insertions(+), 103 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 4fa504529e..25ff86c29d 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -1044,8 +1044,8 @@ impl TrackState { } } - /// The live edge a subscription bounded at `cap` measures drift against: the - /// highest-sequence group that has presented a frame, or `None` while nothing + /// The live edge a subscription served `servable` measures drift against: the + /// highest-sequence group in it that has presented a frame, or `None` while nothing /// stamped is servable (an unstamped track drives no media-time expiry; the wall /// clock alone judges it). /// @@ -1053,42 +1053,49 @@ impl TrackState { /// every candidate below it and above none at or past it. Recomputing per candidate /// would make walking a backlog of N groups off in one poll cost O(N^2). /// - /// Capped because a group can only be late relative to data that would actually be + /// Bounded because a group can only be late relative to data that would actually be /// served in its place: a reader capped by [`Subscriber::set_groups`] can't jump past - /// the cap, so the groups it still wants aren't stale just because the track ran on. + /// the cap, and one reading disjoint ranges never reads the gap between them, so the + /// groups it still wants aren't stale just because the track ran on. /// Fetched backfill is absent from `arrival`, so it cannot age subscription content /// as though it were a live replacement. - fn live_edge(&self, cap: Option) -> Option { - self.lookup - .range((Bound::Unbounded, cap.map_or(Bound::Unbounded, Bound::Excluded))) - .rev() - .find_map(|(_, slot)| { - if !slot.is_shown() { - return None; - } - // The map is ordered by sequence, so the first stamped group from the - // back is the newest content that exists. - let timestamp = slot.group.timestamp()?; - Some(PresentationEdge { - sequence: slot.group.sequence, - stamp: slot.stamp, - timestamp: slot.group.latest().unwrap_or(timestamp), - }) + fn live_edge(&self, servable: &Servable) -> Option { + self.servable(servable, 0).rev().find_map(|slot| { + if !slot.is_shown() { + return None; + } + // The map is ordered by sequence, so the first stamped group from the + // back is the newest content that exists. + let timestamp = slot.group.timestamp()?; + Some(PresentationEdge { + sequence: slot.group.sequence, + stamp: slot.stamp, + timestamp: slot.group.latest().unwrap_or(timestamp), }) + }) } - /// This track's own edge under the exclusive `cap`, for measuring drift. + /// This track's own edge within `servable`, for measuring drift. /// /// None on an untimed track, where nothing is ever stale: its groups carry no /// timestamp, so looking for one would scan the whole cache on every poll. - fn drift_edge(&self, cap: Option) -> Edge { + fn drift_edge(&self, servable: &Servable) -> Edge { let untimed = self.info.as_ref().is_some_and(|info| info.timescale.is_none()); Edge { - presentation: if untimed { None } else { self.live_edge(cap) }, - cap, + presentation: if untimed { None } else { self.live_edge(servable) }, + servable: servable.clone(), } } + /// The cached groups at or above `from` that `servable` covers, lowest first: one + /// lookup per span. + fn servable<'a>(&'a self, servable: &'a Servable, from: u64) -> impl DoubleEndedIterator + 'a { + servable + .spans(from) + .flat_map(|span| self.lookup.range(span)) + .map(|(_, slot)| slot) + } + /// Whether `sequence` still holds the servable incarnation `stamp`. fn holds(&self, sequence: u64, stamp: u32) -> bool { self.lookup @@ -1103,30 +1110,24 @@ impl TrackState { /// own last timestamp says where it *starts* presenting, not where it ends; only its /// successor's start proves it cannot run past it. The bound is the *immediate* /// servable successor's start, so an unstamped successor leaves the reach unbounded - /// until it presents its first frame, and so does having no servable successor below - /// `cap` at all. - fn reach(&self, sequence: u64, cap: Option) -> Option { - self.successor(sequence, cap)?.group.timestamp() + /// until it presents its first frame, and so does having no successor in `servable` + /// at all. + fn reach(&self, sequence: u64, servable: &Servable) -> Option { + self.successor(sequence, servable)?.group.timestamp() } - /// The group after `sequence` that bounds it: the first servable one below `cap`, + /// The group after `sequence` that bounds it: the first servable one in `servable`, /// stamped or not, empty or not. - fn successor(&self, sequence: u64, cap: Option) -> Option<&Slot> { - self.lookup - .range(sequence.saturating_add(1)..) - .map(|(_, slot)| slot) - .take_while(|slot| super::subscription::before_end(slot.group.sequence, cap)) + fn successor<'a>(&'a self, sequence: u64, servable: &'a Servable) -> Option<&'a Slot> { + self.servable(servable, sequence.checked_add(1)?) .find(|slot| slot.is_shown()) } /// The group after `sequence` that starts its wall clock: like [`Self::successor`], but /// counting a group a route is still withholding. Its header arrived, so a relay ages the /// group before it as the origin does, even if its first frame never comes. - fn arrival(&self, sequence: u64, cap: Option) -> Option<&Slot> { - self.lookup - .range(sequence.saturating_add(1)..) - .map(|(_, slot)| slot) - .take_while(|slot| super::subscription::before_end(slot.group.sequence, cap)) + fn arrival<'a>(&'a self, sequence: u64, servable: &'a Servable) -> Option<&'a Slot> { + self.servable(servable, sequence.checked_add(1)?) .find(|slot| slot.is_live()) } @@ -1153,8 +1154,8 @@ impl TrackState { /// a group that lands between the two restarts the clock, and an aborted successor /// hands it to the next one. The clock is read only once there is a successor, /// so judging the newest group, the common case, costs no clock read. - fn superseded(&self, sequence: u64, cap: Option, budget: Duration) -> bool { - self.arrival(sequence, cap) + fn superseded(&self, sequence: u64, servable: &Servable, budget: Duration) -> bool { + self.arrival(sequence, servable) .is_some_and(|successor| self.aged(successor, budget)) } @@ -1199,11 +1200,11 @@ impl TrackState { /// [`Self::is_stale`] for group `sequence`, whether this track holds it or not: its /// successor here starts a full `budget` behind the edge, or arrived a full `budget` ago. fn drifted(&self, sequence: u64, edge: &Edge, budget: Duration) -> bool { - if self.superseded(sequence, edge.cap, budget) { + if self.superseded(sequence, &edge.servable, budget) { return true; } self.live_above(sequence, edge) - .is_some_and(|live| live.behind(self.reach(sequence, edge.cap), budget)) + .is_some_and(|live| live.behind(self.reach(sequence, &edge.servable), budget)) } /// The edge that could convict group `sequence`: above it, and still holding the @@ -1214,18 +1215,18 @@ impl TrackState { .filter(|live| live.sequence > sequence && self.holds(live.sequence, live.stamp)) } - /// [`Self::drifted`] under `cap`, parking `waiter` on exactly what else can move the + /// [`Self::drifted`] within `servable`, parking `waiter` on exactly what else can move the /// verdict: the successor's first frame or abort, the edge's abort, the edge crossing /// the media deadline, and the wall clock crossing the successor's arrival plus /// `budget`. Parking on every track change instead wakes each of N parked reads per append. /// /// The caller parks on a group landing above `sequence` first, in the way that fits /// whether this track holds it. - fn poll_drifted(&self, sequence: u64, cap: Option, budget: Duration, waiter: &kio::Waiter) -> bool { + fn poll_drifted(&self, sequence: u64, servable: &Servable, budget: Duration, waiter: &kio::Waiter) -> bool { // Unbounded until a group lands above. Registered before judging, so a racing abort // is observed or wakes us, and armed for whichever successor this pass selected, so // re-selecting re-arms it. - let Some(arrival) = self.watch_abort(waiter, |state| state.arrival(sequence, cap)) else { + let Some(arrival) = self.watch_abort(waiter, |state| state.arrival(sequence, servable)) else { return false; }; if let Some(deadline) = arrival.arrived.checked_add(budget) { @@ -1237,7 +1238,7 @@ impl TrackState { let reach = loop { // An abort can change reach even after the successor is stamped. Unbounded // until a group shows above, or the successor presents its first frame. - let Some(successor) = self.watch_abort(waiter, |state| state.successor(sequence, cap)) else { + let Some(successor) = self.watch_abort(waiter, |state| state.successor(sequence, servable)) else { return false; }; let successor = &successor.group; @@ -1252,7 +1253,7 @@ impl TrackState { // either seen here or wakes us. self.cache.wakes().watch_deadline(reach, budget, waiter); loop { - let edge = self.drift_edge(cap); + let edge = self.drift_edge(servable); // The edge's abort presents nothing and never touches the track, yet hands the // edge to a lower group that may already sit past the deadline. Watch it, and // re-resolve if it already landed. Terminates: an aborted group is never the @@ -3879,7 +3880,7 @@ impl Consumer { let state = state.read(); let budget = clamp_max_delay(budget, state.max_age_bound()); state.cache.wakes().watch_held(sequence, waiter); - match state.poll_drifted(sequence, None, budget, waiter) { + match state.poll_drifted(sequence, &Servable::ALL, budget, waiter) { true => Poll::Ready(()), false => Poll::Pending, } @@ -4464,7 +4465,7 @@ impl kio::Task for Fetching { let state = state.read(); let budget = clamp_max_delay(self.max_delay.unwrap_or(Duration::MAX), state.max_age_bound()); state.cache.wakes().watch_landing(sequence, waiter); - if state.poll_drifted(sequence, None, budget, waiter) { + if state.poll_drifted(sequence, &Servable::ALL, budget, waiter) { return Poll::Ready(Err(Error::Old)); } } @@ -4613,7 +4614,7 @@ enum Inner { /// One poll's view of how far this subscription may drift: the clamped budget and the /// live edge to measure a candidate group against. Resolved once, then applied to every /// group that poll considers. -#[derive(Clone, Copy)] +#[derive(Clone)] struct Drift { budget: Duration, edge: Edge, @@ -4628,8 +4629,8 @@ struct GroupExpiry { /// the last real subscriber. state: kio::ConsumerWeak, subscription: kio::Consumer, - /// The cursor's drift cap; see [`Cursor::drift_cap`]. - cap: kio::Consumer>, + /// What the cursor can be served; see [`Cursor::servable`]. + servable: kio::Consumer, sequence: u64, } @@ -4661,7 +4662,7 @@ impl group::Expiry for FetchExpiry { }; let budget = clamp_max_delay(budget, state.max_age_bound()); state.cache.wakes().watch_landing(self.sequence, waiter); - state.poll_drifted(self.sequence, None, budget, waiter) && state.lookup.contains_key(&self.sequence) + state.poll_drifted(self.sequence, &Servable::ALL, budget, waiter) && state.lookup.contains_key(&self.sequence) } } @@ -4672,7 +4673,7 @@ impl group::Expiry for GroupExpiry { Arc::new(Self { state: state.unwrap_or_else(|| self.state.clone()), subscription: self.subscription.clone(), - cap: self.cap.clone(), + servable: self.servable.clone(), sequence: self.sequence, }) } @@ -4687,28 +4688,47 @@ impl group::Expiry for GroupExpiry { max_delay }); - let mut cap = None; - let _ = self.cap.poll(waiter, |current| { - cap = **current; + let mut servable = Servable::ALL; + let _ = self.servable.poll(waiter, |current| { + servable = (**current).clone(); Poll::<()>::Pending }); let state = self.state.read(); let budget = clamp_max_delay(max_delay, state.max_age_bound()); state.cache.wakes().watch_landing(self.sequence, waiter); - state.poll_drifted(self.sequence, cap, budget, waiter) && state.lookup.contains_key(&self.sequence) + state.poll_drifted(self.sequence, &servable, budget, waiter) && state.lookup.contains_key(&self.sequence) } } /// The group a poll's drift is measured against, identified well enough to tell it apart /// from whatever may occupy its sequence by the time a candidate is judged. -#[derive(Clone, Copy)] +#[derive(Clone)] struct Edge { /// This track's own edge, revalidated before it convicts anything. presentation: Option, - /// The cap the edge was resolved under, so per-candidate reach lookups measure + /// What the edge was resolved within, so per-candidate reach lookups measure /// against the same servable window. - cap: Option, + servable: Servable, +} + +/// The groups a reader can be served, the only ones its drift is measured against: one it +/// never reads, past its cap or in a gap between its ranges, never stands in for one it does. +#[derive(Clone, PartialEq)] +struct Servable(Option>>); + +impl Servable { + /// Every group, for a reader no window narrows. + const ALL: Self = Self(None); + + /// The spans at or above `from`, lowest first. + fn spans(&self, from: u64) -> impl DoubleEndedIterator> + '_ { + let set = self.0.as_deref(); + set.into_iter() + .flat_map(move |set| set.overlapping(from..=u64::MAX)) + .map(move |span| (*span.start()).max(from)..=*span.end()) + .chain(set.is_none().then_some(from..=u64::MAX)) + } } /// The newest servable group that has presented at least one frame. @@ -4755,9 +4775,10 @@ struct Cursor { /// the producer's cache and become eligible again when the cap rises (or is removed). /// `Some(0)` is the empty range. end_sequence: Option, - /// [`Self::edge_cap`], shared with the groups this cursor hands out so their expiry - /// measures drift against the same servable window after the cap or ranges move. - drift_cap: kio::Producer>, + /// What this cursor can be served: its window below its cap. Shared with the groups it + /// hands out so their expiry measures drift against the same groups after the cap or + /// ranges move. + servable: kio::Producer, /// Groups the drift budget skipped since the count was last drained. Accumulated /// here rather than metered in place because the stats scope lives on the owning /// [`Subscriber`], which drains this after every read. @@ -4918,7 +4939,7 @@ impl Cursor { }; (filter, state.datagram_offset + start) }; - Self { + let mut cursor = Self { state, subscription, min_sequence: 0, @@ -4927,11 +4948,13 @@ impl Cursor { datagram_index, next_sequence: Some(0), end_sequence: None, - drift_cap: kio::Producer::new(None), + servable: kio::Producer::new(Servable::ALL), stale: stats::Content::default(), seek_pending: BTreeMap::new(), drops: None, - } + }; + cursor.update_servable(); + cursor } #[cfg_attr(not(feature = "async"), allow(dead_code))] @@ -4960,24 +4983,22 @@ impl Cursor { } } - /// The exclusive end drift is measured under: the cap, lowered to where the ranges - /// end, so a range behind the live edge is judged against its own newest group rather - /// than skipped for groups it never reads. - fn edge_cap(&self) -> Option { - match (self.end_sequence, self.filter.window.end()) { - (Some(cap), Some(end)) => Some(cap.min(end)), - (cap, end) => cap.or(end), - } + /// The groups drift is measured within: the window below the cap, so a range behind + /// the live edge is judged against its own newest group, and a group in a gap between + /// ranges against nothing, rather than skipped for groups it never reads. + fn servable(&self) -> Servable { + self.servable.read().clone() } - /// Publish [`Self::edge_cap`] to the groups already handed out. - fn update_drift_cap(&mut self) { - let cap = self.edge_cap(); + /// Resolve [`Self::servable`] again once the window or cap moved, and publish it to the + /// groups already handed out. Costs the ranges, never the cache. + fn update_servable(&mut self) { + let servable = Servable(Some(Arc::new(self.admitted(0, self.end_sequence)))); // Skip a no-op write: every handed-out group's expiry watches this channel. - if *self.drift_cap.read() != cap - && let Ok(mut current) = self.drift_cap.write() + if *self.servable.read() != servable + && let Ok(mut current) = self.servable.write() { - *current = cap; + *current = servable; } } @@ -5012,7 +5033,7 @@ impl Cursor { drop(state); if moved { self.filter.revision += 1; - self.update_drift_cap(); + self.update_servable(); } } @@ -5027,9 +5048,8 @@ impl Cursor { super::subscription::before_end(sequence, self.end_sequence) && self.filter.window.contains(sequence) } - /// The groups the arrival walk reads under the exclusive `cap`: the window from the - /// floor, below it. - fn admitted(&self, cap: Option) -> RangeInclusiveSet { + /// The window from `from`, below the exclusive `cap`. + fn admitted(&self, from: u64, cap: Option) -> RangeInclusiveSet { let mut admitted = RangeInclusiveSet::new(); let last = match cap { Some(end) => match end.checked_sub(1) { @@ -5038,7 +5058,7 @@ impl Cursor { }, None => u64::MAX, }; - for run in self.filter.window.runs(self.floor()) { + for run in self.filter.window.runs(from) { if *run.start() > last { break; } @@ -5073,7 +5093,8 @@ impl Cursor { if self.filter.admitted_at == Some(at) { return; } - let admitted = self.admitted(self.end_sequence); + // The arrival walk reads the window from the floor. + let admitted = self.admitted(self.floor(), self.end_sequence); self.filter.owed = self.owed(&admitted); self.filter.admitted = admitted; self.filter.admitted_at = Some(at); @@ -5134,14 +5155,14 @@ impl Cursor { /// This subscriber's clamped drift budget and the live edge to measure against, /// resolved once per poll. /// - /// `cap` bounds the edge: the caller passes the same window it reads from, so a + /// `servable` bounds the edge: the caller passes the same window it reads from, so a /// group is only ever judged against content that could actually be served in its /// place. Read fresh each poll, so a mid-stream [`Control::update`] applies /// to the very next group, and shared across every candidate that poll walks off, so /// discarding a backlog of N groups costs one scan rather than N. Only ever /// [`Poll::Ready`]; the track ending surfaces as the error the caller was going to /// get anyway. - fn poll_drift(&self, cap: Option, waiter: &kio::Waiter) -> Poll> { + fn poll_drift(&self, servable: &Servable, waiter: &kio::Waiter) -> Poll> { let mut max_delay = Duration::default(); let _ = self.subscription.poll_ref(waiter, |subscription| { max_delay = subscription.max_delay; @@ -5150,14 +5171,14 @@ impl Cursor { self.poll(waiter, |state| { Poll::Ready(Ok(Drift { budget: clamp_max_delay(max_delay, state.max_age_bound()), - edge: state.drift_edge(cap), + edge: state.drift_edge(servable), })) }) } /// Whether the drift budget says to skip `group`, against a [`Drift`] already resolved /// for this poll. - fn poll_stale(&self, group: &group::Consumer, drift: Drift, waiter: &kio::Waiter) -> Poll> { + fn poll_stale(&self, group: &group::Consumer, drift: &Drift, waiter: &kio::Waiter) -> Poll> { self.poll(waiter, |state| { Poll::Ready(Ok(state.is_stale(group.sequence, &drift.edge, drift.budget))) }) @@ -5174,21 +5195,21 @@ impl Cursor { let max_delay = self.subscription.read().max_delay; let state = self.state.read(); let budget = clamp_max_delay(max_delay, state.max_age_bound()); - let edge = state.drift_edge(self.edge_cap()); + let edge = state.drift_edge(&self.servable()); // Every group below the last successor found shares it, so a run of cached groups // that cannot serve is walked once per pass, not once per gap. let mut arrival: Option<&Slot> = None; let mut successor: Option<&Slot> = None; while let Some(oldest) = delivered.above_gap() { if arrival.is_none_or(|slot| slot.group.sequence <= oldest) { - arrival = state.arrival(oldest, edge.cap); + arrival = state.arrival(oldest, &edge.servable); } // Either clock, as `drifted` judges a group: without a successor, neither. let Some(first) = arrival else { break; }; if successor.is_none_or(|slot| slot.group.sequence <= oldest) { - successor = state.successor(oldest, edge.cap); + successor = state.successor(oldest, &edge.servable); } let past = state.aged(first, budget) || successor.is_some_and(|slot| { @@ -5208,7 +5229,7 @@ impl Cursor { group.with_expiry(Arc::new(GroupExpiry { state: self.state.weak(), subscription: self.subscription.consume(), - cap: self.drift_cap.consume(), + servable: self.servable.consume(), sequence, })) } @@ -5221,7 +5242,7 @@ impl Cursor { self.admit(); // One scan for the whole poll, so walking a backlog off stays linear in its size. - let drift = ready!(self.poll_drift(self.edge_cap(), waiter))?; + let drift = ready!(self.poll_drift(&self.servable(), waiter))?; loop { // Re-offer what a widening newly covers ahead of the arrival walk, lowest first: @@ -5269,7 +5290,7 @@ impl Cursor { // Drop a group the drift budget has given up on and keep scanning, so one // poll walks a whole backlog off rather than handing it out group by group. - if ready!(self.poll_stale(&consumer, drift, waiter))? { + if ready!(self.poll_stale(&consumer, &drift, waiter))? { self.stale.add(consumer.content()); if let Some(drops) = &mut self.drops && let Some(end) = consumer.sequence.checked_add(1) @@ -5327,7 +5348,7 @@ impl Cursor { let mut floor = next.max(self.floor()); let end = self.end_sequence; // One scan for the whole poll, so walking a backlog off stays linear in its size. - let drift = ready!(self.poll_drift(self.edge_cap(), waiter))?; + let drift = ready!(self.poll_drift(&self.servable(), waiter))?; let group = loop { let window = &self.filter.window; @@ -5353,7 +5374,7 @@ impl Cursor { // Skip a group the budget has given up on and keep scanning, so one poll // walks a whole backlog off rather than handing it out group by group. - if ready!(self.poll_stale(&group, drift, waiter))? { + if ready!(self.poll_stale(&group, &drift, waiter))? { // Not counted yet: a conviction is not permanent (the budget can widen, // the edge can be evicted), so the group may still be delivered. // Re-snapshot on every re-examination so the eventual count reflects the @@ -5586,7 +5607,7 @@ impl Subscriber { }; // What a widening owes, the walk has yet to take in, or lifting the local cap would: // a group past it is unread, not gone. - let owed = cursor.owed(&cursor.admitted(None)); + let owed = cursor.owed(&cursor.admitted(cursor.floor(), None)); let last = unread.end.map(|end| end.checked_sub(1)); let owed = || { owed.iter().any(|run| { @@ -5877,7 +5898,7 @@ impl Subscriber { match &mut self.inner { Inner::Plain(cursor) => { cursor.end_sequence = end.into().exclusive(); - cursor.update_drift_cap(); + cursor.update_servable(); } Inner::Resume(resume, _) => resume.end_at(end.into()), } @@ -9508,7 +9529,7 @@ mod test { let state = producer.state.read(); let drift = Drift { budget: Duration::ZERO, - edge: state.drift_edge(None), + edge: state.drift_edge(&Servable::ALL), }; assert!( state.is_stale(0, &drift.edge, drift.budget), @@ -9572,6 +9593,31 @@ mod test { assert_eq!(drain(&mut subscriber), vec![0, 1]); } + /// Drift is measured only against groups the ranges cover: a group in the gap between + /// two spans is never read, so it neither sets the edge nor replaces the newest group + /// below it. A group in the later span is read, so it does. + #[test] + #[cfg(feature = "async")] + fn a_gap_between_ranges_is_not_a_drift_edge() { + let ranges = || Subscription::default().with_ranges([Range::groups(0..2), Range::groups(10..12)]); + let producer = track_producer("test", Info::timed()); + for (sequence, millis) in [(0, 0), (1, 1000), (9, 9000)] { + let mut group = producer.create_group(group::Info { sequence }).unwrap(); + group + .write_frame(Timestamp::from_millis(millis).unwrap(), bytes::Bytes::from_static(b"x")) + .unwrap(); + group.finish().unwrap(); + } + // With no budget, the newest group the ranges cover is the edge. + assert_eq!(drain(&mut producer.subscribe(ranges())), vec![1]); + + let mut ten = producer.create_group(group::Info { sequence: 10 }).unwrap(); + ten.write_frame(Timestamp::from_millis(10_000).unwrap(), bytes::Bytes::from_static(b"x")) + .unwrap(); + ten.finish().unwrap(); + assert_eq!(drain(&mut producer.subscribe(ranges())), vec![10]); + } + #[test] #[cfg(feature = "async")] fn a_capped_subscriber_measures_drift_against_its_cap() { From 472a40f9cbf8746370083fb58c2f004a569b5936 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 20:09:40 -0700 Subject: [PATCH 29/36] perf(moq-net): keep a single servable run as cheap as a cap One run is held as an exclusive end, and the cursor keeps its own copy rather than reading the shared channel per poll, so the single-range drift scan is one lookup with no allocation or atomic. Disjoint runs scan one lookup each. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 125 ++++++++++++++++++++++------------ 1 file changed, 80 insertions(+), 45 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 25ff86c29d..79515f736f 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -1060,7 +1060,7 @@ impl TrackState { /// Fetched backfill is absent from `arrival`, so it cannot age subscription content /// as though it were a live replacement. fn live_edge(&self, servable: &Servable) -> Option { - self.servable(servable, 0).rev().find_map(|slot| { + self.find_servable(servable, 0, true, |slot| { if !slot.is_shown() { return None; } @@ -1087,13 +1087,39 @@ impl TrackState { } } - /// The cached groups at or above `from` that `servable` covers, lowest first: one - /// lookup per span. - fn servable<'a>(&'a self, servable: &'a Servable, from: u64) -> impl DoubleEndedIterator + 'a { - servable - .spans(from) - .flat_map(|span| self.lookup.range(span)) - .map(|(_, slot)| slot) + /// The first cached group at or above `from` that `servable` covers for which `f` + /// answers, lowest first or, with `rev`, highest first: one lookup for a single run, + /// one per run otherwise. + fn find_servable<'a, R>( + &'a self, + servable: &Servable, + from: u64, + rev: bool, + mut f: impl FnMut(&'a Slot) -> Option, + ) -> Option { + let mut scan = |range: std::collections::btree_map::Range<'a, u64, Slot>| match rev { + true => range.rev().find_map(|(_, slot)| f(slot)), + false => range.map(|(_, slot)| slot).find_map(&mut f), + }; + match servable { + Servable::Below(end) => { + // A range ending before it starts would panic, and holds nothing anyway. + if end.is_some_and(|end| end < from) { + return None; + } + let end = end.map_or(Bound::Unbounded, Bound::Excluded); + scan(self.lookup.range((Bound::Included(from), end))) + } + Servable::Runs(runs) => { + let mut runs = runs.overlapping(from..=u64::MAX); + let each = + |run: &ops::RangeInclusive| scan(self.lookup.range((*run.start()).max(from)..=*run.end())); + match rev { + true => runs.rev().find_map(each), + false => runs.find_map(each), + } + } + } } /// Whether `sequence` still holds the servable incarnation `stamp`. @@ -1118,17 +1144,19 @@ impl TrackState { /// The group after `sequence` that bounds it: the first servable one in `servable`, /// stamped or not, empty or not. - fn successor<'a>(&'a self, sequence: u64, servable: &'a Servable) -> Option<&'a Slot> { - self.servable(servable, sequence.checked_add(1)?) - .find(|slot| slot.is_shown()) + fn successor(&self, sequence: u64, servable: &Servable) -> Option<&Slot> { + self.find_servable(servable, sequence.checked_add(1)?, false, |slot| { + slot.is_shown().then_some(slot) + }) } /// The group after `sequence` that starts its wall clock: like [`Self::successor`], but /// counting a group a route is still withholding. Its header arrived, so a relay ages the /// group before it as the origin does, even if its first frame never comes. - fn arrival<'a>(&'a self, sequence: u64, servable: &'a Servable) -> Option<&'a Slot> { - self.servable(servable, sequence.checked_add(1)?) - .find(|slot| slot.is_live()) + fn arrival(&self, sequence: u64, servable: &Servable) -> Option<&Slot> { + self.find_servable(servable, sequence.checked_add(1)?, false, |slot| { + slot.is_live().then_some(slot) + }) } /// The slot `select` picks, with `waiter` parked on its abort. @@ -4629,7 +4657,7 @@ struct GroupExpiry { /// the last real subscriber. state: kio::ConsumerWeak, subscription: kio::Consumer, - /// What the cursor can be served; see [`Cursor::servable`]. + /// What the cursor can be served; see [`Cursor::published`]. servable: kio::Consumer, sequence: u64, } @@ -4715,19 +4743,27 @@ struct Edge { /// The groups a reader can be served, the only ones its drift is measured against: one it /// never reads, past its cap or in a gap between its ranges, never stands in for one it does. #[derive(Clone, PartialEq)] -struct Servable(Option>>); +enum Servable { + /// Everything below the exclusive end, or everything: one run, the common case, kept + /// as cheap to copy and look up as a cap. Its start is dropped, since every group a + /// reader is judged on lies at or above it, and so does anything measured against one. + Below(Option), + /// Disjoint runs, lowest first. + Runs(Arc>), +} impl Servable { /// Every group, for a reader no window narrows. - const ALL: Self = Self(None); + const ALL: Self = Self::Below(None); - /// The spans at or above `from`, lowest first. - fn spans(&self, from: u64) -> impl DoubleEndedIterator> + '_ { - let set = self.0.as_deref(); - set.into_iter() - .flat_map(move |set| set.overlapping(from..=u64::MAX)) - .map(move |span| (*span.start()).max(from)..=*span.end()) - .chain(set.is_none().then_some(from..=u64::MAX)) + /// The groups in `runs`. + fn new(runs: RangeInclusiveSet) -> Self { + let mut iter = runs.iter(); + match (iter.next(), iter.next()) { + (None, _) => Self::Below(Some(0)), + (Some(run), None) => Self::Below(run.end().checked_add(1)), + _ => Self::Runs(Arc::new(runs)), + } } } @@ -4775,10 +4811,13 @@ struct Cursor { /// the producer's cache and become eligible again when the cap rises (or is removed). /// `Some(0)` is the empty range. end_sequence: Option, - /// What this cursor can be served: its window below its cap. Shared with the groups it - /// hands out so their expiry measures drift against the same groups after the cap or - /// ranges move. - servable: kio::Producer, + /// What this cursor can be served, the only groups its drift is measured within: its + /// window below its cap, so a range behind the live edge is judged against its own + /// newest group, and a group in a gap between ranges against nothing. + servable: Servable, + /// [`Self::servable`], shared with the groups this cursor hands out so their expiry + /// measures drift against the same groups after the cap or ranges move. + published: kio::Producer, /// Groups the drift budget skipped since the count was last drained. Accumulated /// here rather than metered in place because the stats scope lives on the owning /// [`Subscriber`], which drains this after every read. @@ -4948,7 +4987,8 @@ impl Cursor { datagram_index, next_sequence: Some(0), end_sequence: None, - servable: kio::Producer::new(Servable::ALL), + servable: Servable::ALL, + published: kio::Producer::new(Servable::ALL), stale: stats::Content::default(), seek_pending: BTreeMap::new(), drops: None, @@ -4983,22 +5023,17 @@ impl Cursor { } } - /// The groups drift is measured within: the window below the cap, so a range behind - /// the live edge is judged against its own newest group, and a group in a gap between - /// ranges against nothing, rather than skipped for groups it never reads. - fn servable(&self) -> Servable { - self.servable.read().clone() - } - /// Resolve [`Self::servable`] again once the window or cap moved, and publish it to the /// groups already handed out. Costs the ranges, never the cache. fn update_servable(&mut self) { - let servable = Servable(Some(Arc::new(self.admitted(0, self.end_sequence)))); + let servable = Servable::new(self.admitted(0, self.end_sequence)); // Skip a no-op write: every handed-out group's expiry watches this channel. - if *self.servable.read() != servable - && let Ok(mut current) = self.servable.write() - { - *current = servable; + if self.servable == servable { + return; + } + self.servable = servable; + if let Ok(mut current) = self.published.write() { + *current = self.servable.clone(); } } @@ -5195,7 +5230,7 @@ impl Cursor { let max_delay = self.subscription.read().max_delay; let state = self.state.read(); let budget = clamp_max_delay(max_delay, state.max_age_bound()); - let edge = state.drift_edge(&self.servable()); + let edge = state.drift_edge(&self.servable); // Every group below the last successor found shares it, so a run of cached groups // that cannot serve is walked once per pass, not once per gap. let mut arrival: Option<&Slot> = None; @@ -5229,7 +5264,7 @@ impl Cursor { group.with_expiry(Arc::new(GroupExpiry { state: self.state.weak(), subscription: self.subscription.consume(), - servable: self.servable.consume(), + servable: self.published.consume(), sequence, })) } @@ -5242,7 +5277,7 @@ impl Cursor { self.admit(); // One scan for the whole poll, so walking a backlog off stays linear in its size. - let drift = ready!(self.poll_drift(&self.servable(), waiter))?; + let drift = ready!(self.poll_drift(&self.servable, waiter))?; loop { // Re-offer what a widening newly covers ahead of the arrival walk, lowest first: @@ -5348,7 +5383,7 @@ impl Cursor { let mut floor = next.max(self.floor()); let end = self.end_sequence; // One scan for the whole poll, so walking a backlog off stays linear in its size. - let drift = ready!(self.poll_drift(&self.servable(), waiter))?; + let drift = ready!(self.poll_drift(&self.servable, waiter))?; let group = loop { let window = &self.filter.window; From a4c9be0cfe7976b9eff6a92ba44599f236af8b54 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 20:16:45 -0700 Subject: [PATCH 30/36] refactor(moq-net)!: drop time anchors from subscription ranges Time ends need ongoing settlement state that belongs in its own PR, preserved on quest/m1/subscribe-ranges/time-end, so ranges here are absolute or relative only. Time starts go too, rather than a start-only anchor type in every binding. A single servable run keeps its lower bound, so a group handed out before the ranges moved above it is judged against the run, not a cached group below it. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/group.rs | 7 - rs/moq-net/src/model/resume.rs | 27 +- rs/moq-net/src/model/subscription.rs | 70 +--- rs/moq-net/src/model/subscription/window.rs | 240 ++----------- rs/moq-net/src/model/track.rs | 364 +++----------------- rs/moq-net/src/model/track/demand/ranges.rs | 5 +- 6 files changed, 102 insertions(+), 611 deletions(-) diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index 811c69dee5..8fc2bb600d 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -1478,13 +1478,6 @@ impl Producer { self.state.read().started() } - /// Whether this copy may yet present the group's frame 0: it holds it, and nothing - /// opened or ended it. - pub(super) fn may_start(&self) -> bool { - let state = self.state.read(); - state.offset == 0 && !state.started() - } - /// Where the group starts in presentation time: its first frame's timestamp, or /// `None` while no frame has been opened or when that frame is untimed. /// diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 545507768b..df63b01ccc 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -2167,23 +2167,16 @@ mod test { assert_eq!(recv(&mut sub).sequence, 6); } - /// A replacement copy keeps each range where the serving copy resolved it: the floor - /// raises a later time range's start rather than lowering it to the floor, and capping - /// the replaced copy never widens a time end. + /// A replacement copy keeps a later range disjoint: the floor raises nothing above it, + /// and capping the replaced copy never widens its end. #[test] #[cfg(feature = "async")] - fn a_replacement_keeps_a_later_time_range_disjoint() { + fn a_replacement_keeps_a_later_range_disjoint() { let routes = Producer::new(); let logical = logical(&routes); let a = copy(); routes.serve(a.consume()); - let ranges = [ - track::Range::groups(0..1), - track::Range { - start: track::Anchor::Time(ts(10_000)), - end: Some(track::Anchor::Time(ts(11_000))), - }, - ]; + let ranges = [track::Range::groups(0..1), track::Range::groups(10..12)]; let subscription = Subscription::default() .with_max_delay(Duration::from_secs(30)) .with_ranges(ranges); @@ -2219,11 +2212,11 @@ mod test { assert_eq!(rest, vec![10, 11]); } - /// A replacement updated to a time range it cannot resolve yet asks for that range, + /// A replacement updated to a relative range it cannot resolve yet asks for that range, /// not everything from the floor: before it answers, and once it answers with nothing /// to resolve it against. #[test] - fn a_replacement_keeps_an_updated_time_start() { + fn a_replacement_keeps_an_updated_relative_start() { let routes = Producer::new(); let a = copy(); routes.serve(a.consume()); @@ -2234,7 +2227,7 @@ mod test { let copy = &mut reader.copies[0]; copy.floor = Some(Position::group(0)); let ranges = vec![track::Range { - start: track::Anchor::Time(ts(10_000)), + start: track::Anchor::Relative(3), end: None, }]; let updated = Subscription::default().with_ranges(ranges.clone()); @@ -2303,7 +2296,7 @@ mod test { // The read subscribes the replacement, which has not answered yet. assert!(sub.recv_group().now_or_never().is_none()); let ranges = [track::Range { - start: track::Anchor::Time(ts(10_000)), + start: track::Anchor::Relative(2), end: None, }]; sub.update(sub.subscription().with_ranges(ranges)).unwrap(); @@ -2322,8 +2315,8 @@ mod test { assert!(sub.recv_group().now_or_never().is_none()); let start = c.subscription().expect("subscribed").ranges[0].start; assert!( - matches!(start, track::Anchor::Absolute(position) if position.group >= 10), - "the replacement asks from where b resolved the time: {start:?}" + matches!(start, track::Anchor::Absolute(_)), + "the replacement asks from where b resolved the relative start: {start:?}" ); } diff --git a/rs/moq-net/src/model/subscription.rs b/rs/moq-net/src/model/subscription.rs index e97dd02131..81f53a3846 100644 --- a/rs/moq-net/src/model/subscription.rs +++ b/rs/moq-net/src/model/subscription.rs @@ -3,10 +3,8 @@ use std::{ time::Duration, }; -use crate::Timestamp; - mod window; -pub(super) use window::{Containing, Holdings, Resolved, Window}; +pub(super) use window::{Resolved, Window}; /// Subscriber-side preferences for receiving a track. /// @@ -103,41 +101,28 @@ pub enum Order { Descending, } -/// Where one end of a [`Range`] falls: a fixed position, a count of groups back from the -/// latest one, or a presentation time. +/// Where one end of a [`Range`] falls: a fixed position, or a count of groups back from the +/// latest one. /// -/// A relative or time anchor resolves at the hop that serves it, when it starts serving -/// and again whenever the subscriber replaces its ranges. Before any group exists a -/// relative anchor resolves at the first one, so [`Range::live`] starts there. +/// A relative anchor resolves against the latest group of the hop that serves it, when it +/// starts serving and again whenever the subscriber replaces its ranges. Before any group +/// exists it resolves at the first one, so [`Range::live`] starts there. /// -/// No published wire carries either yet: upstream, a relative or time start is no floor, -/// the same request as [`Range::oldest`], and a relative or time end is no end. Behind a -/// moq-transport upstream, which joins without a floor at the live group, `Relative(n)..` -/// starts there. +/// No published wire carries one yet: upstream, a relative start is no floor, the same +/// request as [`Range::oldest`], and a relative end is no end. Behind a moq-transport +/// upstream, which joins without a floor at the live group, `Relative(n)..` starts there. #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)] pub enum Anchor { /// This exact position. Absolute(Position), /// The group this many before the latest, clamped to group 0: `Relative(0)` is the latest group. Relative(u64), - /// The group containing this presentation time: the newest one starting at or before it. - /// - /// Resolved from the starts of the groups the serving hop holds. When none starts at - /// or before it, a start reaches back to the oldest group, and while no group has - /// started at all it waits like `Relative(0)`. An end stays open until a group - /// starting after it is known and every sequence below that one is accounted for - /// (started, dropped, or never to start), so at the live edge a reader may get every - /// group handed out before its first frame showed it starts past the end, and a feed - /// that reports no drops leaves it open across a missing group until the wait for that - /// group runs out. An untimed track - /// refuses it with [`Error::TimestampMismatch`](crate::Error::TimestampMismatch). - Time(Timestamp), } /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is `None`. /// -/// Relative and time anchors cover whole groups: such a start is frame 0 of its group, and -/// such an end includes its group, so `Absolute(0)..Relative(0)` delivers through the latest +/// Relative anchors cover whole groups: a relative start is frame 0 of its group, and a +/// relative end includes its group, so `Absolute(0)..Relative(0)` delivers through the latest /// group. An absolute end is exclusive, like the end of a [`std::ops::Range`]: see /// [`Position::after`] and [`Position::after_group`]. #[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord, Hash)] @@ -245,8 +230,6 @@ pub(super) fn union(ranges: impl IntoIterator) -> Vec { (Anchor::Absolute(_), None | Some(Anchor::Absolute(_))) | (Anchor::Relative(_), None) => false, (Anchor::Absolute(start), _) => tail.is_none_or(|tail| start < tail), (Anchor::Relative(back), _) => farthest.is_none_or(|farthest| back > farthest), - // A time can't be placed against the others without the hop's groups. - (Anchor::Time(_), _) => true, })); union.extend(farthest.map(|back| Range { start: Anchor::Relative(back), @@ -363,7 +346,7 @@ impl Subscription { /// These preferences with every range raised to start no earlier than `floor` and /// stop no later than `end`, the exclusive end, without ever widening one. /// - /// `resolved` names where each range resolved, by index, so a relative or time anchor + /// `resolved` names where each range resolved, by index, so a relative anchor /// becomes its own group, raised to `floor` or capped at `end`. A start not resolved /// keeps its anchor, since the floor may lie before it, so the caller drops what it /// resolves to below `floor`; an end not resolved becomes the bound. @@ -381,15 +364,11 @@ impl Subscription { .map(|(index, range)| { let resolved = resolved.get(index).copied().flatten().filter(|_| clamping); let start = match (range.start, resolved) { - (Anchor::Relative(_) | Anchor::Time(_), Some(resolved)) => { - Anchor::Absolute(Position::group(resolved.start)) - } + (Anchor::Relative(_), Some(resolved)) => Anchor::Absolute(Position::group(resolved.start)), (start, _) => start, }; let requested = match (range.end, resolved.and_then(|resolved| resolved.end)) { - (Some(Anchor::Relative(_) | Anchor::Time(_)), Some(resolved)) => { - Some(Anchor::Absolute(Position::group(resolved))) - } + (Some(Anchor::Relative(_)), Some(resolved)) => Some(Anchor::Absolute(Position::group(resolved))), (requested, _) => requested, }; Range { @@ -408,13 +387,6 @@ impl Subscription { Self { ranges, ..self.clone() } } - /// Whether any range ends at a presentation time, which an untimed track refuses. - pub(super) fn has_time(&self) -> bool { - self.ranges - .iter() - .any(|range| matches!(range.start, Anchor::Time(_)) || matches!(range.end, Some(Anchor::Time(_)))) - } - /// Whether a subscriber joins at the live edge: a range counting back from the latest /// group and following it. A fixed start resumes a known group instead, such as a route /// switch picking up where the reader stopped. @@ -657,19 +629,7 @@ mod tests { /// group exists only the farthest offset resolves, since it is group 0 regardless. #[test] fn relative_starts_resolve_against_the_latest_group() { - struct Untimed; - impl Holdings for Untimed { - fn timed(&self) -> bool { - false - } - fn containing(&self, _: Timestamp) -> Containing { - Containing::Unknown - } - fn after(&self, _: Timestamp, _: u64) -> Option { - None - } - } - let floor = |sub: &Subscription, latest| Window::new(&sub.ranges, latest, &Untimed).unwrap().floor(); + let floor = |sub: &Subscription, latest| Window::new(&sub.ranges, latest).floor(); let live = Subscription::default().with_ranges([Range::live()]); assert_eq!(floor(&live, Some(7)), Some(7)); diff --git a/rs/moq-net/src/model/subscription/window.rs b/rs/moq-net/src/model/subscription/window.rs index d7360c6afa..8fd46645bf 100644 --- a/rs/moq-net/src/model/subscription/window.rs +++ b/rs/moq-net/src/model/subscription/window.rs @@ -9,7 +9,6 @@ use std::ops; use rangemap::RangeInclusiveSet; use super::{Anchor, Range}; -use crate::{Error, Result, Timestamp}; /// One range resolved to groups: from `start` up to the exclusive `end`, `None` while it /// stays open. @@ -19,43 +18,16 @@ pub(in crate::model) struct Resolved { pub end: Option, } -/// When a serving hop's groups start, for resolving time anchors. -pub(in crate::model) trait Holdings { - /// Whether the track carries timestamps at all. - fn timed(&self) -> bool; - - /// Where `pts` falls among the groups with a known start. - fn containing(&self, pts: Timestamp) -> Containing; - - /// The lowest group at or above `from` known to start after `pts`, `None` while a - /// sequence below it, cached or not, may yet start at or before `pts`. - fn after(&self, pts: Timestamp, from: u64) -> Option; -} - -/// Where a presentation time falls among the groups a hop holds. -#[derive(Clone, Copy, Debug, PartialEq, Eq)] -pub(in crate::model) enum Containing { - /// The newest group starting at or before it. - Group(u64), - /// Every group with a known start begins after it. - Before, - /// No group has a known start. - Unknown, -} - /// A subscriber's [`Range`]s resolved to the group sequences it reads. #[derive(Debug)] pub(in crate::model) struct Window { /// What these were resolved from, so only replacing the ranges resolves again. ranges: Vec, spans: Vec, - /// The groups the spans cover, an unresolved time end counting as open. + /// The groups the spans cover. groups: RangeInclusiveSet, /// A relative anchor waits on a first group, so nothing is filtered yet. pending: bool, - /// Nothing is left to resolve: see [`Self::is_settled`]. Kept so a read checks it once - /// per group, not once per range. - settled: bool, } /// One non-empty range, resolved. @@ -75,30 +47,19 @@ enum End { Before(u64), /// Past every group. Open, - /// Before the first group starting after this time, once one is known and no sequence - /// below it may yet start before the time. - Time(Timestamp), } impl Window { /// Resolve `ranges`, counting relative anchors back from `latest`. - /// - /// Refuses a time anchor on an untimed track with [`Error::TimestampMismatch`]. - pub fn new(ranges: &[Range], latest: Option, holdings: &impl Holdings) -> Result { + pub fn new(ranges: &[Range], latest: Option) -> Self { let mut window = Self { ranges: ranges.to_vec(), spans: Vec::new(), groups: RangeInclusiveSet::new(), pending: false, - settled: false, }; - window.place(latest, holdings)?; - Ok(window) - } - - /// Whether nothing is left to resolve as groups arrive. - pub fn is_settled(&self) -> bool { - self.settled + window.place(latest); + window } /// Whether a relative anchor still waits on a first group. @@ -106,30 +67,14 @@ impl Window { self.pending } - /// Resolve what the groups now held settle: relative anchors once `latest` exists, and - /// time ends once a group starting after them is known. Returns whether anything moved. - pub fn settle(&mut self, latest: Option, holdings: &impl Holdings) -> bool { - if self.pending { - if latest.is_none() { - return false; - } - // Checked when the ranges were first resolved, against the same track. - let _ = self.place(latest, holdings); - return !self.pending; - } - let mut moved = false; - for span in &mut self.spans { - if let End::Time(pts) = span.end - && let Some(end) = holdings.after(pts, span.start) - { - span.end = End::Before(end); - moved = true; - } + /// Resolve the relative anchors waiting on a first group once `latest` exists. + /// Returns whether anything moved. + pub fn settle(&mut self, latest: Option) -> bool { + if !self.pending || latest.is_none() { + return false; } - if moved { - self.rebuild(); - } - moved + self.place(latest); + !self.pending } /// Whether group `sequence` is in range. Everything is while a relative anchor waits. @@ -164,7 +109,7 @@ impl Window { start: span.start, end: match span.end { End::Before(end) => Some(end), - End::Open | End::Time(_) => None, + End::Open => None, }, }); } @@ -207,28 +152,14 @@ impl Window { Some((*run.start()).max(range.start)..end) } - fn place(&mut self, latest: Option, holdings: &impl Holdings) -> Result<()> { - let time = |anchor: Option| matches!(anchor, Some(Anchor::Time(_))); - if !holdings.timed() - && self - .ranges - .iter() - .any(|range| time(Some(range.start)) || time(range.end)) - { - return Err(Error::TimestampMismatch); - } + fn place(&mut self, latest: Option) { self.spans.clear(); self.pending = false; for (index, range) in self.ranges.iter().enumerate().filter(|(_, range)| !range.is_empty()) { - let (Some((start, floored)), Some(end)) = (start(range.start, latest, holdings), end(range.end, latest)) - else { + let (Some((start, floored)), Some(end)) = (start(range.start, latest), end(range.end, latest)) else { self.pending = true; continue; }; - let end = match end { - End::Time(pts) => holdings.after(pts, start).map_or(end, End::Before), - end => end, - }; self.spans.push(Span { index, start, @@ -236,12 +167,6 @@ impl Window { floored, }); } - self.rebuild(); - Ok(()) - } - - fn rebuild(&mut self) { - self.settled = !self.pending && !self.spans.iter().any(|span| matches!(span.end, End::Time(_))); self.groups = RangeInclusiveSet::new(); for span in &self.spans { let last = match span.end { @@ -249,7 +174,7 @@ impl Window { Some(last) if last >= span.start => last, _ => continue, }, - End::Open | End::Time(_) => u64::MAX, + End::Open => u64::MAX, }; self.groups.insert(span.start..=last); } @@ -258,19 +183,12 @@ impl Window { /// The first group a range starts at, and whether it names that group rather than /// reaching back to the oldest one. `None` while it waits on a latest group. -fn start(anchor: Anchor, latest: Option, holdings: &impl Holdings) -> Option<(u64, bool)> { +fn start(anchor: Anchor, latest: Option) -> Option<(u64, bool)> { match anchor { Anchor::Absolute(start) => Some((start.group, true)), // Nothing sits below group 0, so the farthest offset needs no latest group. Anchor::Relative(u64::MAX) => Some((0, false)), Anchor::Relative(back) => Some((latest?.saturating_sub(back), true)), - Anchor::Time(pts) => match holdings.containing(pts) { - Containing::Group(group) => Some((group, true)), - // The group holding it is older than any held, so reach back to the oldest. - Containing::Before => Some((0, false)), - // Nothing has started yet: like the live edge, the latest group. - Containing::Unknown => Some((latest?, true)), - }, } } @@ -283,7 +201,6 @@ fn end(anchor: Option, latest: Option) -> Option { .saturating_sub(back) .checked_add(1) .map_or(End::Open, End::Before), - Some(Anchor::Time(pts)) => End::Time(pts), }) } @@ -292,59 +209,6 @@ mod tests { use super::super::Position; use super::*; - /// Groups by sequence with their start in milliseconds, `None` for one not started. - struct Starts(Vec<(u64, Option)>); - - impl Holdings for Starts { - fn timed(&self) -> bool { - true - } - - fn containing(&self, pts: Timestamp) -> Containing { - let mut stamped = false; - for (sequence, start) in self.0.iter().rev() { - let Some(start) = start else { continue }; - stamped = true; - if Timestamp::from_millis(*start).unwrap() <= pts { - return Containing::Group(*sequence); - } - } - match stamped { - true => Containing::Before, - false => Containing::Unknown, - } - } - - fn after(&self, pts: Timestamp, from: u64) -> Option { - self.0 - .iter() - .filter(|(sequence, _)| *sequence >= from) - .find_map(|(sequence, start)| { - start - .filter(|start| Timestamp::from_millis(*start).unwrap() > pts) - .map(|_| *sequence) - }) - } - } - - struct Untimed; - - impl Holdings for Untimed { - fn timed(&self) -> bool { - false - } - fn containing(&self, _: Timestamp) -> Containing { - Containing::Unknown - } - fn after(&self, _: Timestamp, _: u64) -> Option { - None - } - } - - fn time(millis: u64) -> Anchor { - Anchor::Time(Timestamp::from_millis(millis).unwrap()) - } - fn range(start: Anchor, end: impl Into>) -> Range { Range { start, end: end.into() } } @@ -356,7 +220,7 @@ mod tests { /// Disjoint ranges filter to exactly their groups; the gap between them is out. #[test] fn ranges_cover_only_their_groups() { - let window = Window::new(&[Range::groups(2..4), Range::groups(7..)], Some(9), &Untimed).unwrap(); + let window = Window::new(&[Range::groups(2..4), Range::groups(7..)], Some(9)); assert_eq!(covered(&window, 10), vec![2, 3, 7, 8, 9]); assert_eq!(window.floor(), Some(2)); assert_eq!(window.end(), None); @@ -379,14 +243,12 @@ mod tests { Anchor::Absolute(Position { group: 3, frame: 1 }), )], None, - &Untimed, - ) - .unwrap(); + ); assert_eq!(covered(&window, 6), vec![1, 2, 3]); assert_eq!(window.end(), Some(4)); // Nothing asked for is nothing in range, floored nowhere. - let window = Window::new(&[], None, &Untimed).unwrap(); + let window = Window::new(&[], None); assert_eq!(covered(&window, 4), Vec::::new()); assert_eq!((window.floor(), window.end()), (Some(0), Some(0))); } @@ -395,18 +257,18 @@ mod tests { /// for a first group when there is none. #[test] fn relative_anchors_count_back_from_the_latest_group() { - let window = Window::new(&[range(Anchor::Relative(3), Anchor::Relative(1))], Some(7), &Untimed).unwrap(); + let window = Window::new(&[range(Anchor::Relative(3), Anchor::Relative(1))], Some(7)); assert_eq!(covered(&window, 10), vec![4, 5, 6]); - let mut window = Window::new(&[Range::live()], None, &Untimed).unwrap(); + let mut window = Window::new(&[Range::live()], None); assert!(window.is_pending() && window.contains(3) && window.floor().is_none()); - assert!(!window.settle(None, &Untimed)); - assert!(window.settle(Some(5), &Untimed)); + assert!(!window.settle(None)); + assert!(window.settle(Some(5))); assert_eq!(covered(&window, 8), vec![5, 6, 7]); - assert!(window.is_settled()); + assert!(!window.is_pending()); // The farthest offset needs no latest group, and names no floor. - let window = Window::new(&[Range::oldest()], None, &Untimed).unwrap(); + let window = Window::new(&[Range::oldest()], None); assert_eq!((window.floor(), window.start()), (Some(0), None)); } @@ -414,7 +276,7 @@ mod tests { /// group or still waits on a first one; an explicit group 0 names one. #[test] fn the_named_start_is_the_lowest_unless_one_reaches_back() { - let start = |ranges: &[Range], latest| Window::new(ranges, latest, &Untimed).unwrap().start(); + let start = |ranges: &[Range], latest| Window::new(ranges, latest).start(); assert_eq!(start(&[Range::groups(4..), Range::live()], Some(9)), Some(4)); assert_eq!(start(&[Range::groups(4..), Range::live()], Some(2)), Some(2)); assert_eq!(start(&[Range::groups(0..)], None), Some(0)); @@ -422,54 +284,4 @@ mod tests { assert_eq!(start(&[Range::groups(4..), Range::oldest()], Some(9)), None); assert_eq!(start(&[Range::groups(4..), Range::groups(..2)], Some(9)), None); } - - /// A time start is the group containing it, an end stops just past that group, and an - /// end nothing has started after yet stays open until something does. - #[test] - fn time_anchors_resolve_from_group_starts() { - let starts = Starts(vec![(2, Some(2000)), (3, Some(3000)), (5, Some(5000)), (6, None)]); - let window = Window::new(&[range(time(3500), time(4500))], Some(6), &starts).unwrap(); - assert_eq!(covered(&window, 8), vec![3, 4]); - assert!(window.is_settled()); - - // Group 6 may yet start at 5000 too, so an end there waits on it. - let window = Window::new(&[range(time(3500), time(5000))], Some(6), &starts).unwrap(); - assert!(!window.is_settled()); - assert_eq!(covered(&window, 8), vec![3, 4, 5, 6, 7]); - - // Before every start held: reach back to the oldest. - let window = Window::new(&[range(time(1000), time(2500))], Some(6), &starts).unwrap(); - assert_eq!(covered(&window, 8), vec![0, 1, 2]); - - // Past the newest start: open until a later group is known to start after it. - let mut starts = Starts(vec![(2, Some(2000)), (3, Some(3000))]); - let mut window = Window::new(&[range(time(3500), time(3600))], Some(3), &starts).unwrap(); - assert!(!window.is_settled()); - assert!(window.contains(4)); - starts.0.push((4, None)); - assert!(!window.settle(Some(4), &starts), "an unstarted group proves nothing"); - starts.0.push((5, Some(5000))); - assert!(window.settle(Some(5), &starts)); - assert_eq!(covered(&window, 8), vec![3, 4]); - - // Nothing started yet: a time start waits like the live edge. - let starts = Starts(Vec::new()); - let mut window = Window::new(&[range(time(1000), None)], None, &starts).unwrap(); - assert!(window.is_pending()); - let starts = Starts(vec![(4, None)]); - assert!(window.settle(Some(4), &starts)); - assert_eq!(window.floor(), Some(4)); - } - - #[test] - fn an_untimed_track_refuses_time_anchors() { - assert!(matches!( - Window::new(&[range(time(0), None)], None, &Untimed), - Err(Error::TimestampMismatch) - )); - assert!(matches!( - Window::new(&[range(Anchor::Relative(0), time(0))], None, &Untimed), - Err(Error::TimestampMismatch) - )); - } } diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 79515f736f..955fb7c8d9 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -22,7 +22,7 @@ use super::{Datagram, Requests}; use super::Cap; pub use super::subscription::{Anchor, Order, Position, Range, Subscription}; -use super::subscription::{Containing, Holdings, Resolved, Window}; +use super::subscription::{Resolved, Window}; use rangemap::{RangeInclusiveSet, RangeSet}; mod demand; @@ -589,20 +589,6 @@ impl Slot { } } - /// Where the group starts in presentation time, from a live copy holding frame 0. A - /// copy joined mid-group stamps a later frame, which proves nothing about the start. - fn start(&self) -> Option { - self.copies() - .filter(|group| group.live_first_frame() == Some(0)) - .find_map(group::Producer::timestamp) - } - - /// Whether the group has no start yet but may still get one: a copy holding frame 0 - /// has not presented it. - fn may_start(&self) -> bool { - self.start().is_none() && self.copies().any(group::Producer::may_start) - } - /// The checked start of whichever copy began at frame 0. fn bound(&self) -> Option { self.copies().find_map(group::Producer::bound) @@ -665,72 +651,7 @@ pub(crate) struct FetchOutcome { accepted: bool, } -/// Whether `time` is strictly after `other`, as instants: exact across scales, unlike -/// [`Timestamp`]'s `Ord`, which breaks a tie between scales. -fn later(time: Timestamp, other: Timestamp) -> bool { - u128::from(time.value()) * u128::from(other.scale().as_u64()) - > u128::from(other.value()) * u128::from(time.scale().as_u64()) -} - -/// Where the groups this hop holds start, for resolving time anchors. Walks down from the -/// newest group, so a time near the live edge costs a few groups whatever the cache holds. -impl Holdings for TrackState { - fn timed(&self) -> bool { - self.info.as_ref().is_none_or(|info| info.timescale.is_some()) - } - - fn containing(&self, pts: Timestamp) -> Containing { - let mut started = false; - for (&sequence, slot) in self.lookup.iter().rev() { - let Some(start) = slot.start() else { continue }; - if !later(start, pts) { - return Containing::Group(sequence); - } - started = true; - } - match started { - true => Containing::Before, - false => Containing::Unknown, - } - } - - fn after(&self, pts: Timestamp, from: u64) -> Option { - // Timestamps never rewind, so the end falls before the lowest group starting past - // `pts`, once every sequence below it, down to one starting at or before `pts`, is - // accounted for: none may still start before `pts`. - let mut after = None; - // The lowest sequence walked, and whether one below `after` may still start. - let mut low = None; - let mut open = false; - for (&sequence, slot) in self.lookup.range(from..).rev() { - let gap = sequence.saturating_add(1)..low.unwrap_or(sequence.saturating_add(1)); - low = Some(sequence); - match slot.start() { - Some(start) if later(start, pts) => { - after = Some(sequence); - open = false; - } - Some(_) => return after.filter(|_| !open && self.accounted(gap)), - // Not started: one that still may blocks the end, one that never will - // proves nothing either way. - None => open |= after.is_some() && (slot.may_start() || !self.accounted(gap)), - } - } - after.filter(|_| !open && self.accounted(from..low.unwrap_or(from))) - } -} - impl TrackState { - /// Whether no sequence in `groups` can still arrive: each arrived, was dropped, lies - /// below where the live feed starts, or was given up once the arrivals' grace ran out. - /// Ranges throughout, so a wide gap costs no more than a narrow one. - fn accounted(&self, groups: ops::Range) -> bool { - // Without a declared start, the first arrival stands in for it, as the tail's does. - let floor = self.start_sequence.or(self.arrivals.first()).unwrap_or(0); - let groups = groups.start.max(floor)..groups.end; - groups.is_empty() || self.drops.gaps(&groups).all(|gap| self.arrivals.covers(gap)) - } - /// A consumer joined at the live edge: ask for a group past every sequence written when /// the newest group's bytes after its first frame exceed [`Info::refresh`] times that /// frame. @@ -1102,7 +1023,8 @@ impl TrackState { false => range.map(|(_, slot)| slot).find_map(&mut f), }; match servable { - Servable::Below(end) => { + Servable::Run { start, end } => { + let from = from.max(*start); // A range ending before it starts would panic, and holds nothing anyway. if end.is_some_and(|end| end < from) { return None; @@ -4163,7 +4085,6 @@ impl Subscribing { // Wait until the track info is available let info = ready!(self.state.poll(waiter, |state| state.poll_info())) .map_err(|e| e.abort.clone().unwrap_or(Error::Dropped))??; - refuse_untimed(&info, &self.subscription.read())?; let resume = self.state.read().routes.clone(); let inner = match resume { @@ -4744,24 +4665,27 @@ struct Edge { /// never reads, past its cap or in a gap between its ranges, never stands in for one it does. #[derive(Clone, PartialEq)] enum Servable { - /// Everything below the exclusive end, or everything: one run, the common case, kept - /// as cheap to copy and look up as a cap. Its start is dropped, since every group a - /// reader is judged on lies at or above it, and so does anything measured against one. - Below(Option), + /// From `start` up to the exclusive `end`, or onward without one: one run, the common + /// case, kept as cheap to copy and look up as a cap. Its start still bounds a group + /// handed out before the run moved above it, which is judged against the run. + Run { start: u64, end: Option }, /// Disjoint runs, lowest first. Runs(Arc>), } impl Servable { /// Every group, for a reader no window narrows. - const ALL: Self = Self::Below(None); + const ALL: Self = Self::Run { start: 0, end: None }; /// The groups in `runs`. fn new(runs: RangeInclusiveSet) -> Self { let mut iter = runs.iter(); match (iter.next(), iter.next()) { - (None, _) => Self::Below(Some(0)), - (Some(run), None) => Self::Below(run.end().checked_add(1)), + (None, _) => Self::Run { start: 0, end: Some(0) }, + (Some(run), None) => Self::Run { + start: *run.start(), + end: run.end().checked_add(1), + }, _ => Self::Runs(Arc::new(runs)), } } @@ -4924,8 +4848,6 @@ struct Filter { resolved: u64, /// Bumped whenever the window moves, so the drop scan knows to look again. revision: u64, - /// Why the ranges can't be served here: a time anchor on an untimed track. - refused: Option, /// What the arrival walk reads: the window from the floor, below the cap. The walk /// passes a group outside it holding nothing, so widening it re-scans the cache. admitted: RangeInclusiveSet, @@ -4948,10 +4870,7 @@ impl Cursor { let preferences = subscription.read(); let ranges_revision = subscription.ranges_revision(); let state = state.read(); - let (window, refused) = match Window::new(&preferences.ranges, state.newest_produced, &*state) { - Ok(window) => (window, None), - Err(err) => (Window::new(&[], None, &*state).expect("nothing to refuse"), Some(err)), - }; + let window = Window::new(&preferences.ranges, state.newest_produced); // Arrival order need not be timestamp order. Only skip the initial stale prefix; // subsequent delivery stays in arrival order, including late datagrams. let newest = state.datagrams.iter().filter_map(|datagram| datagram.timestamp).max(); @@ -4971,7 +4890,6 @@ impl Cursor { window, resolved: ranges_revision, revision: 0, - refused, admitted: RangeInclusiveSet::new(), admitted_at: None, owed: RangeInclusiveSet::new(), @@ -5050,20 +4968,12 @@ impl Cursor { let moved = match replaced { Some((revision, ranges)) => { self.filter.resolved = revision; - match Window::new(&ranges, state.newest_produced, &*state) { - Ok(window) => { - self.filter.window = window; - self.filter.refused = None; - } - Err(err) => self.filter.refused = Some(err), - } + self.filter.window = Window::new(&ranges, state.newest_produced); true } - None if self.filter.window.is_settled() => false, // The track had no produced group when the ranges resolved, so the newest one // then was the first to arrive, however many have arrived or left since. - None if self.filter.window.is_pending() => self.filter.window.settle(state.first_produced, &*state), - None => self.filter.window.settle(state.newest_produced, &*state), + None => self.filter.window.settle(state.first_produced), }; drop(state); if moved { @@ -5271,9 +5181,6 @@ impl Cursor { fn poll_recv_group(&mut self, waiter: &kio::Waiter) -> Poll>> { self.sync(); - if let Some(err) = &self.filter.refused { - return Poll::Ready(Err(err.clone())); - } self.admit(); // One scan for the whole poll, so walking a backlog off stays linear in its size. @@ -5304,8 +5211,8 @@ impl Cursor { self.index = found_index + 1; // The first group to arrive resolves a relative floor, which may land - // above it, and a later one may settle where a time end falls. - if !self.filter.window.is_settled() { + // above it. + if self.filter.window.is_pending() { self.sync(); self.admit(); if consumer.sequence < self.floor() { @@ -5346,11 +5253,6 @@ impl Cursor { /// group past the cap, nothing holds it for a later raise. fn poll_recv_datagram(&mut self, waiter: &kio::Waiter) -> Poll>> { loop { - // Refuse before reading, so a refused subscription consumes no datagram. - self.sync(); - if let Some(err) = &self.filter.refused { - return Poll::Ready(Err(err.clone())); - } let Some((datagram, found_index)) = ready!(self.poll(waiter, |state| state.poll_recv_datagram(self.datagram_index))?) else { @@ -5377,9 +5279,6 @@ impl Cursor { return Poll::Ready(Ok(None)); }; self.sync(); - if let Some(err) = &self.filter.refused { - return Poll::Ready(Err(err.clone())); - } let mut floor = next.max(self.floor()); let end = self.end_sequence; // One scan for the whole poll, so walking a backlog off stays linear in its size. @@ -5397,9 +5296,8 @@ impl Cursor { }; let group = producer.consume(); - // The first group to arrive resolves a relative floor, which may land above it, and - // a later one may settle where a time end falls. - if !self.filter.window.is_settled() { + // The first group to arrive resolves a relative floor, which may land above it. + if self.filter.window.is_pending() { self.sync(); if group.sequence < self.floor() || !self.filter.window.contains(group.sequence) { floor = floor.max(self.floor()).max(group.sequence.saturating_add(1)); @@ -5458,8 +5356,6 @@ impl Cursor { #[derive(Clone)] pub struct Control { subscription: Preferences, - // Whether the track is timed, so a time anchor on an untimed one is refused. - timed: bool, } impl Control { @@ -5471,24 +5367,12 @@ impl Control { /// Replace this subscriber's preferences, updating the producer's aggregate. /// /// Returns [`Error::Closed`] if the track already ended; the update is - /// meaningless at that point and can usually be ignored. Refuses a time anchor on - /// an untimed track with [`Error::TimestampMismatch`], leaving the preferences as they were. + /// meaningless at that point and can usually be ignored. pub fn update(&self, subscription: Subscription) -> Result<()> { - if !self.timed && subscription.has_time() { - return Err(Error::TimestampMismatch); - } self.subscription.update(subscription) } } -/// Refuse `subscription` when it asks an untimed track for a presentation time. -fn refuse_untimed(info: &Info, subscription: &Subscription) -> Result<()> { - match info.timescale.is_none() && subscription.has_time() { - true => Err(Error::TimestampMismatch), - false => Ok(()), - } -} - impl Subscriber { /// The track's [`Info`], resolved when the subscription was established. /// @@ -5533,7 +5417,6 @@ impl Subscriber { Inner::Plain(cursor) => cursor.subscription.clone(), Inner::Resume(resume, _) => resume.subscription().clone(), }, - timed: self.info.timescale.is_some(), } } @@ -5812,9 +5695,7 @@ impl Subscriber { match &self.inner { Inner::Plain(cursor) => { let state = cursor.state.read(); - Window::new(ranges, state.newest_produced, &*state) - .map(|window| window.resolved()) - .unwrap_or_default() + Window::new(ranges, state.newest_produced).resolved() } Inner::Resume(..) => Vec::new(), } @@ -5952,10 +5833,8 @@ impl Subscriber { /// the cached groups they newly cover, even ones that arrived before the groups /// already read. Returns [`Error::Closed`] /// if the track already ended; the update is meaningless at that point and can usually - /// be ignored. Refuses a time anchor on an untimed track with - /// [`Error::TimestampMismatch`], leaving the preferences as they were. + /// be ignored. pub fn update(&mut self, subscription: Subscription) -> Result<()> { - refuse_untimed(&self.info, &subscription)?; let channel = match &self.inner { Inner::Plain(cursor) => &cursor.subscription, Inner::Resume(resume, _) => resume.subscription(), @@ -9653,6 +9532,33 @@ mod test { assert_eq!(drain(&mut producer.subscribe(ranges())), vec![10]); } + /// A group handed out before the ranges moved above it is judged against the new run, + /// not a cached group below its start: here group 1 would put group 0 18s behind the + /// edge, past the budget, where the run's own group 10 puts it 9s behind. + #[moq_clock::test] + #[cfg(feature = "async")] + async fn a_held_group_is_judged_against_the_moved_run() { + let at = |millis| Timestamp::from_millis(millis).unwrap(); + let budget = Subscription::default().with_max_delay(Duration::from_secs(15)); + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(budget.clone()); + let mut zero = producer.create_group(group::Info { sequence: 0 }).unwrap(); + zero.write_frame(at(0), bytes::Bytes::from_static(b"x")).unwrap(); + let mut held = subscriber.recv_group().await.unwrap().expect("group 0"); + assert!(held.read_frame().await.unwrap().is_some()); + + subscriber.update(budget.with_groups(10..20)).unwrap(); + for (sequence, millis) in [(1, 1000), (10, 10_000), (19, 19_000)] { + let mut group = producer.create_group(group::Info { sequence }).unwrap(); + group.write_frame(at(millis), bytes::Bytes::from_static(b"x")).unwrap(); + group.finish().unwrap(); + } + + let pending = moq_clock::sim::spawn(async move { held.read_frame().await }); + moq_clock::sim::yield_now().await; + assert!(!pending.is_finished(), "group 0 is within the budget of its run"); + } + #[test] #[cfg(feature = "async")] fn a_capped_subscriber_measures_drift_against_its_cap() { @@ -14068,174 +13974,4 @@ mod test { } assert_eq!(dropped, vec![2..4, 6..8]); } - - /// A time start is the cached group containing it and a time end stops just past that - /// group; one past every start held stays open until a group starts after it. - #[test] - #[cfg(feature = "async")] - fn time_anchors_resolve_against_the_cache() { - let mut producer = track_producer("test", Info::timed()); - for second in 0..4 { - append_at(&mut producer, second * 1000); - } - let time = |millis| Anchor::Time(Timestamp::from_millis(millis).unwrap()); - - let mut seek = producer.subscribe(replay().with_ranges([Range { - start: time(1500), - end: Some(time(2000)), - }])); - assert_eq!(drain(&mut seek), vec![1, 2]); - - // A start before every group reaches back to the oldest one. - let mut early = producer.subscribe(replay().with_ranges([Range { - start: Anchor::Time(Timestamp::from_micros(0).unwrap()), - end: Some(time(500)), - }])); - assert_eq!(drain(&mut early), vec![0]); - - // Past the newest start: open until a group is known to start after the end. - let mut live = producer.subscribe(replay().with_ranges([Range { - start: time(3000), - end: Some(time(4500)), - }])); - assert_eq!(drain(&mut live), vec![3]); - append_at(&mut producer, 4000); - append_at(&mut producer, 5000); - append_at(&mut producer, 6000); - assert_eq!(drain(&mut live), vec![4]); - } - - /// A group joined mid-group stamps a later frame than its start, so it never proves a - /// time end falls before it. - #[test] - #[cfg(feature = "async")] - fn a_partial_group_proves_no_start() { - let mut producer = track_producer("test", Info::timed()); - append_at(&mut producer, 0); - append_at(&mut producer, 1000); - let mut partial = producer.append_group().unwrap(); - partial.start_at(1).unwrap(); - partial - .write_frame(Timestamp::from_millis(2500).unwrap(), bytes::Bytes::from_static(b"x")) - .unwrap(); - partial.finish().unwrap(); - - let time = |millis| Anchor::Time(Timestamp::from_millis(millis).unwrap()); - let mut seek = producer.subscribe(replay().with_ranges([Range { - start: time(1500), - end: Some(time(2200)), - }])); - assert_eq!(drain(&mut seek), vec![1, 2]); - } - - /// A time end stays open across a group that has yet to start: it may start before the - /// end, so a later group starting after it settles nothing. - #[test] - #[cfg(feature = "async")] - fn a_time_end_waits_on_an_unstarted_group_below_it() { - let producer = track_producer("test", Info::timed()); - let at = |millis| Timestamp::from_millis(millis).unwrap(); - let mut zero = producer.create_group(group::Info { sequence: 0 }).unwrap(); - zero.write_frame(at(0), bytes::Bytes::from_static(b"x")).unwrap(); - let mut hole = producer.create_group(group::Info { sequence: 1 }).unwrap(); - let mut two = producer.create_group(group::Info { sequence: 2 }).unwrap(); - two.write_frame(at(2000), bytes::Bytes::from_static(b"x")).unwrap(); - - let mut subscriber = producer.subscribe(replay().with_ranges([Range { - start: Anchor::Absolute(Position::group(0)), - end: Some(Anchor::Time(at(500))), - }])); - hole.write_frame(at(1000), bytes::Bytes::from_static(b"x")).unwrap(); - assert_eq!(drain(&mut subscriber), vec![0]); - } - - /// A time end stays open across a sequence yet to arrive: it may start before the end, - /// so a later group starting after it settles nothing. Once that sequence is reported - /// dropped, nothing below the later group can start before the end, so it settles. - #[test] - #[cfg(feature = "async")] - fn a_time_end_waits_on_an_absent_group_below_it() { - let at = |millis| Timestamp::from_millis(millis).unwrap(); - let ranges = [Range { - start: Anchor::Absolute(Position::group(0)), - end: Some(Anchor::Time(at(500))), - }]; - for dropped in [false, true] { - let producer = track_producer("test", Info::timed()); - let mut zero = producer.create_group(group::Info { sequence: 0 }).unwrap(); - zero.write_frame(at(0), bytes::Bytes::from_static(b"x")).unwrap(); - let mut two = producer.create_group(group::Info { sequence: 2 }).unwrap(); - two.write_frame(at(2000), bytes::Bytes::from_static(b"x")).unwrap(); - - let mut subscriber = producer.subscribe(replay().with_ranges(ranges)); - assert_eq!( - subscriber.resolved_ranges(), - vec![Some(Resolved { start: 0, end: None })], - "group 1 may still start before the end" - ); - if dropped { - producer.drop_groups(1..2); - assert_eq!(drain(&mut subscriber), vec![0]); - assert_eq!( - subscriber.resolved_ranges(), - vec![Some(Resolved { start: 0, end: Some(2) })], - "the drop settles the end" - ); - continue; - } - let mut one = producer.create_group(group::Info { sequence: 1 }).unwrap(); - one.write_frame(at(1000), bytes::Bytes::from_static(b"x")).unwrap(); - assert_eq!(drain(&mut subscriber), vec![0]); - } - } - - /// An untimed track refuses a time anchor wherever the ranges reach it. - #[test] - #[cfg(feature = "async")] - fn an_untimed_track_refuses_time_anchors() { - let mut producer = untimed_producer(); - let timed = replay().with_ranges([Range { - start: Anchor::Time(Timestamp::from_millis(0).unwrap()), - end: None, - }]); - - let pending = producer.consume().subscribe(timed.clone()); - assert!(matches!( - pending.poll_ok(&kio::Waiter::noop()), - Poll::Ready(Err(Error::TimestampMismatch)) - )); - - let mut subscriber = producer.subscribe(replay()); - assert!(matches!( - subscriber.update(timed.clone()), - Err(Error::TimestampMismatch) - )); - assert!(matches!( - subscriber.control().update(timed.clone()), - Err(Error::TimestampMismatch) - )); - assert_eq!(subscriber.subscription(), replay(), "a refused update changes nothing"); - - let mut refused = producer.subscribe(timed); - assert!(matches!( - refused.recv_group().now_or_never(), - Some(Err(Error::TimestampMismatch)) - )); - producer.append_datagram(None, &b"x"[..]).unwrap(); - assert!(matches!( - refused.recv_datagram().now_or_never(), - Some(Err(Error::TimestampMismatch)) - )); - - // Ranges it can serve lift the refusal. - refused.update(replay()).unwrap(); - let group = producer.append_group().unwrap(); - group.finish().unwrap(); - let read = refused.recv_group().now_or_never().expect("the group is cached"); - assert_eq!( - read.unwrap().map(|group| group.sequence), - Some(1), - "the datagram took sequence 0" - ); - } } diff --git a/rs/moq-net/src/model/track/demand/ranges.rs b/rs/moq-net/src/model/track/demand/ranges.rs index 88bae86d29..120723920d 100644 --- a/rs/moq-net/src/model/track/demand/ranges.rs +++ b/rs/moq-net/src/model/track/demand/ranges.rs @@ -4,8 +4,7 @@ //! where it ends, so a position is covered while the running sum stays above zero. A treap //! keeps each subtree's sum and lowest running sum, so a change touches two boundaries and //! asks one range-minimum question, whatever else the audience holds. Ranges anchored to -//! the latest group or to a time can't be placed on that line and are counted by value -//! instead. +//! the latest group can't be placed on that line and are counted by value instead. use std::cell::OnceCell; use std::collections::BTreeMap; @@ -140,8 +139,6 @@ impl Ranges { match range.start { Anchor::Absolute(start) => self.absolute.tail().is_some_and(|tail| tail <= start), Anchor::Relative(back) => self.farthest().is_some_and(|farthest| back <= farthest), - // Where a time falls among the groups is the serving hop's to say. - Anchor::Time(_) => false, } } } From ae8eff01db1eae0125be57eb7c303f0cd7d40f80 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 20:23:47 -0700 Subject: [PATCH 31/36] fix(moq-mux): walk past gaps between a subscriber's ranges; keep the resumed floor Time anchors moved out of #5929, so the bindings drop the Time anchor and the fallible update again, and the quest records the split. The mux ordering cursor waited out its budget on sequences a subscriber's ranges exclude, which never arrive. `track::Subscriber::next_in_range` lets it step straight to the next covered group. A resumed reader caches its first answered floor, so it outlives the copy that answered it, and resets it when the ranges change. Co-Authored-By: Claude Opus 5.5 --- cpp/moq/MIGRATE.md | 4 +- dart/moq/MIGRATE.md | 4 +- dart/moq/lib/src/aliases.dart | 2 +- dart/moq_ffi/lib/src/moq.dart | 48 +---- doc/concept/moq-lite.md | 9 +- go/wrapper/MIGRATE.md | 4 +- go/wrapper/subscribe.go | 7 +- go/wrapper/types.go | 4 +- kt/moq/MIGRATE.md | 4 +- .../kotlin/dev/moq/Aliases.kt | 4 +- py/moq-rs/MIGRATE.md | 4 +- py/moq-rs/moq/subscribe.py | 5 +- quest/m1/binding-timestamps.md | 6 +- quest/m1/subscribe-ranges/README.md | 8 +- quest/m1/subscribe-ranges/dynamic.md | 11 +- quest/m1/subscribe-ranges/time-anchors.md | 56 ++++++ quest/m1/subscribe-ranges/time-end-cap.md | 34 ---- rs/moq-c/MIGRATE.md | 4 +- rs/moq-c/src/api.rs | 16 +- rs/moq-c/src/consume.rs | 91 +++++---- rs/moq-c/src/test.rs | 179 ------------------ rs/moq-ffi/MIGRATE.md | 4 +- rs/moq-ffi/src/consumer.rs | 61 ++---- rs/moq-ffi/src/media.rs | 5 +- rs/moq-ffi/src/producer.rs | 2 +- rs/moq-ffi/src/test.rs | 100 ++-------- rs/moq-mux/src/container/consumer.rs | 45 ++++- rs/moq-net/MIGRATE.md | 2 - rs/moq-net/src/model/resume.rs | 53 +++++- rs/moq-net/src/model/track.rs | 14 +- swift/MIGRATE.md | 4 +- swift/Sources/Moq/Aliases.swift | 4 +- swift/Sources/Moq/Track.swift | 7 +- 33 files changed, 282 insertions(+), 523 deletions(-) create mode 100644 quest/m1/subscribe-ranges/time-anchors.md delete mode 100644 quest/m1/subscribe-ranges/time-end-cap.md diff --git a/cpp/moq/MIGRATE.md b/cpp/moq/MIGRATE.md index 5b470aceed..b95a4e9769 100644 --- a/cpp/moq/MIGRATE.md +++ b/cpp/moq/MIGRATE.md @@ -4,11 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time (#5929) +### Ranges filter reads (#5929) A subscription now reads only the groups its own `ranges` cover. -`Anchor` gains a `Time` case holding `timestamp_us`, the group containing that presentation time, so exhaustive visits need a case for it. -`TrackConsumer::update` now returns a result, failing with `TimestampMismatch` on a time anchor an untimed track can't serve. ### Track demand reports the wanted group (#5930) diff --git a/dart/moq/MIGRATE.md b/dart/moq/MIGRATE.md index a52afe354e..3e45fc90d9 100644 --- a/dart/moq/MIGRATE.md +++ b/dart/moq/MIGRATE.md @@ -4,11 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time (#5929) +### Ranges filter reads (#5929) A subscription now reads only the groups its own `ranges` cover. -`Anchor` gains a time variant, so exhaustive switches need a case for it. -`TrackConsumer.update` now throws on a time anchor an untimed track can't serve. ### Track demand reports the wanted group (#5930) diff --git a/dart/moq/lib/src/aliases.dart b/dart/moq/lib/src/aliases.dart index ee9d97750c..a72a77f225 100644 --- a/dart/moq/lib/src/aliases.dart +++ b/dart/moq/lib/src/aliases.dart @@ -120,7 +120,7 @@ typedef Subscription = MoqSubscription; /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is null. typedef Range = MoqRange; -/// Where one end of a [Range] falls: absolute, relative to the latest group, or the group containing a time. +/// Where one end of a [Range] falls: absolute or relative to the latest group. typedef Anchor = MoqAnchor; /// Which pending group the publisher sends first. diff --git a/dart/moq_ffi/lib/src/moq.dart b/dart/moq_ffi/lib/src/moq.dart index e9c78a3a31..2947697cf5 100644 --- a/dart/moq_ffi/lib/src/moq.dart +++ b/dart/moq_ffi/lib/src/moq.dart @@ -4209,12 +4209,6 @@ class FfiConverterMoqAnchor { lifted.value, lifted.bytesRead - subview.offsetInBytes + 4, ); - case 3: - final lifted = TimeMoqAnchor.read(subview); - return LiftRetVal( - lifted.value, - lifted.bytesRead - subview.offsetInBytes + 4, - ); default: throw UniffiInternalError( UniffiInternalError.unexpectedEnumCase, @@ -4324,44 +4318,6 @@ class RelativeMoqAnchor extends MoqAnchor { } } -class TimeMoqAnchor extends MoqAnchor { - final int timestampUs; - TimeMoqAnchor(int this.timestampUs); - TimeMoqAnchor._(int this.timestampUs); - static LiftRetVal read(Uint8List buf) { - int new_offset = buf.offsetInBytes; - final timestampUs_lifted = FfiConverterUInt64.read( - Uint8List.view(buf.buffer, new_offset), - ); - final timestampUs = timestampUs_lifted.value; - new_offset += timestampUs_lifted.bytesRead; - return LiftRetVal(TimeMoqAnchor._(timestampUs), new_offset); - } - - @override - RustBuffer lower() { - final buf = Uint8List(allocationSize()); - write(buf); - return toRustBuffer(buf); - } - - @override - int allocationSize() { - return FfiConverterUInt64.allocationSize(timestampUs) + 4; - } - - @override - int write(Uint8List buf) { - buf.buffer.asByteData(buf.offsetInBytes).setInt32(0, 3); - int new_offset = buf.offsetInBytes + 4; - new_offset += FfiConverterUInt64.write( - timestampUs, - Uint8List.view(buf.buffer, new_offset), - ); - return new_offset; - } -} - enum MoqOrder { ascending, descending } class FfiConverterMoqOrder { @@ -8339,7 +8295,7 @@ class MoqTrackConsumer implements MoqTrackConsumerInterface { FfiConverterMoqSubscription.lower(subscription), status, ); - }, moqExceptionErrorHandler); + }, null); } } @@ -18474,7 +18430,7 @@ void _checkApiChecksums() { if (uniffi_moq_ffi_checksum_method_moqtrackconsumer_recv_group() != 28712) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } - if (uniffi_moq_ffi_checksum_method_moqtrackconsumer_update() != 39083) { + if (uniffi_moq_ffi_checksum_method_moqtrackconsumer_update() != 24851) { throw UniffiInternalError.panicked("UniFFI API checksum mismatch"); } if (uniffi_moq_ffi_checksum_method_moqgroupdemand_is_used() != 55523) { diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index 8931bc066c..3291f5158c 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -221,19 +221,14 @@ Each subscription carries the knobs that decide behavior under congestion: | **Priority** (0..255) | Higher-priority tracks get bandwidth first. Audio above video, base layer above enhancement. From a native Rust publisher over QUIC or WebTransport (iroh excepted), subscriptions at the same priority split bandwidth evenly by bytes, streams and datagrams alike, however many groups each has queued. A browser publisher and other transports without send groups are not covered. | | **Order** | Which group to send first when several are pending. Newest first for live (the default), oldest first for catch-up. Needs moq-lite 07 or moq-transport; older moq-lite always sends newest first. | | **Max delay** | How long a non-latest group stays worth delivering before it is skipped. Zero means "live edge only", and raising it is also what asks for history. | -| **Ranges** | Which groups to ask for: from a group, a number of groups back from the latest, or the group containing a presentation time, optionally up to an end. The default asks for everything max delay allows. Max delay still caps every range. | +| **Ranges** | Which groups to ask for: from a group, or a number of groups back from the latest, optionally up to an end. The default asks for everything max delay allows. Max delay still caps every range. | Ranges are both a request and a filter. A subscriber reads only the groups its own ranges cover, resolved again whenever it replaces them, even when another subscriber of the same track asked for more. Widening them, including lowering the start, also serves the cached groups they newly cover, never one already served. A relay merges its subscribers' ranges into one upstream request, which -today's wire versions carry as the single span covering them. A time anchor is -resolved from the groups the serving hop holds; upstream it asks for everything -max delay allows, and an untimed track refuses it. At the live edge, a time end -lets through every group handed out before its first frame showed it starts past -the end, since holding each group until its first frame arrives would cost -latency. +today's wire versions carry as the single span covering them. Order only decides what goes first, never what is skipped, so catching up oldest first needs a max delay that covers the backlog. When a relay serves subscribers of diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index a5eb141922..5dbc9ca53e 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -4,11 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time (#5929) +### Ranges filter reads (#5929) A subscription now reads only the groups its own `Ranges` cover. -`AnchorTime` names the group containing a presentation time, and type switches over `Anchor` need a case for it. -`TrackConsumer.Update` also fails on a time anchor an untimed track can't serve. ### Track demand reports the wanted group (#5930) diff --git a/go/wrapper/subscribe.go b/go/wrapper/subscribe.go index 74f3d7cdae..ce9764b6b7 100644 --- a/go/wrapper/subscribe.go +++ b/go/wrapper/subscribe.go @@ -156,14 +156,15 @@ func (t *TrackConsumer) Info() (TrackInfo, error) { return trackInfoFromFFI(info), nil } -// Update changes this subscriber's delivery preferences. It fails on a -// negative MaxDelay, or a time anchor the track can't serve. +// Update changes this subscriber's delivery preferences. It fails only on a +// negative MaxDelay. func (t *TrackConsumer) Update(subscription Subscription) error { sub, err := subscription.ffi() if err != nil { return err } - return t.inner.Update(sub) + t.inner.Update(sub) + return nil } // Groups ranges over groups in sequence order. diff --git a/go/wrapper/types.go b/go/wrapper/types.go index c55269463c..0ce689b03d 100644 --- a/go/wrapper/types.go +++ b/go/wrapper/types.go @@ -58,14 +58,12 @@ type ( // A relative start is the first frame of its group, a relative end includes its whole // group, and an absolute end is exclusive. Range = ffi.MoqRange - // Anchor is where one end of a Range falls: AnchorAbsolute, AnchorRelative, or AnchorTime. + // Anchor is where one end of a Range falls: AnchorAbsolute or AnchorRelative. Anchor = ffi.MoqAnchor // AnchorAbsolute is an exact group and frame. AnchorAbsolute = ffi.MoqAnchorAbsolute // AnchorRelative is the group Groups before the latest one, clamped to group 0: 0 is the latest. AnchorRelative = ffi.MoqAnchorRelative - // AnchorTime is the group containing TimestampUs, a presentation time in microseconds. - AnchorTime = ffi.MoqAnchorTime // Order is which of a subscription's pending groups the publisher sends first. Order = ffi.MoqOrder // VideoDecoderOutput configures what a video.Consumer delivers: an optional resize, a max delay, and whether frames keep the decoder's surface (macOS only; refused elsewhere). diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index 21287bb9c0..880c0c67e5 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -4,11 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time (#5929) +### Ranges filter reads (#5929) A subscription now reads only the groups its own `ranges` cover. -`Anchor` gains `AnchorTime`, so exhaustive `when` expressions need a branch for it. -`TrackConsumer.update` now throws `MoqException` on a time anchor an untimed track can't serve. ### Track demand reports the wanted group (#5930) diff --git a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt index 6430cf04fc..87c02cff36 100644 --- a/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt +++ b/kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt @@ -103,14 +103,12 @@ typealias Route = uniffi.moq.MoqRoute typealias Subscription = uniffi.moq.MoqSubscription /** A span of a track to deliver: from `start` up to `end`, or onward when `end` is null. */ typealias Range = uniffi.moq.MoqRange -/** Where one end of a [Range] falls: [AnchorAbsolute], [AnchorRelative], or [AnchorTime]. */ +/** Where one end of a [Range] falls: [AnchorAbsolute] or [AnchorRelative]. */ typealias Anchor = uniffi.moq.MoqAnchor /** An exact group and frame. */ typealias AnchorAbsolute = uniffi.moq.MoqAnchor.Absolute /** The group `groups` before the latest one, clamped to group 0: 0 is the latest. */ typealias AnchorRelative = uniffi.moq.MoqAnchor.Relative -/** The group containing `timestampUs`, a presentation time in microseconds. */ -typealias AnchorTime = uniffi.moq.MoqAnchor.Time /** Which pending group the publisher sends first: `ASCENDING` or `DESCENDING`. */ typealias Order = uniffi.moq.MoqOrder /** Options for fetching one past group by sequence. */ diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 695c7c7654..3a49615cdf 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -4,11 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time (#5929) +### Ranges filter reads (#5929) A subscription now reads only the groups its own `ranges` cover. -`Anchor` gains a `TIME` variant, the group containing a presentation time. -`TrackConsumer.update` now raises on a time anchor an untimed track can't serve. ### Track demand reports the wanted group (#5930) diff --git a/py/moq-rs/moq/subscribe.py b/py/moq-rs/moq/subscribe.py index c425149408..58cdd2901b 100644 --- a/py/moq-rs/moq/subscribe.py +++ b/py/moq-rs/moq/subscribe.py @@ -149,10 +149,7 @@ def info(self) -> TrackInfo: return TrackInfo._from_ffi(self._inner.info()) def update(self, subscription: Subscription) -> None: - """Change this subscriber's delivery preferences. - - Raises on a time anchor the track can't serve. - """ + """Change this subscriber's delivery preferences.""" self._inner.update(subscription._ffi()) def cancel(self) -> None: diff --git a/quest/m1/binding-timestamps.md b/quest/m1/binding-timestamps.md index 9e85c153dd..fd8058c6a2 100644 --- a/quest/m1/binding-timestamps.md +++ b/quest/m1/binding-timestamps.md @@ -16,10 +16,8 @@ Public API: breaking in every binding. Today every binding speaks microseconds: moq-ffi `timestamp_us: Option` (about 160 sites in 9 files: `MoqFrame`, `MoqDatagram`, `MoqMediaFrame`, audio -frames, json/flate producer `update`/`append`, media `flush`, the -subscription start `MoqAnchor::Time`), moq-c `timestamp_us` plus -`timestamp_present` pairs (about 140 sites in 6 files, plus -`moq_anchor.timestamp_us` read as `MOQ_ANCHOR_TIME`), and +frames, json/flate producer `update`/`append`, media `flush`), moq-c +`timestamp_us` plus `timestamp_present` pairs (about 140 sites in 6 files), and each wrapper's native type (Python `timedelta`, Go `*time.Duration`, Swift `UInt64`, Kotlin `Duration`, Dart `int`, C++ `std::optional`, OBS microseconds). Reads floor through `as_micros()`, so any scale that does not diff --git a/quest/m1/subscribe-ranges/README.md b/quest/m1/subscribe-ranges/README.md index 299fb91510..bc6094d0c2 100644 --- a/quest/m1/subscribe-ranges/README.md +++ b/quest/m1/subscribe-ranges/README.md @@ -73,7 +73,9 @@ relay bounds a sparse FETCH only if the model can request ranges: has no time filter. It is part of lite-07 and must land before [lite-07 finalizes](/quest/m1/lite07-finalize.md), which this line gates. This amends the 09-29 premise that ranges are group-only because a group's timestamp isn't always - known: timed tracks know it, and untimed ones refuse. + known: timed tracks know it, and untimed ones refuse. Split out of #5929 + (2026-10-19) into [time anchors](/quest/m1/subscribe-ranges/time-anchors.md), + which lands in its own PR. - Ranges are frame-precise (`Position`), not whole groups. The IETF joining FETCH for a mid-group SUBSCRIBE's uncached prefix stays, because today's bridge relies on it (maintainer, 09-29). @@ -86,13 +88,13 @@ every sequence delivered or dropped. ## Required -- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - `Dynamic` fills misses by range, a subscriber reads only its own ranges, and a range end may be a time +- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - `Dynamic` fills misses by range, and a subscriber reads only its own ranges - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - publishers name every group they won't deliver - [Lite-07 ranges](/quest/m1/subscribe-ranges/lite.md) - the lite-07 wire, Rust publisher and subscriber, and the draft - [moq-transport ranges](/quest/m1/subscribe-ranges/ietf.md) - range FETCH upstream per missing run, and non-blocking FETCH served downstream - [JS ranges](/quest/m1/subscribe-ranges/js.md) - `@moq/net` model and lite-07 parity - [Frame bounds](/quest/m1/subscribe-ranges/frame-bounds.md) - a local subscriber reads only the frames its ranges cover -- [Time end cap](/quest/m1/subscribe-ranges/time-end-cap.md) - at most one group past a time end at the live edge +- [Time anchors](/quest/m1/subscribe-ranges/time-anchors.md) - a range start or end may name a presentation time ## Related diff --git a/quest/m1/subscribe-ranges/dynamic.md b/quest/m1/subscribe-ranges/dynamic.md index fb2ee1b3fa..10c3e9a701 100644 --- a/quest/m1/subscribe-ranges/dynamic.md +++ b/quest/m1/subscribe-ranges/dynamic.md @@ -5,19 +5,16 @@ A publisher's `track::Dynamic` receives range requests for misses instead of one request per group, and a track serves a subscription's ranges from one cursor, in its order, without duplicating a group, naming every sequence in -range it won't deliver. A range end may be a `Time(pts)`, resolved by -whoever holds the track. `requested_group` and `fetch_group` are replaced, not +range it won't deliver. `requested_group` and `fetch_group` are replaced, not kept beside it. ## Plan A subscriber already reads only its own ranges, re-resolved when they are -replaced, and a `Time(pts)` anchor resolves from the cached groups' starts, or -is refused on an untimed track. What remains is the miss: one below the cache -becomes a range request on `Dynamic` rather than a group request. +replaced. What remains is the miss: one below the cache becomes a range request +on `Dynamic` rather than a group request. Time anchors land separately; see +[time anchors](/quest/m1/subscribe-ranges/time-anchors.md). -- Pass a time anchor the cache can't resolve to the `track::Dynamic` handler - in its range request, and test one passed through. - Replace `requested_group` and `fetch_group` rather than keep both. Update every caller (relay, moq-archive, hls, the ladder) in the same PR, including the archive consumer's time resolution from its key listing if it diff --git a/quest/m1/subscribe-ranges/time-anchors.md b/quest/m1/subscribe-ranges/time-anchors.md new file mode 100644 index 0000000000..b66dd31b47 --- /dev/null +++ b/quest/m1/subscribe-ranges/time-anchors.md @@ -0,0 +1,56 @@ +# [L] Time anchors + +## Goal + +A range start or end may name a presentation time: `Time(pts)` in the track's +timescale. A start resolves to the group containing pts, an end to just after that +group, and an untimed track refuses either. It lands in its own PR, before +[lite-07 finalizes](/quest/m1/lite07-finalize.md). + +## Plan + +Split out of #5929 (maintainer, 2026-10-19), which ships absolute and relative +anchors only. Start from the `quest/m1/subscribe-ranges/time-end` branch, which +holds that PR's time anchors through every binding: `Anchor::Time`, the +`Holdings` resolution in `Window`, the end settlement (`Holdings::after`, +`TrackState::accounted`), and the failover handling. Merge main into it and drop +what #5929 changed since. + +Decide first whether time starts ship alone. A start resolves once from the +cache, but keeping it without time ends needs a start-only anchor type in every +binding, which is why #5929 dropped both. + +Open findings from #5929's review, each needing a regression test: + +- **Per-read rescan behind a peer-controlled gap** (Codex 4320985614). Settling an + unresolved end walks back from the live edge on every unsettled read. A peer that + leaves an early sequence missing and keeps sending later groups makes each read + revisit a growing prefix until the gap's grace runs out. Keep an incremental + settle boundary, so each read scans only what arrived since. +- **Failover replacing an unsettled end** (Codex 4320985627). A copy capped at a + route switch reports an unsettled time end as unresolved, so the cap replaces + the `Time` anchor for good, and a group starting before the time that arrives + later on the old copy is delivered. Keep the time filter on the reader, separate + from the copy cap. +- **One speculative group past an end.** An unresolved end covers every sequence + through `u64::MAX`, so each group handed out before its first frame can leak past + it at the live edge. Cap it at the newest held group known to start at or before + pts, plus the first unstarted group above it, and recompute as groups start. + Raising the cap must wake readers: register on the speculative group's start + (`group::Producer::poll_started`) from both `Cursor::poll_recv_group` and + `poll_next_group`. A held group that never shows a start (a copy joined + mid-group) must not pin the cap. Datagrams are not in `lookup`, so on a + datagram-only timed track an end never resolves: resolve datagram starts too, + or leave datagram reads uncapped. Reproduced on #5929: range + `time(3000)..time(3500)` with three groups opened before any frame delivers + all three. + +Also pass a time anchor the cache can't resolve to the `track::Dynamic` handler +in its range request, once [dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) +land, and update `quest/m1/binding-timestamps.md` for the anchor's +`timestamp_us` fields. + +## Related + +- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - range requests a `Dynamic` handler resolves +- [Archive line](/quest/m1/archive/README.md) - DVR seek, which needs time anchors diff --git a/quest/m1/subscribe-ranges/time-end-cap.md b/quest/m1/subscribe-ranges/time-end-cap.md deleted file mode 100644 index 717e1e359f..0000000000 --- a/quest/m1/subscribe-ranges/time-end-cap.md +++ /dev/null @@ -1,34 +0,0 @@ -# [M] One speculative group past a time end - -## Goal - -A subscriber whose range ends at `Time(pts)` receives at most one group past that -end at the live edge, however many groups the publisher opens before their first -frames land. Today an unresolved time end covers every sequence through `u64::MAX`, -so each group handed out before its first frame can leak past the end; the docs -state that real bound (`track::Anchor::Time`, `doc/concept/moq-lite.md`). - -## Plan - -Found by Codex on #5929 and reproduced there: range `time(3000)..time(3500)` with -three groups opened before any frame delivers all three, not one. - -- `Window` caps an unresolved time end at the newest held group known to start at or - before pts, plus the first unstarted group above it, instead of `u64::MAX`. - `settle` recomputes the cap as groups start. -- Raising the cap must wake readers: register on the speculative group's start - (`group::Producer::poll_started`) from both `Cursor::poll_recv_group` and - `poll_next_group`, so a parked group past the cap is re-offered once the group - before it starts, without waiting on the next arrival. -- A held group that never shows a start (a copy joined mid-group) must not pin the - cap: a later group starting at or before pts lifts it past. -- Datagram-only timed tracks: datagrams are not in `lookup`, so a time end on them - never resolves and the cap would stay at the start group, dropping every later - datagram. Resolve datagram starts too, or leave datagram reads uncapped. - -Regression test: the #5929 repro, plus the wake (a parked group re-offered when its -predecessor's first frame lands) under mock time. - -## Related - -- [Dynamic range requests](/quest/m1/subscribe-ranges/dynamic.md) - where time anchors are resolved for a `Dynamic` handler diff --git a/rs/moq-c/MIGRATE.md b/rs/moq-c/MIGRATE.md index 93f78e4b04..6d7a0db516 100644 --- a/rs/moq-c/MIGRATE.md +++ b/rs/moq-c/MIGRATE.md @@ -4,11 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time (#5929) +### Ranges filter reads (#5929) A raw track consumer now reads only the groups its `ranges` cover, rather than every group from the lowest start to the last absolute end. -`moq_anchor` gains `timestamp_us`, read with the new `MOQ_ANCHOR_TIME` kind, which changes its ABI. -`moq_consume_track_update` fails with a negative code when the track refuses an update, such as a time anchor on an untimed track, leaving the subscription as it was. ### Demand reports the wanted group (#5930) diff --git a/rs/moq-c/src/api.rs b/rs/moq-c/src/api.rs index 82ee81bf3f..dcf61f5736 100644 --- a/rs/moq-c/src/api.rs +++ b/rs/moq-c/src/api.rs @@ -593,8 +593,8 @@ pub struct moq_subscription { /// A span of a track to deliver: from `start` up to `end`, or onward without one. /// -/// A relative or time start is the first frame of its group, and a relative or time end -/// includes its whole group. An absolute end is exclusive. +/// A relative start is the first frame of its group, and a relative end includes its whole +/// group. An absolute end is exclusive. #[repr(C)] #[allow(non_camel_case_types)] #[derive(Clone, Copy)] @@ -619,8 +619,6 @@ pub struct moq_anchor { pub group: u64, /// The frame within `group`, read only with `MOQ_ANCHOR_ABSOLUTE`. pub frame: u64, - /// The presentation time in microseconds, read only with `MOQ_ANCHOR_TIME`. - pub timestamp_us: u64, } /// How a [moq_anchor] names its position. @@ -632,8 +630,6 @@ pub enum moq_anchor_kind { MOQ_ANCHOR_ABSOLUTE = 0, /// A count of groups back from the latest one. MOQ_ANCHOR_RELATIVE = 1, - /// The group containing a presentation time. An untimed track refuses it. - MOQ_ANCHOR_TIME = 2, } /// Which of a subscription's pending groups the publisher sends first. Zero is the default. @@ -656,9 +652,6 @@ fn parse_anchor(anchor: &moq_anchor) -> Result { }) } v if v == moq_anchor_kind::MOQ_ANCHOR_RELATIVE as u32 => moq_net::track::Anchor::Relative(anchor.group), - v if v == moq_anchor_kind::MOQ_ANCHOR_TIME as u32 => { - moq_net::track::Anchor::Time(moq_net::Timestamp::from_micros(anchor.timestamp_us)?) - } _ => return Err(Error::InvalidCode), }) } @@ -3729,10 +3722,7 @@ pub unsafe extern "C" fn moq_consume_track( /// /// Pass NULL for `subscription` to reset to moq-net defaults. /// -/// Returns a zero on success, or a negative code on failure, such as a time anchor on an -/// untimed track, which leaves the subscription as it was. An update made before the -/// subscription is established applies once it is, so a refusal then ends the consumer -/// through its callback instead. +/// Returns a zero on success, or a negative code on failure. /// /// # Safety /// - The caller must ensure that subscription is either NULL or a valid pointer to a [moq_subscription] struct, diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index ab941b6e6c..ad730b082c 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -4,7 +4,7 @@ use std::{ pin::Pin, task::{Poll, ready}, }; -use tokio::sync::oneshot; +use tokio::sync::{mpsc, oneshot}; use crate::ffi::OnStatus; use crate::{ @@ -106,19 +106,11 @@ struct TaskEntry { /// A raw track task also accepts subscription updates while it is running. struct RawTaskEntry { close: Option>, - control: RawControl, + update: mpsc::UnboundedSender>, callback: OnStatus, } -/// Where a raw track's subscription updates go. -enum RawControl { - /// Waiting on SUBSCRIBE_OK, holding the latest update to apply once it arrives. - Pending(Option), - /// Subscribed: updates apply at once, so a refusal reaches the caller. - Live(moq_net::track::Control), -} - -/// Outcome of polling a raw track source alongside its close signal. +/// Outcome of polling a raw track source alongside its control channels. enum RawStep { /// A delivered value: the next group, or the next frame within a group. Item(T), @@ -661,9 +653,10 @@ impl Consume { let name = name.to_string(); let channel = oneshot::channel(); + let (update, updates) = mpsc::unbounded_channel(); let entry = RawTaskEntry { close: Some(channel.0), - control: RawControl::Pending(None), + update, callback: on_frame, }; let id = self.raw_task.insert(Some(entry))?; @@ -671,22 +664,9 @@ impl Consume { // `subscribe` blocks on SUBSCRIBE_OK, so run it inside the task. tokio::spawn(async move { let res = crate::ffi::run(driver.watch(async move { - let track = broadcast.track(&name)?.subscribe(subscription.clone()).await?.ordered(); - let control = track.control(); - Self::update_raw(&control, Self::raw_subscription(subscription))?; - // Go live under the lock, so no later update lands before the pending one. - if let Some(entry) = State::lock() - .consume - .raw_task - .get_mut(id) - .and_then(|entry| entry.as_mut()) - { - let pending = std::mem::replace(&mut entry.control, RawControl::Live(control.clone())); - if let RawControl::Pending(Some(subscription)) = pending { - Self::update_raw(&control, subscription)?; - } - } - Self::run_raw(on_frame, track, channel.1).await + let mut track = broadcast.track(&name)?.subscribe(subscription.clone()).await?.ordered(); + Self::apply_raw_subscription(&mut track, subscription); + Self::run_raw(on_frame, track, channel.1, updates).await })) .await; @@ -701,30 +681,24 @@ impl Consume { Ok(id) } - /// A raw reader's preferences, which join at the latest group unless a range names - /// a floor of its own. - fn raw_subscription(subscription: Option) -> moq_net::track::Subscription { + /// Apply a raw reader's preferences, which join at the latest group unless a range + /// names a floor of its own. + fn apply_raw_subscription(track: &mut moq_net::track::Ordered, subscription: Option) { let mut subscription = subscription.unwrap_or_default(); for range in &mut subscription.ranges { if range.start == moq_net::track::Range::oldest().start { range.start = moq_net::track::Range::live().start; } } - subscription - } - - fn update_raw(control: &moq_net::track::Control, subscription: moq_net::track::Subscription) -> Result<(), Error> { - match control.update(subscription) { - // A closed track makes the update meaningless; the reader already sees the close. - Ok(()) | Err(moq_net::Error::Closed) => Ok(()), - Err(err) => Err(err.into()), - } + // A closed track makes the update meaningless; the reader already sees the close. + let _ = track.update(subscription); } async fn run_raw( callback: OnStatus, mut track: moq_net::track::Ordered, mut close: oneshot::Receiver<()>, + mut updates: mpsc::UnboundedReceiver>, ) -> Result<(), Error> { // Deliver every frame in sequence order, reading all frames within each // group rather than the one-frame-per-group convenience. This is the @@ -736,7 +710,7 @@ impl Consume { loop { let mut group = match moq_net::kio::wait(|waiter| -> Poll, Error>> { - if Pin::new(&mut close).poll(&mut waiter.context()).is_ready() { + if Self::poll_raw_control(&mut close, &mut updates, &mut track, waiter) { return Poll::Ready(Ok(RawStep::Stop)); } match ready!(track.poll_next_group(waiter))? { @@ -753,7 +727,7 @@ impl Consume { loop { let frame = match moq_net::kio::wait(|waiter| -> Poll, Error>> { - if Pin::new(&mut close).poll(&mut waiter.context()).is_ready() { + if Self::poll_raw_control(&mut close, &mut updates, &mut track, waiter) { return Poll::Ready(Ok(RawStep::Stop)); } match ready!(group.poll_read_frame(waiter))? { @@ -777,6 +751,33 @@ impl Consume { } } + /// Poll the close and update channels, applying any subscription updates inline. + /// + /// Returns `true` when the task must stop: either the consumer was closed + /// (`close` fired) or the update channel was dropped. The caller must then + /// unwind rather than poll `close` again, since a completed oneshot panics if + /// re-polled. Borrows `track` only for the duration of the call so the caller + /// can poll a track/group source afterwards. + fn poll_raw_control( + close: &mut oneshot::Receiver<()>, + updates: &mut mpsc::UnboundedReceiver>, + track: &mut moq_net::track::Ordered, + waiter: &moq_net::kio::Waiter, + ) -> bool { + let mut cx = waiter.context(); + if Pin::new(close).poll(&mut cx).is_ready() { + return true; + } + + loop { + match updates.poll_recv(&mut cx) { + Poll::Ready(Some(subscription)) => Self::apply_raw_subscription(track, subscription), + Poll::Ready(None) => return true, + Poll::Pending => return false, + } + } + } + pub fn raw_track_update( &mut self, track: Id, @@ -787,11 +788,7 @@ impl Consume { .get_mut(track) .and_then(|entry| entry.as_mut()) .ok_or(Error::TrackNotFound)?; - let subscription = Self::raw_subscription(subscription); - match &mut entry.control { - RawControl::Pending(pending) => *pending = Some(subscription), - RawControl::Live(control) => Self::update_raw(control, subscription)?, - } + entry.update.send(subscription).map_err(|_| Error::TrackNotFound)?; Ok(()) } diff --git a/rs/moq-c/src/test.rs b/rs/moq-c/src/test.rs index c958ea50bd..3364763c90 100644 --- a/rs/moq-c/src/test.rs +++ b/rs/moq-c/src/test.rs @@ -1735,13 +1735,11 @@ fn raw_track_subscription_options_and_update() { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 1, frame: 0, - timestamp_us: 0, }, end: moq_anchor { kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, group: 2, frame: 0, - timestamp_us: 0, }, end_present: true, }]; @@ -1812,183 +1810,6 @@ fn raw_track_subscription_options_and_update() { assert_eq!(moq_origin_close(origin), 0); } -/// A time anchor starts at the group containing that presentation time. -#[test] -fn raw_track_time_anchor() { - let origin = id(moq_origin_create()); - let path = b"raw-track-time"; - let broadcast = publish_broadcast(origin, path); - - let track_name = b"data"; - let info = moq_track_info { - priority: 3, - max_age_us: 1_000_000, - max_age_present: true, - timescale: 1_000_000, - timescale_present: true, - refresh: 0, - refresh_present: false, - }; - let track = - id(unsafe { moq_publish_track(broadcast, track_name.as_ptr() as *const c_char, track_name.len(), &info) }); - - let payloads: [&[u8]; 3] = [b"zero", b"one", b"two"]; - for (i, payload) in payloads.into_iter().enumerate() { - assert_eq!( - unsafe { moq_publish_track_frame(track, payload.as_ptr(), payload.len(), i as u64 * 20_000) }, - 0 - ); - } - - let consume = request_broadcast(origin, path); - - let frame_cb = Callback::new(); - let ranges = [moq_range { - start: moq_anchor { - kind: moq_anchor_kind::MOQ_ANCHOR_TIME as u32, - group: 0, - frame: 0, - timestamp_us: 30_000, - }, - end: moq_anchor { - kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, - group: 0, - frame: 0, - timestamp_us: 0, - }, - end_present: false, - }]; - let subscription = moq_subscription { - priority: 5, - max_delay_us: 1_000_000, - ranges: ranges.as_ptr(), - ranges_len: ranges.len(), - order: 0, - }; - let consumer = id(unsafe { - moq_consume_track( - consume, - track_name.as_ptr() as *const c_char, - track_name.len(), - &subscription, - Some(channel_callback), - frame_cb.ptr, - ) - }); - - for expected in [&b"one"[..], b"two"] { - let frame_id = id(frame_cb.recv()); - let mut frame = moq_frame { - payload: std::ptr::null(), - payload_size: 0, - timestamp_us: 0, - timestamp_present: false, - keyframe: false, - }; - assert_eq!(unsafe { moq_consume_track_frame(frame_id, &mut frame) }, 0); - let received = unsafe { std::slice::from_raw_parts(frame.payload, frame.payload_size) }; - assert_eq!(received, expected); - assert_eq!(moq_consume_track_frame_free(frame_id), 0); - } - - assert_eq!(moq_consume_track_cancel(consumer), 0); - assert_eq!(frame_cb.recv_terminal(), 0); - assert_eq!(moq_publish_track_finish(track), 0); - assert_eq!(moq_consume_close(consume), 0); - assert_eq!(moq_publish_close(broadcast), 0); - assert_eq!(moq_origin_close(origin), 0); -} - -/// An untimed track refuses a time anchor update at the call, keeping its ranges. The C -/// API only publishes timed tracks, so the track comes from moq-net. -#[test] -fn raw_track_update_refuses_time_on_untimed() { - let origin = id(moq_origin_create()); - let path = b"raw-track-untimed"; - let broadcast = publish_broadcast(origin, path); - - let track_name = b"data"; - let track = State::lock() - .publish - .track( - Id::try_from(broadcast).unwrap(), - "data", - Some(moq_net::track::Info::default().with_timescale(None)), - ) - .unwrap(); - let write = |payload: &'static [u8]| { - State::lock() - .publish - .track_producer(track) - .unwrap() - .write_frame(None, payload) - .unwrap() - }; - - let consume = request_broadcast(origin, path); - let frame_cb = Callback::new(); - let consumer = id(unsafe { - moq_consume_track( - consume, - track_name.as_ptr() as *const c_char, - track_name.len(), - std::ptr::null(), - Some(channel_callback), - frame_cb.ptr, - ) - }); - - // The first frame proves the subscription is established. - write(b"zero"); - let frame_id = id(frame_cb.recv()); - assert_eq!(moq_consume_track_frame_free(frame_id), 0); - - let ranges = [moq_range { - start: moq_anchor { - kind: moq_anchor_kind::MOQ_ANCHOR_TIME as u32, - group: 0, - frame: 0, - timestamp_us: 0, - }, - end: moq_anchor { - kind: moq_anchor_kind::MOQ_ANCHOR_ABSOLUTE as u32, - group: 0, - frame: 0, - timestamp_us: 0, - }, - end_present: false, - }]; - let update = moq_subscription { - priority: 3, - max_delay_us: 1_000_000, - ranges: ranges.as_ptr(), - ranges_len: ranges.len(), - order: 0, - }; - assert!(unsafe { moq_consume_track_update(consumer, &update) } < 0); - - // The consumer keeps reading under its old ranges. - write(b"one"); - let frame_id = id(frame_cb.recv()); - let mut frame = moq_frame { - payload: std::ptr::null(), - payload_size: 0, - timestamp_us: 0, - timestamp_present: false, - keyframe: false, - }; - assert_eq!(unsafe { moq_consume_track_frame(frame_id, &mut frame) }, 0); - let received = unsafe { std::slice::from_raw_parts(frame.payload, frame.payload_size) }; - assert_eq!(received, b"one"); - assert_eq!(moq_consume_track_frame_free(frame_id), 0); - - assert_eq!(moq_consume_track_cancel(consumer), 0); - assert_eq!(frame_cb.recv_terminal(), 0); - assert_eq!(moq_consume_close(consume), 0); - assert_eq!(moq_publish_close(broadcast), 0); - assert_eq!(moq_origin_close(origin), 0); -} - #[test] fn json_snapshot_publish_consume() { let origin = id(moq_origin_create()); diff --git a/rs/moq-ffi/MIGRATE.md b/rs/moq-ffi/MIGRATE.md index d89e202a65..c71da0c5e1 100644 --- a/rs/moq-ffi/MIGRATE.md +++ b/rs/moq-ffi/MIGRATE.md @@ -5,11 +5,9 @@ These are the UniFFI names; each language wrapper has its own `MIGRATE.md` in it ## Unreleased -### Ranges filter reads, and anchors can name a time (#5929) +### Ranges filter reads (#5929) A subscription now reads only the groups its own `ranges` cover, rather than starting at the lowest one and taking whatever arrives. -`MoqAnchor` gains `Time { timestamp_us }`, the group containing that presentation time, so exhaustive matches need an arm for it. -`MoqTrackConsumer::update` now returns `Result<(), MoqError>`, failing with `TimestampMismatch` on a time anchor an untimed track can't serve. ### Track demand reports the wanted group (#5930) diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index 40be2cadba..728f08e199 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -49,8 +49,8 @@ pub struct MoqSubscription { /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is null. /// -/// A relative or time start is the first frame of its group, and a relative or time end -/// includes its whole group. An absolute end is exclusive. +/// A relative start is the first frame of its group, and a relative end includes its whole +/// group. An absolute end is exclusive. #[derive(Clone, Copy, uniffi::Record)] pub struct MoqRange { /// Where delivery starts. @@ -67,9 +67,6 @@ pub enum MoqAnchor { Absolute { group: u64, frame: u64 }, /// The group this many before the latest, clamped to group 0: `0` is the latest group. Relative { groups: u64 }, - /// The group containing this presentation time, in microseconds. An untimed track - /// refuses it. - Time { timestamp_us: u64 }, } /// Which of a subscription's pending groups the publisher sends first, mirroring @@ -82,28 +79,21 @@ pub enum MoqOrder { Descending, } -impl TryFrom for moq_net::track::Anchor { - type Error = MoqError; - - fn try_from(anchor: MoqAnchor) -> Result { - Ok(match anchor { +impl From for moq_net::track::Anchor { + fn from(anchor: MoqAnchor) -> Self { + match anchor { MoqAnchor::Absolute { group, frame } => Self::Absolute(moq_net::track::Position { group, frame }), MoqAnchor::Relative { groups } => Self::Relative(groups), - MoqAnchor::Time { timestamp_us } => { - Self::Time(moq_net::Timestamp::from_micros(timestamp_us).map_err(|_| MoqError::TimeOverflow)?) - } - }) + } } } -impl TryFrom for moq_net::track::Range { - type Error = MoqError; - - fn try_from(range: MoqRange) -> Result { - Ok(Self { - start: range.start.try_into()?, - end: range.end.map(TryInto::try_into).transpose()?, - }) +impl From for moq_net::track::Range { + fn from(range: MoqRange) -> Self { + Self { + start: range.start.into(), + end: range.end.map(Into::into), + } } } @@ -137,25 +127,18 @@ impl From for moq_net::group::Fetch { } } -impl TryFrom for moq_net::track::Subscription { - type Error = MoqError; - - fn try_from(s: MoqSubscription) -> Result { +impl From for moq_net::track::Subscription { + fn from(s: MoqSubscription) -> Self { let mut subscription = moq_net::track::Subscription::default() .with_priority(s.priority) .with_max_delay(std::time::Duration::from_micros(s.max_delay_us)); if let Some(ranges) = s.ranges { - subscription = subscription.with_ranges( - ranges - .into_iter() - .map(TryInto::try_into) - .collect::, _>>()?, - ); + subscription = subscription.with_ranges(ranges.into_iter().map(Into::into)); } if let Some(order) = s.order { subscription = subscription.with_order(order.into()); } - Ok(subscription) + subscription } } @@ -271,7 +254,7 @@ impl MoqBroadcastConsumer { ) -> Result, MoqError> { self.driver .watch(async { - let subscription = subscription.map(moq_net::track::Subscription::try_from).transpose()?; + let subscription = subscription.map(moq_net::track::Subscription::from); let track = self.inner.track(&name)?.subscribe(subscription).await?; Ok(Arc::new(MoqTrackConsumer::new(track, self.driver.clone()))) }) @@ -681,13 +664,9 @@ impl MoqTrackConsumer { /// Change this subscriber's delivery preferences. /// /// Silently ignored if the track already ended; the update is meaningless at - /// that point. Errors on a time anchor the track can't serve, leaving the - /// preferences as they were. - pub fn update(&self, subscription: MoqSubscription) -> Result<(), MoqError> { - match self.control.update(subscription.try_into()?) { - Ok(()) | Err(moq_net::Error::Closed) => Ok(()), - Err(err) => Err(err.into()), - } + /// that point. + pub fn update(&self, subscription: MoqSubscription) { + let _ = self.control.update(subscription.into()); } /// Cancel all current and future reads. diff --git a/rs/moq-ffi/src/media.rs b/rs/moq-ffi/src/media.rs index 2ada402823..23e05fc653 100644 --- a/rs/moq-ffi/src/media.rs +++ b/rs/moq-ffi/src/media.rs @@ -1330,10 +1330,7 @@ impl MoqMediaTrackConsumer { // subscription if init parsing fails. let media = media_container(container)?; let resolved = broadcast.resolve_inner(reference.as_deref()).await?; - let subscription = subscription - .map(moq_net::track::Subscription::try_from) - .transpose()? - .unwrap_or_default(); + let subscription = subscription.map(moq_net::track::Subscription::from).unwrap_or_default(); let track = broadcast .driver .watch(async { Ok(resolved.track(&name)?.subscribe(subscription).await?) }) diff --git a/rs/moq-ffi/src/producer.rs b/rs/moq-ffi/src/producer.rs index 168131edd7..e079850664 100644 --- a/rs/moq-ffi/src/producer.rs +++ b/rs/moq-ffi/src/producer.rs @@ -525,7 +525,7 @@ impl MoqTrackProducer { let _guard = crate::ffi::enter(); let guard = self.inner.lock().unwrap(); let track = guard.as_ref().ok_or(MoqError::Closed)?; - let subscription = subscription.map(moq_net::track::Subscription::try_from).transpose()?; + let subscription = subscription.map(moq_net::track::Subscription::from); Ok(Arc::new(MoqTrackConsumer::new( track.subscribe(subscription), Background::default(), diff --git a/rs/moq-ffi/src/test.rs b/rs/moq-ffi/src/test.rs index a24faffb92..58ea35e056 100644 --- a/rs/moq-ffi/src/test.rs +++ b/rs/moq-ffi/src/test.rs @@ -915,17 +915,15 @@ async fn raw_track_update_does_not_wait_for_pending_read() { tokio::spawn(async move { consumer.read_frame().await }) }; - consumer - .update(MoqSubscription { - priority: 10, - max_delay_us: 25_000, - ranges: Some(vec![MoqRange { - start: MoqAnchor::Absolute { group: 0, frame: 0 }, - end: None, - }]), - order: None, - }) - .unwrap(); + consumer.update(MoqSubscription { + priority: 10, + max_delay_us: 25_000, + ranges: Some(vec![MoqRange { + start: MoqAnchor::Absolute { group: 0, frame: 0 }, + end: None, + }]), + order: None, + }); let payload = b"updated subscription".to_vec(); track @@ -3910,17 +3908,15 @@ async fn raw_track_update_during_pending_group_still_reads_datagram() { spawn_parked(async move { consumer.next_group().await }).await }; - consumer - .update(MoqSubscription { - priority: 10, - max_delay_us: 25_000, - ranges: Some(vec![MoqRange { - start: MoqAnchor::Absolute { group: 0, frame: 0 }, - end: None, - }]), - order: None, - }) - .unwrap(); + consumer.update(MoqSubscription { + priority: 10, + max_delay_us: 25_000, + ranges: Some(vec![MoqRange { + start: MoqAnchor::Absolute { group: 0, frame: 0 }, + end: None, + }]), + order: None, + }); let sequence = track.append_datagram(datagram_frame(b"after-update")).unwrap(); let datagram = tokio::time::timeout(TIMEOUT, consumer.recv_datagram()) @@ -6715,7 +6711,7 @@ async fn ascending_subscription_reads_a_cached_backlog_oldest_first() { }]), order: Some(MoqOrder::Ascending), }; - let converted = moq_net::track::Subscription::try_from(subscription.clone()).unwrap(); + let converted = moq_net::track::Subscription::from(subscription.clone()); assert_eq!(converted.order, moq_net::track::Order::Ascending); assert_eq!( converted.ranges, @@ -6737,61 +6733,3 @@ async fn ascending_subscription_reads_a_cached_backlog_oldest_first() { } assert_eq!(sequences, vec![0, 1, 2]); } - -/// A time anchor starts at the group containing it, and an untimed track refuses one. -#[tokio::test] -async fn time_anchors_start_at_the_containing_group() { - let broadcast = MoqBroadcastProducer::new().unwrap(); - let track = broadcast.publish_track("events".into(), Some(timed_info())).unwrap(); - for timestamp_us in [0, 1_000, 2_000] { - let group = track.append_group().unwrap(); - group - .write_frame(MoqFrame { - payload: b"x".to_vec(), - timestamp_us: Some(timestamp_us), - }) - .unwrap(); - group.finish().unwrap(); - } - - let since = |timestamp_us| MoqSubscription { - priority: 0, - max_delay_us: 10_000_000, - ranges: Some(vec![MoqRange { - start: MoqAnchor::Time { timestamp_us }, - end: None, - }]), - order: Some(MoqOrder::Ascending), - }; - let consumer = track.consume(Some(since(1_500))).unwrap(); - let mut sequences = Vec::new(); - for _ in 0..2 { - let group = tokio::time::timeout(TIMEOUT, consumer.recv_group()) - .await - .expect("timed out waiting for a cached group") - .unwrap() - .expect("expected a group"); - sequences.push(group.sequence()); - } - assert_eq!(sequences, vec![1, 2]); - - let untimed = broadcast - .publish_track( - "status".into(), - Some(MoqTrackInfo { - timescale: None, - ..timed_info() - }), - ) - .unwrap(); - let consumer = untimed.consume(None).unwrap(); - assert!(matches!( - consumer.update(since(0)), - Err(MoqError::Protocol { - details: MoqProtocolError { - kind: MoqProtocolKind::TimestampMismatch, - .. - } - }) - )); -} diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 87332d8a1a..90531322d8 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -496,12 +496,19 @@ impl Consumer { // the gap outright, since no new group can arrive. That proof covers only // sequences that never arrived: finishing the track ends new groups, not // the frames still flowing on ones already open, so arrived groups are - // settled below by their own FIN, abort, or the budget. + // settled below by their own FIN, abort, or the budget. Sequences the + // subscriber's own ranges exclude never arrive either, so a gap between two + // ranges closes at once. if let Some(front_sequence) = self.pending.front().map(|g| g.sequence) && front_sequence > self.current && let Some((_, next_start)) = next_group - && (finished || max_timestamp.saturating_sub(next_start) >= self.max_delay) - { + && (finished + || max_timestamp.saturating_sub(next_start) >= self.max_delay + || self.track.as_ref().is_some_and(|track| { + track + .next_in_range(self.current) + .is_none_or(|next| next >= front_sequence) + })) { // A data marker has no timestamp to show the missing sequences held nothing. let bare = self.pending.front().is_some_and(|group| group.bare); if bare || !pts_contiguous(current_end.or(self.presented_end), next_start) { @@ -3249,6 +3256,38 @@ mod tests { assert_eq!(micros, vec![0, 100_000], "the open group's late frames are delivered"); } + /// A gap between two ranges is never read, so the reader walks over it at once rather + /// than waiting a budget, or forever on an open track, for groups it never gets. + #[tokio::test] + async fn an_excluded_gap_between_ranges_does_not_stall() { + tokio::time::pause(); + let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); + let subscription = moq_net::track::Subscription::default() + .with_max_delay(Duration::from_secs(10)) + .with_ranges([ + moq_net::track::Range::groups(0..1), + moq_net::track::Range::groups(10..11), + ]); + let mut consumer = Consumer::new( + track.subscribe(subscription), + Container::Legacy(crate::container::Kind::Data), + ); + for sequence in [0, 5, 10] { + write_group(&mut track, sequence, &[ts(sequence * 100_000)]); + } + + let mut micros = Vec::new(); + for _ in 0..2 { + let frame = tokio::time::timeout(Duration::from_millis(200), consumer.read_frame()) + .await + .expect("the open track still delivers the later range") + .unwrap() + .expect("a frame"); + micros.push(frame.timestamp.as_micros()); + } + assert_eq!(micros, vec![0, 1_000_000]); + } + /// A relative start names its floor once resolved: delivery starts that many groups /// back from the latest, rather than at whichever group arrives first. #[tokio::test] diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index 7c4e788c72..9e3e6ff40c 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -8,8 +8,6 @@ How to update your code for each breaking change, newest first. A `track::Subscriber` now reads only the groups its own `Subscription::ranges` cover, resolved again whenever `update` replaces them; before, they only set where its reads started. A reader that relied on groups another subscriber asked for must widen its own ranges. -`track::Anchor` gains `Time(Timestamp)`, the group containing a presentation time, so exhaustive matches need an arm for it. -Subscribing or updating with one on an untimed track fails with `Error::TimestampMismatch`. Once a track finishes, a subscriber with nothing left under its `set_groups` cap ends rather than waiting for the cap to rise; raise it before the track finishes to read past it. ### Track demand reports the wanted group (#5930) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index df63b01ccc..e1480f871d 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -176,6 +176,7 @@ impl Consumer { generation: None, copies: Vec::new(), newest: None, + floor: None, datagram: None, groups: (0, Cap::from(..)), groups_read: false, @@ -428,12 +429,17 @@ impl Subscriber { } /// The floor the first copy answered with; see [`track::Subscriber::floor`]. Later - /// copies start where the reader got to, which names no floor of the reader's own. + /// copies start where the reader got to, which names no floor of the reader's own, so + /// it is kept once answered, past the first copy being let go. pub(super) fn floor(&self) -> Option { - self.reader().copies.iter().find_map(|copy| match ©.sub { - Sub::Ready(sub) if copy.floor.is_none() => Some(sub.floor()), - _ => None, - })? + let mut reader = self.reader(); + if reader.floor.is_none() { + reader.floor = reader.copies.iter().find_map(|copy| match ©.sub { + Sub::Ready(sub) if copy.floor.is_none() => Some(sub.floor()), + _ => None, + }); + } + reader.floor.flatten() } /// Poll for sequences the serving copy gave up; see [`track::Subscriber::poll_drop`]. @@ -522,6 +528,8 @@ struct Reader { copies: Vec, /// The newest group handed out: where a new copy is asked to start. newest: Option, + /// The floor the first copy answered for the current ranges, once it answered. + floor: Option>, /// The newest datagram handed out, by Group and Object ID. datagram: Option<(u64, u64)>, /// The local group filter; see [`track::Subscriber::set_groups`]. @@ -544,6 +552,9 @@ impl Reader { Poll::<()>::Pending }); if let Some(sub) = changed { + if sub.ranges != self.mirrored.ranges { + self.floor = None; + } self.mirrored = sub; for copy in &mut self.copies { copy.update(&self.mirrored); @@ -3312,6 +3323,38 @@ mod test { } } + /// The floor the first copy answered stays once a route switch lets that copy go. + #[test] + #[cfg(feature = "async")] + fn the_floor_outlives_the_first_copy() { + let routes = Producer::new(); + let logical = logical(&routes); + let a = copy(); + routes.serve(a.consume()); + let subscription = Subscription::default() + .with_max_delay(Duration::from_secs(10)) + .with_groups(5..); + let mut sub = logical + .consume() + .subscribe(subscription) + .now_or_never() + .unwrap() + .unwrap(); + write(&a, 5); + assert_eq!(drain(&mut sub, false), [5]); + assert_eq!(sub.floor(), Some(5)); + + let b = copy(); + routes.serve(b.consume()); + write(&b, 6); + assert_eq!(drain(&mut sub, false), [6]); + let readers = routes.readers.lock().unwrap(); + let reader = readers[0].upgrade().unwrap(); + assert_eq!(reader.lock().unwrap().copies.len(), 1, "the first copy was let go"); + drop(readers); + assert_eq!(sub.floor(), Some(5)); + } + #[test] #[cfg(feature = "async")] fn datagram_readers_release_replaced_subscriptions() { diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 955fb7c8d9..3f528b1ae4 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4514,8 +4514,7 @@ impl kio::Task for Fetching { /// publisher** and **this subscriber's read filter**. The publisher sees the union across /// every live subscriber, but each subscriber reads only the groups its own ranges cover, /// resolved where it is served: a relative anchor counted back from the latest group (or -/// the first to arrive, before any exists) and a time anchor from the cached groups' -/// starts, again whenever the ranges are replaced. +/// the first to arrive, before any exists), again whenever the ranges are replaced. /// - [`Self::set_groups`] narrows **this subscriber's reads** further, without changing the /// publisher's demand. /// @@ -5678,6 +5677,17 @@ impl Subscriber { } } + /// The lowest group at or above `from` this subscriber's ranges cover, `None` when + /// none does: the groups between are never read, so a reader waiting on each group in + /// sequence need not wait for them. `Some(from)` while a relative start waits on the + /// first group, and for a reader that follows a route, which can't say. + pub fn next_in_range(&self, from: u64) -> Option { + match &self.inner { + Inner::Plain(cursor) => cursor.filter.window.runs(from).next().map(|run| *run.start()), + Inner::Resume(..) => Some(from), + } + } + /// This subscriber's own ranges, by index, as resolved here: `None` for one empty or /// still waiting, and for every range of a front's reader. pub(super) fn resolved_ranges(&mut self) -> Vec> { diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 85431c7543..4a3ab3425d 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -4,11 +4,9 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Ranges filter reads, and anchors can name a time (#5929) +### Ranges filter reads (#5929) A subscription now reads only the groups its own `ranges` cover. -`Subscription.Anchor` gains `.time(timestampUs:)`, so exhaustive switches need a case for it. -`TrackConsumer.update(subscription:)` now `throws`, on a time anchor an untimed track can't serve; call it with `try`. ### Track demand reports the wanted group (#5930) diff --git a/swift/Sources/Moq/Aliases.swift b/swift/Sources/Moq/Aliases.swift index 9b9fd71b54..97c8cce74e 100644 --- a/swift/Sources/Moq/Aliases.swift +++ b/swift/Sources/Moq/Aliases.swift @@ -115,8 +115,8 @@ public typealias ProtocolError = MoqFFI.MoqProtocolError extension Subscription { /// A span of a track to deliver: from `start` up to `end`, or onward when `end` is `nil`. public typealias Range = MoqFFI.MoqRange - /// Where one end of a `Range` falls: `.absolute(group:frame:)`, `.relative(groups:)` - /// back from the latest group, or `.time(timestampUs:)` for the group containing it. + /// Where one end of a `Range` falls: `.absolute(group:frame:)` or + /// `.relative(groups:)` back from the latest group. public typealias Anchor = MoqFFI.MoqAnchor /// Which pending group the publisher sends first: `.ascending` or `.descending`. public typealias Order = MoqFFI.MoqOrder diff --git a/swift/Sources/Moq/Track.swift b/swift/Sources/Moq/Track.swift index efd34bfe13..c4566ba328 100644 --- a/swift/Sources/Moq/Track.swift +++ b/swift/Sources/Moq/Track.swift @@ -44,10 +44,9 @@ public final class TrackConsumer: AsyncSequence, Sendable { try ffi.info() } - /// Change this subscriber's delivery preferences. Throws on a time anchor the track - /// can't serve. - public func update(subscription: Subscription) throws { - try ffi.update(subscription: subscription) + /// Change this subscriber's delivery preferences. + public func update(subscription: Subscription) { + ffi.update(subscription: subscription) } /// Cancel all current and future reads. From 1a3cf34b99e3ba763baab957b2f3ec4dba7a4f82 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 21:08:29 -0700 Subject: [PATCH 32/36] fix(moq-net): name the next group in range for a resumed reader A resumed reader answered `next_in_range` with `from` itself, so a reader waiting on each group in sequence, such as the mux, still waited out a gap between two ranges. It now answers from its own ranges, resolved as the plain cursor does, with a relative anchor as the first copy to answer resolved it, kept once resolved. Only a relative anchor still waiting answers `from`. `floor` reads the same resolution, so it no longer caches a floor the first copy hadn't resolved. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 123 +++++++++++++++++++++++++++------ rs/moq-net/src/model/track.rs | 6 +- 2 files changed, 104 insertions(+), 25 deletions(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index e1480f871d..dee0cf83e5 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -27,7 +27,7 @@ use std::task::{Poll, ready}; use crate::{Error, Result, StreamError, group, stats, track}; use super::delivered::Delivered; -use super::subscription::{Cap, Position, Resolved, Subscription}; +use super::subscription::{Anchor, Cap, Position, Resolved, Subscription, Window}; /// What the front says about a logical track: the copy to read now, and how it ended. #[derive(Default)] @@ -167,6 +167,7 @@ impl Consumer { /// Subscribe to the logical track with the reader's own preferences. pub(super) fn subscribe(&self, subscription: track::Preferences) -> Subscriber { let mirrored = subscription.read().clone(); + let mirrored_ranges = mirrored.ranges.clone(); let reader = Arc::new_cyclic(|this| { Mutex::new(Reader { this: this.clone(), @@ -176,7 +177,7 @@ impl Consumer { generation: None, copies: Vec::new(), newest: None, - floor: None, + window: Window::new(&mirrored_ranges, None), datagram: None, groups: (0, Cap::from(..)), groups_read: false, @@ -428,18 +429,15 @@ impl Subscriber { } } - /// The floor the first copy answered with; see [`track::Subscriber::floor`]. Later - /// copies start where the reader got to, which names no floor of the reader's own, so - /// it is kept once answered, past the first copy being let go. + /// The lowest group the reader's ranges name; see [`track::Subscriber::floor`]. pub(super) fn floor(&self) -> Option { - let mut reader = self.reader(); - if reader.floor.is_none() { - reader.floor = reader.copies.iter().find_map(|copy| match ©.sub { - Sub::Ready(sub) if copy.floor.is_none() => Some(sub.floor()), - _ => None, - }); - } - reader.floor.flatten() + self.reader().window().start() + } + + /// The lowest group at or above `from` the reader's ranges cover; see + /// [`track::Subscriber::next_in_range`]. + pub(super) fn next_in_range(&self, from: u64) -> Option { + self.reader().window().runs(from).next().map(|run| *run.start()) } /// Poll for sequences the serving copy gave up; see [`track::Subscriber::poll_drop`]. @@ -528,8 +526,8 @@ struct Reader { copies: Vec, /// The newest group handed out: where a new copy is asked to start. newest: Option, - /// The floor the first copy answered for the current ranges, once it answered. - floor: Option>, + /// The reader's ranges resolved; see [`Reader::window`]. + window: Window, /// The newest datagram handed out, by Group and Object ID. datagram: Option<(u64, u64)>, /// The local group filter; see [`track::Subscriber::set_groups`]. @@ -539,6 +537,46 @@ struct Reader { } impl Reader { + /// The reader's ranges resolved, a relative anchor as the first copy to answer for them + /// resolved it: a later copy starts where the reader got to, which would raise it. + /// Kept once resolved, past that copy being let go. Pending while a relative anchor + /// waits on that copy, or when the ranges changed after it was let go. + fn window(&mut self) -> &Window { + if self.window.is_pending() { + let ranges = &self.mirrored.ranges; + let resolved = self.copies.iter_mut().find_map(|copy| match &mut copy.sub { + Sub::Ready(sub) if copy.floor.is_none() && copy.from == *ranges => Some(sub.resolved_ranges()), + _ => None, + }); + if let Some(resolved) = resolved { + let ranges: Vec<_> = ranges + .iter() + .enumerate() + .map(|(index, range)| { + let Some(resolved) = resolved.get(index).copied().flatten() else { + return *range; + }; + track::Range { + start: match range.start { + // The farthest offset reaches back to the oldest group, which names no floor. + Anchor::Relative(back) if back != u64::MAX => { + Anchor::Absolute(Position::group(resolved.start)) + } + start => start, + }, + end: match (range.end, resolved.end) { + (Some(Anchor::Relative(_)), Some(end)) => Some(Anchor::Absolute(Position::group(end))), + (end, _) => end, + }, + } + }) + .collect(); + self.window = Window::new(&ranges, None); + } + } + &self.window + } + /// Follow the route: subscribe to a new copy, mark the ones it replaced, and mirror /// the reader's preferences into each. fn sync(&mut self, waiter: &kio::Waiter) { @@ -553,7 +591,7 @@ impl Reader { }); if let Some(sub) = changed { if sub.ranges != self.mirrored.ranges { - self.floor = None; + self.window = Window::new(&sub.ranges, None); } self.mirrored = sub; for copy in &mut self.copies { @@ -3323,7 +3361,8 @@ mod test { } } - /// The floor the first copy answered stays once a route switch lets that copy go. + /// A relative start, as the first copy resolved it, stays once a route switch lets that + /// copy go. #[test] #[cfg(feature = "async")] fn the_floor_outlives_the_first_copy() { @@ -3331,18 +3370,25 @@ mod test { let logical = logical(&routes); let a = copy(); routes.serve(a.consume()); + write(&a, 4); + write(&a, 5); let subscription = Subscription::default() .with_max_delay(Duration::from_secs(10)) - .with_groups(5..); + .with_ranges([track::Range { + start: Anchor::Relative(1), + end: None, + }]); let mut sub = logical .consume() .subscribe(subscription) .now_or_never() .unwrap() .unwrap(); - write(&a, 5); - assert_eq!(drain(&mut sub, false), [5]); - assert_eq!(sub.floor(), Some(5)); + let mut read = drain(&mut sub, false); + read.sort(); + assert_eq!(read, [4, 5]); + assert_eq!(sub.floor(), Some(4)); + assert_eq!(sub.next_in_range(0), Some(4)); let b = copy(); routes.serve(b.consume()); @@ -3352,7 +3398,40 @@ mod test { let reader = readers[0].upgrade().unwrap(); assert_eq!(reader.lock().unwrap().copies.len(), 1, "the first copy was let go"); drop(readers); - assert_eq!(sub.floor(), Some(5)); + assert_eq!(sub.floor(), Some(4)); + assert_eq!(sub.next_in_range(0), Some(4)); + } + + /// A resumed reader names the next group its ranges cover, so a reader waiting on each + /// group in sequence walks over the gap between two ranges instead of waiting for it. + #[test] + #[cfg(feature = "async")] + fn a_resumed_reader_names_the_next_group_in_range() { + let routes = Producer::new(); + let logical = logical(&routes); + let a = copy(); + routes.serve(a.consume()); + let subscription = Subscription::default() + .with_max_delay(Duration::from_secs(10)) + .with_ranges([track::Range::groups(0..1), track::Range::groups(10..11)]); + let mut sub = logical + .consume() + .subscribe(subscription) + .now_or_never() + .unwrap() + .unwrap(); + for sequence in [0, 5, 10] { + write(&a, sequence); + } + let mut read = drain(&mut sub, false); + read.sort(); + assert_eq!(read, [0, 10]); + assert_eq!( + sub.next_in_range(1), + Some(10), + "the gap between the ranges is never read" + ); + assert_eq!(sub.next_in_range(11), None); } #[test] diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 3f528b1ae4..6559cb4174 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -5679,12 +5679,12 @@ impl Subscriber { /// The lowest group at or above `from` this subscriber's ranges cover, `None` when /// none does: the groups between are never read, so a reader waiting on each group in - /// sequence need not wait for them. `Some(from)` while a relative start waits on the - /// first group, and for a reader that follows a route, which can't say. + /// sequence need not wait for them. `Some(from)` while a relative anchor waits on the + /// first group, which can't say yet. pub fn next_in_range(&self, from: u64) -> Option { match &self.inner { Inner::Plain(cursor) => cursor.filter.window.runs(from).next().map(|run| *run.start()), - Inner::Resume(..) => Some(from), + Inner::Resume(resume, _) => resume.next_in_range(from), } } From 275626010d89df3495173556224884be63c6d528 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 21:36:54 -0700 Subject: [PATCH 33/36] fix(moq-net): resolve replaced ranges on any copy; republish on Control updates A resumed reader resolved a relative range only on the floor-less first copy, so once a route switch let that copy go, replaced ranges never resolved and `next_in_range` stayed at `from`. It now resolves them on whichever ready copy answers for them, taking the resolution from before the copy's failover cap raised it. Ranges replaced through a `Control` reached a cursor's servable window only on its next read, so a held group's expiry was judged against the old ranges. The cursor now publishes its cap beside the window, and a `Control` replacing the ranges republishes the window at once. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/resume.rs | 140 +++++++++++++++++++++++-------- rs/moq-net/src/model/track.rs | 148 ++++++++++++++++++++++++++------- 2 files changed, 224 insertions(+), 64 deletions(-) diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index dee0cf83e5..72fd0a5bc2 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -537,46 +537,56 @@ struct Reader { } impl Reader { - /// The reader's ranges resolved, a relative anchor as the first copy to answer for them - /// resolved it: a later copy starts where the reader got to, which would raise it. - /// Kept once resolved, past that copy being let go. Pending while a relative anchor - /// waits on that copy, or when the ranges changed after it was let go. + /// The reader's ranges resolved, a relative anchor as the first ready copy to answer for + /// them resolved it, whatever its failover cap: that bounds what the copy delivers, not + /// where the ranges resolve. Kept once resolved, past that copy being let go. Pending + /// while a relative anchor waits on a copy to resolve it. fn window(&mut self) -> &Window { - if self.window.is_pending() { - let ranges = &self.mirrored.ranges; - let resolved = self.copies.iter_mut().find_map(|copy| match &mut copy.sub { - Sub::Ready(sub) if copy.floor.is_none() && copy.from == *ranges => Some(sub.resolved_ranges()), - _ => None, - }); - if let Some(resolved) = resolved { - let ranges: Vec<_> = ranges - .iter() - .enumerate() - .map(|(index, range)| { - let Some(resolved) = resolved.get(index).copied().flatten() else { - return *range; - }; - track::Range { - start: match range.start { - // The farthest offset reaches back to the oldest group, which names no floor. - Anchor::Relative(back) if back != u64::MAX => { - Anchor::Absolute(Position::group(resolved.start)) - } - start => start, - }, - end: match (range.end, resolved.end) { - (Some(Anchor::Relative(_)), Some(end)) => Some(Anchor::Absolute(Position::group(end))), - (end, _) => end, - }, - } - }) - .collect(); - self.window = Window::new(&ranges, None); + for index in 0..self.copies.len() { + if !self.window.is_pending() { + break; } + let copy = &mut self.copies[index]; + let resolved = match &mut copy.sub { + Sub::Ready(sub) if copy.from == self.mirrored.ranges => sub.resolved_ranges(), + _ => continue, + }; + self.resolve(&resolved); } &self.window } + /// Resolve a pending [`Self::window`] with a copy's resolution of the reader's ranges, + /// unless a relative anchor still waits there too. + fn resolve(&mut self, resolved: &[Option]) { + let ranges: Vec<_> = self + .mirrored + .ranges + .iter() + .enumerate() + .map(|(index, range)| { + let Some(resolved) = resolved.get(index).copied().flatten() else { + return *range; + }; + track::Range { + start: match range.start { + // The farthest offset reaches back to the oldest group, which names no floor. + Anchor::Relative(back) if back != u64::MAX => Anchor::Absolute(Position::group(resolved.start)), + start => start, + }, + end: match (range.end, resolved.end) { + (Some(Anchor::Relative(_)), Some(end)) => Some(Anchor::Absolute(Position::group(end))), + (end, _) => end, + }, + } + }) + .collect(); + let window = Window::new(&ranges, None); + if !window.is_pending() { + self.window = window; + } + } + /// Follow the route: subscribe to a new copy, mark the ones it replaced, and mirror /// the reader's preferences into each. fn sync(&mut self, waiter: &kio::Waiter) { @@ -590,13 +600,27 @@ impl Reader { Poll::<()>::Pending }); if let Some(sub) = changed { - if sub.ranges != self.mirrored.ranges { + let replaced = sub.ranges != self.mirrored.ranges; + if replaced { self.window = Window::new(&sub.ranges, None); } self.mirrored = sub; for copy in &mut self.copies { copy.update(&self.mirrored); } + // A ready copy just resolved the replaced ranges against its cache, before its + // failover cap raised them, so the reader takes that resolution now. + if replaced { + for index in (0..self.copies.len()).rev() { + if !self.window.is_pending() { + break; + } + if matches!(self.copies[index].sub, Sub::Ready(_)) { + let resolved = self.copies[index].resolved.clone(); + self.resolve(&resolved); + } + } + } } let (generation, serving) = { @@ -3402,6 +3426,52 @@ mod test { assert_eq!(sub.next_in_range(0), Some(4)); } + /// Ranges replaced once a route switch let the first copy go still resolve, from the + /// serving copy, whose failover cap bounds what it delivers and not where they resolve. + #[test] + #[cfg(feature = "async")] + fn replaced_ranges_resolve_after_the_first_copy_is_let_go() { + let routes = Producer::new(); + let logical = logical(&routes); + let a = copy(); + routes.serve(a.consume()); + let mut sub = subscribe(&logical, Duration::from_secs(10)); + write(&a, 0); + assert_eq!(drain(&mut sub, false), [0]); + + let b = copy(); + routes.serve(b.consume()); + for sequence in 1..=5 { + write(&b, sequence); + } + assert!(!drain(&mut sub, false).is_empty()); + { + let readers = routes.readers.lock().unwrap(); + let reader = readers[0].upgrade().unwrap(); + let reader = reader.lock().unwrap(); + assert_eq!(reader.copies.len(), 1, "the first copy was let go"); + assert!(reader.copies[0].floor.is_some(), "the serving copy has a failover cap"); + } + + let subscription = Subscription::default() + .with_max_delay(Duration::from_secs(10)) + .with_ranges([ + track::Range::groups(0..1), + track::Range { + start: Anchor::Relative(1), + end: None, + }, + ]); + sub.update(subscription).unwrap(); + drain(&mut sub, false); + assert_eq!( + sub.next_in_range(1), + Some(4), + "the relative range resolved on the serving copy" + ); + assert_eq!(sub.floor(), Some(0)); + } + /// A resumed reader names the next group its ranges cover, so a reader waiting on each /// group in sequence walks over the gap between two ranges instead of waiting for it. #[test] diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 6559cb4174..e4517f6324 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4578,7 +4578,7 @@ struct GroupExpiry { state: kio::ConsumerWeak, subscription: kio::Consumer, /// What the cursor can be served; see [`Cursor::published`]. - servable: kio::Consumer, + servable: kio::Consumer, sequence: u64, } @@ -4638,7 +4638,7 @@ impl group::Expiry for GroupExpiry { let mut servable = Servable::ALL; let _ = self.servable.poll(waiter, |current| { - servable = (**current).clone(); + servable = current.servable.clone(); Poll::<()>::Pending }); @@ -4660,6 +4660,33 @@ struct Edge { servable: Servable, } +/// The groups `window` covers from `from`, below the exclusive `cap`. +fn admitted(window: &Window, from: u64, cap: Option) -> RangeInclusiveSet { + let mut admitted = RangeInclusiveSet::new(); + let last = match cap { + Some(end) => match end.checked_sub(1) { + Some(last) => last, + None => return admitted, + }, + None => u64::MAX, + }; + for run in window.runs(from) { + if *run.start() > last { + break; + } + admitted.insert(*run.start()..=(*run.end()).min(last)); + } + admitted +} + +/// A cursor's [`Servable`] as the groups it handed out see it, with the cap it was cut to, +/// so a [`Control`] replacing the ranges can republish it without the cursor. +#[derive(Clone, PartialEq)] +struct Published { + servable: Servable, + cap: Option, +} + /// The groups a reader can be served, the only ones its drift is measured against: one it /// never reads, past its cap or in a gap between its ranges, never stands in for one it does. #[derive(Clone, PartialEq)] @@ -4739,8 +4766,9 @@ struct Cursor { /// newest group, and a group in a gap between ranges against nothing. servable: Servable, /// [`Self::servable`], shared with the groups this cursor hands out so their expiry - /// measures drift against the same groups after the cap or ranges move. - published: kio::Producer, + /// measures drift against the same groups after the cap or ranges move, and with its + /// [`Control`]s, which republish it when they replace the ranges. + published: kio::Producer, /// Groups the drift budget skipped since the count was last drained. Accumulated /// here rather than metered in place because the stats scope lives on the owning /// [`Subscriber`], which drains this after every read. @@ -4905,7 +4933,10 @@ impl Cursor { next_sequence: Some(0), end_sequence: None, servable: Servable::ALL, - published: kio::Producer::new(Servable::ALL), + published: kio::Producer::new(Published { + servable: Servable::ALL, + cap: None, + }), stale: stats::Content::default(), seek_pending: BTreeMap::new(), drops: None, @@ -4943,14 +4974,18 @@ impl Cursor { /// Resolve [`Self::servable`] again once the window or cap moved, and publish it to the /// groups already handed out. Costs the ranges, never the cache. fn update_servable(&mut self) { - let servable = Servable::new(self.admitted(0, self.end_sequence)); - // Skip a no-op write: every handed-out group's expiry watches this channel. - if self.servable == servable { + self.servable = Servable::new(self.admitted(0, self.end_sequence)); + let published = Published { + servable: self.servable.clone(), + cap: self.end_sequence, + }; + // Compared with the channel, which a `Control` may have republished, and a no-op + // write skipped: every handed-out group's expiry watches it. + if *self.published.read() == published { return; } - self.servable = servable; if let Ok(mut current) = self.published.write() { - *current = self.servable.clone(); + *current = published; } } @@ -4994,21 +5029,7 @@ impl Cursor { /// The window from `from`, below the exclusive `cap`. fn admitted(&self, from: u64, cap: Option) -> RangeInclusiveSet { - let mut admitted = RangeInclusiveSet::new(); - let last = match cap { - Some(end) => match end.checked_sub(1) { - Some(last) => last, - None => return admitted, - }, - None => u64::MAX, - }; - for run in self.filter.window.runs(from) { - if *run.start() > last { - break; - } - admitted.insert(*run.start()..=(*run.end()).min(last)); - } - admitted + admitted(&self.filter.window, from, cap) } /// [`Filter::owed`] as of now: what was owed and what the walk newly reads since @@ -5355,6 +5376,35 @@ impl Cursor { #[derive(Clone)] pub struct Control { subscription: Preferences, + /// A plain cursor's servable window, republished when the ranges are replaced, so a + /// group it handed out is judged against them before the cursor next reads. `None` + /// for a reader that follows a route, whose copies take them on its next read. + republish: Option, +} + +/// What a [`Control`] needs to republish a plain cursor's [`Published`] window. +#[derive(Clone)] +struct Republish { + state: kio::ConsumerWeak, + published: kio::ProducerWeak, +} + +impl Republish { + /// Resolve `ranges` as the cursor will, under its cap, and publish them. + fn publish(&self, ranges: &[Range]) { + let Some(published) = self.published.produce() else { + return; + }; + let window = Window::new(ranges, self.state.read().newest_produced); + let cap = published.read().cap; + let servable = Servable::new(admitted(&window, 0, cap)); + if published.read().servable == servable { + return; + } + if let Ok(mut current) = published.write() { + current.servable = servable; + } + } } impl Control { @@ -5368,6 +5418,11 @@ impl Control { /// Returns [`Error::Closed`] if the track already ended; the update is /// meaningless at that point and can usually be ignored. pub fn update(&self, subscription: Subscription) -> Result<()> { + // Republished first, so the cursor's own resolution, when it next reads, replaces it. + let replaced = self.republish.is_some() && self.subscription.read().ranges != subscription.ranges; + if let Some(republish) = self.republish.as_ref().filter(|_| replaced) { + republish.publish(&subscription.ranges); + } self.subscription.update(subscription) } } @@ -5411,10 +5466,17 @@ impl Subscriber { /// Create a handle for updating this subscriber's delivery preferences. pub fn control(&self) -> Control { - Control { - subscription: match &self.inner { - Inner::Plain(cursor) => cursor.subscription.clone(), - Inner::Resume(resume, _) => resume.subscription().clone(), + match &self.inner { + Inner::Plain(cursor) => Control { + subscription: cursor.subscription.clone(), + republish: Some(Republish { + state: cursor.state.weak(), + published: cursor.published.weak(), + }), + }, + Inner::Resume(resume, _) => Control { + subscription: resume.subscription().clone(), + republish: None, }, } } @@ -9569,6 +9631,34 @@ mod test { assert!(!pending.is_finished(), "group 0 is within the budget of its run"); } + /// Ranges replaced through a [`Control`] republish what a held group is judged against + /// at once, before the cursor next reads: group 0 is judged against the run's own + /// group 10, not group 1, which the new ranges never read. + #[moq_clock::test] + #[cfg(feature = "async")] + async fn a_control_update_moves_the_run_a_held_group_is_judged_against() { + let at = |millis| Timestamp::from_millis(millis).unwrap(); + let budget = Subscription::default().with_max_delay(Duration::from_secs(15)); + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(budget.clone()); + let mut zero = producer.create_group(group::Info { sequence: 0 }).unwrap(); + zero.write_frame(at(0), bytes::Bytes::from_static(b"x")).unwrap(); + let mut held = subscriber.recv_group().await.unwrap().expect("group 0"); + assert!(held.read_frame().await.unwrap().is_some()); + + subscriber.control().update(budget.with_groups(10..20)).unwrap(); + for (sequence, millis) in [(1, 1000), (10, 10_000), (19, 19_000)] { + let mut group = producer.create_group(group::Info { sequence }).unwrap(); + group.write_frame(at(millis), bytes::Bytes::from_static(b"x")).unwrap(); + group.finish().unwrap(); + } + + let pending = moq_clock::sim::spawn(async move { held.read_frame().await }); + moq_clock::sim::yield_now().await; + assert!(!pending.is_finished(), "group 0 is within the budget of its run"); + drop(subscriber); + } + #[test] #[cfg(feature = "async")] fn a_capped_subscriber_measures_drift_against_its_cap() { From 08ae9f7598d85d6a7819091fae39ea5508fc861f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 22:17:09 -0700 Subject: [PATCH 34/36] style(moq-mux): format the merged hole walk Co-Authored-By: Claude Opus 5.5 --- rs/moq-mux/src/container/consumer.rs | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index eab655acaa..40251d198b 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -522,11 +522,10 @@ impl Consumer { let since = self.hole.unwrap().1; let elapsed = since.elapsed() >= self.max_delay; let aged = next_group.is_some_and(|(_, start)| max_timestamp.saturating_sub(start) >= self.max_delay); - let excluded = self.track.as_ref().is_some_and(|track| { - track - .next_in_range(self.current) - .is_none_or(|next| next >= sequence) - }); + let excluded = self + .track + .as_ref() + .is_some_and(|track| track.next_in_range(self.current).is_none_or(|next| next >= sequence)); if finished || elapsed || aged || excluded { let _ = self.pending.front_mut().unwrap().look_all(waiter, &self.format); let front = self.pending.front_mut().unwrap(); From 3f8bb34a699e9f11568922fb0a98862d97965dae Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 23:34:33 -0700 Subject: [PATCH 35/36] fix(moq-net): retake a pending drift snapshot; publish window and cap as one value A read's drift snapshot could be taken from a window still waiting on its first group, which serves everything, just before that group arrived and settled it. A zero-delay `Absolute(0)..Relative(0)` then judged group 0 against group 1, which it never reads, and skipped it. Both read paths now take the snapshot again once the first arrival settles the window. A `Control` republishing the ranges and the cursor moving its cap could interleave and leave a servable cut from one writer's stale input. The published value now holds both inputs, the window's runs by ranges revision and the cap, and each writer replaces only its own and derives the servable under the same write. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 206 ++++++++++++++++++++++++++++------ 1 file changed, 171 insertions(+), 35 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index b1b7958611..57f76f6b49 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -4741,29 +4741,53 @@ struct Edge { /// The groups `window` covers from `from`, below the exclusive `cap`. fn admitted(window: &Window, from: u64, cap: Option) -> RangeInclusiveSet { - let mut admitted = RangeInclusiveSet::new(); + below(window.runs(from), cap) +} + +/// `runs`, lowest first, below the exclusive `cap`. +fn below(runs: impl Iterator>, cap: Option) -> RangeInclusiveSet { + let mut below = RangeInclusiveSet::new(); let last = match cap { Some(end) => match end.checked_sub(1) { Some(last) => last, - None => return admitted, + None => return below, }, None => u64::MAX, }; - for run in window.runs(from) { + for run in runs { if *run.start() > last { break; } - admitted.insert(*run.start()..=(*run.end()).min(last)); + below.insert(*run.start()..=(*run.end()).min(last)); } - admitted + below } -/// A cursor's [`Servable`] as the groups it handed out see it, with the cap it was cut to, -/// so a [`Control`] replacing the ranges can republish it without the cursor. +/// A cursor's [`Servable`] as the groups it handed out see it, derived from its two +/// inputs: the window, which a [`Control`] replacing the ranges republishes, and the cap, +/// which only the cursor moves. Each writer replaces its own input and derives the servable +/// under the same write, so neither can publish one cut from the other's stale input. #[derive(Clone, PartialEq)] struct Published { - servable: Servable, + /// The window's groups, resolved for `revision` of the ranges. + runs: Arc>, + /// The [`Preferences::ranges_revision`] `runs` were resolved for. + revision: u64, cap: Option, + /// `runs` below `cap`. + servable: Servable, +} + +impl Published { + fn new(runs: Arc>, revision: u64, cap: Option) -> Self { + let servable = Servable::new(below(runs.iter().cloned(), cap)); + Self { + runs, + revision, + cap, + servable, + } + } } /// The groups a reader can be served, the only ones its drift is measured against: one it @@ -5012,10 +5036,11 @@ impl Cursor { next_sequence: Some(0), end_sequence: None, servable: Servable::ALL, - published: kio::Producer::new(Published { - servable: Servable::ALL, - cap: None, - }), + published: kio::Producer::new(Published::new( + Arc::new(RangeInclusiveSet::from_iter([0..=u64::MAX])), + 0, + None, + )), stale: stats::Content::default(), seek_pending: BTreeMap::new(), drops: None, @@ -5054,17 +5079,25 @@ impl Cursor { /// groups already handed out. Costs the ranges, never the cache. fn update_servable(&mut self) { self.servable = Servable::new(self.admitted(0, self.end_sequence)); - let published = Published { - servable: self.servable.clone(), - cap: self.end_sequence, + let (revision, cap) = (self.filter.resolved, self.end_sequence); + let next = |current: &Published| { + // Ranges a `Control` republished past this window stay until this cursor syncs + // to them; the cap is this cursor's own. + let runs = match current.revision > revision { + true => current.runs.clone(), + false => Arc::new(self.admitted(0, None)), + }; + let next = Published::new(runs, revision.max(current.revision), cap); + (next != *current).then_some(next) }; - // Compared with the channel, which a `Control` may have republished, and a no-op - // write skipped: every handed-out group's expiry watches it. - if *self.published.read() == published { + // A no-op write is skipped: every handed-out group's expiry watches the channel. + if next(&self.published.read()).is_none() { return; } - if let Ok(mut current) = self.published.write() { - *current = published; + if let Ok(mut current) = self.published.write() + && let Some(next) = next(¤t) + { + *current = next; } } @@ -5284,7 +5317,11 @@ impl Cursor { // One scan for the whole poll, so walking a backlog off stays linear in its size. let drift = ready!(self.poll_drift(&self.servable, waiter))?; + self.poll_recv_with(drift, waiter) + } + /// [`Self::poll_recv_group`] once synced, judging against `drift`. + fn poll_recv_with(&mut self, mut drift: Drift, waiter: &kio::Waiter) -> Poll>> { loop { // Re-offer what a widening newly covers ahead of the arrival walk, lowest first: // it is the oldest thing still owed. A group the walk has yet to reach comes @@ -5314,6 +5351,10 @@ impl Cursor { if self.filter.window.is_pending() { self.sync(); self.admit(); + // The snapshot saw the pending window, which serves everything. + if !self.filter.window.is_pending() { + drift = ready!(self.poll_drift(&self.servable, waiter))?; + } if consumer.sequence < self.floor() { continue; } @@ -5378,11 +5419,20 @@ impl Cursor { return Poll::Ready(Ok(None)); }; self.sync(); - let mut floor = next.max(self.floor()); - let end = self.end_sequence; + let floor = next.max(self.floor()); // One scan for the whole poll, so walking a backlog off stays linear in its size. let drift = ready!(self.poll_drift(&self.servable, waiter))?; + self.poll_next_with(floor, drift, waiter) + } + /// [`Self::poll_next_group`] once synced, from `floor`, judging against `drift`. + fn poll_next_with( + &mut self, + mut floor: u64, + mut drift: Drift, + waiter: &kio::Waiter, + ) -> Poll>> { + let end = self.end_sequence; let group = loop { let window = &self.filter.window; let Some(producer) = ready!(self.poll(waiter, |state| state.poll_next_in_range(floor, end, window))?) @@ -5398,6 +5448,10 @@ impl Cursor { // The first group to arrive resolves a relative floor, which may land above it. if self.filter.window.is_pending() { self.sync(); + // The snapshot saw the pending window, which serves everything. + if !self.filter.window.is_pending() { + drift = ready!(self.poll_drift(&self.servable, waiter))?; + } if group.sequence < self.floor() || !self.filter.window.contains(group.sequence) { floor = floor.max(self.floor()).max(group.sequence.saturating_add(1)); continue; @@ -5469,19 +5523,25 @@ struct Republish { } impl Republish { - /// Resolve `ranges` as the cursor will, under its cap, and publish them. - fn publish(&self, ranges: &[Range]) { + /// Resolve the ranges in `preferences` as the cursor will, and publish them under the + /// cap the cursor holds then, unless the cursor already published them or newer. + fn publish(&self, preferences: &Preferences) { let Some(published) = self.published.produce() else { return; }; - let window = Window::new(ranges, self.state.read().newest_produced); - let cap = published.read().cap; - let servable = Servable::new(admitted(&window, 0, cap)); - if published.read().servable == servable { + let (ranges, revision) = { + let current = preferences.read(); + (current.ranges.clone(), preferences.ranges_revision()) + }; + if published.read().revision >= revision { return; } - if let Ok(mut current) = published.write() { - current.servable = servable; + let window = Window::new(&ranges, self.state.read().newest_produced); + let runs = Arc::new(admitted(&window, 0, None)); + if let Ok(mut current) = published.write() + && current.revision < revision + { + *current = Published::new(runs, revision, current.cap); } } } @@ -5497,12 +5557,11 @@ impl Control { /// Returns [`Error::Closed`] if the track already ended; the update is /// meaningless at that point and can usually be ignored. pub fn update(&self, subscription: Subscription) -> Result<()> { - // Republished first, so the cursor's own resolution, when it next reads, replaces it. - let replaced = self.republish.is_some() && self.subscription.read().ranges != subscription.ranges; - if let Some(republish) = self.republish.as_ref().filter(|_| replaced) { - republish.publish(&subscription.ranges); + self.subscription.update(subscription)?; + if let Some(republish) = &self.republish { + republish.publish(&self.subscription); } - self.subscription.update(subscription) + Ok(()) } } @@ -9975,6 +10034,83 @@ mod test { assert!(!pending.is_finished(), "group 0 is within the budget of its run"); } + /// A drift snapshot taken while the window waits on its first group is taken again once + /// that group settles it. Groups 0 and 1 land between the snapshot and the arrival: the + /// settled `0..=0` judges group 0 against itself, not group 1, which it never reads. + #[test] + fn the_first_arrival_retakes_a_pending_drift_snapshot() { + for ordered in [false, true] { + let mut producer = track_producer("test", Info::timed()); + let subscription = Subscription::default() + .with_max_delay(Duration::ZERO) + .with_ranges([Range { + start: Anchor::Absolute(Position::group(0)), + end: Some(Anchor::Relative(0)), + }]); + let mut subscriber = producer.subscribe(subscription); + let Inner::Plain(cursor) = &mut subscriber.inner else { + unreachable!("a local subscriber reads its own cursor"); + }; + let waiter = kio::Waiter::noop(); + cursor.sync(); + assert!(cursor.filter.window.is_pending()); + let Poll::Ready(Ok(drift)) = cursor.poll_drift(&cursor.servable, &waiter) else { + unreachable!("a drift snapshot is always ready"); + }; + + append_at(&mut producer, 0); + append_at(&mut producer, 1000); + let res = match ordered { + true => cursor.poll_next_with(0, drift, &waiter), + false => cursor.poll_recv_with(drift, &waiter), + }; + let Poll::Ready(Ok(Some(group))) = res else { + panic!("group 0 is the newest group its range reads (ordered: {ordered})"); + }; + assert_eq!(group.sequence, 0); + } + } + + /// A [`Control`] replacing the ranges and the cursor moving its cap each change only + /// their own part of what handed-out groups are judged against, whichever goes first, + /// and with the cap moving between the ranges being written and republished. + #[test] + fn a_control_update_and_a_cap_never_publish_a_mismatched_pair() { + let published = |subscriber: &Subscriber| { + let Inner::Plain(cursor) = &subscriber.inner else { + unreachable!("a local subscriber reads its own cursor"); + }; + let current = cursor.published.read(); + (current.servable.clone(), current.cap) + }; + let ranges = Subscription::default().with_groups(10..20); + // Nothing in 10..20 sits below a cap at 5. + let expected = (Servable::new(RangeInclusiveSet::new()), Some(5)); + for order in ["ranges first", "cap first", "cap between"] { + let producer = track_producer("test", Info::timed()); + let mut subscriber = producer.subscribe(Subscription::default()); + let control = subscriber.control(); + match order { + "ranges first" => { + control.update(ranges.clone()).unwrap(); + subscriber.set_groups(..5); + } + "cap first" => { + subscriber.set_groups(..5); + control.update(ranges.clone()).unwrap(); + } + _ => { + control.subscription.update(ranges.clone()).unwrap(); + subscriber.set_groups(..5); + control.republish.as_ref().unwrap().publish(&control.subscription); + } + } + assert!(published(&subscriber) == expected, "{order}"); + let _ = subscriber.poll_recv_group(&kio::Waiter::noop()); + assert!(published(&subscriber) == expected, "{order}, once the cursor syncs"); + } + } + /// Ranges replaced through a [`Control`] republish what a held group is judged against /// at once, before the cursor next reads: group 0 is judged against the run's own /// group 10, not group 1, which the new ranges never read. From ab1bce64754c31d3d50dfb843b94d197e78aba0c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 19 Oct 2026 23:46:18 -0700 Subject: [PATCH 36/36] fix(moq-net): start a cursor's published window empty The cursor replaces it before handing anything out, and a one-range array trips clippy's `single_range_in_vec_init`. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/model/track.rs | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index 57f76f6b49..24d9ea02c7 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -5036,11 +5036,8 @@ impl Cursor { next_sequence: Some(0), end_sequence: None, servable: Servable::ALL, - published: kio::Producer::new(Published::new( - Arc::new(RangeInclusiveSet::from_iter([0..=u64::MAX])), - 0, - None, - )), + // Replaced by the cursor's own below, before anything is handed out. + published: kio::Producer::new(Published::new(Arc::default(), 0, None)), stale: stats::Content::default(), seek_pending: BTreeMap::new(), drops: None,