From c86405e4080f23dc3e02d6fa0273f3fcab470750 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 20:55:15 -0700 Subject: [PATCH 01/14] quest: claim m1/lite-time-provenance Co-Authored-By: Claude Opus 5.5 From 5d7fb8a4e04975716a079b489fb4b3c964d4063a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 21:33:48 -0700 Subject: [PATCH 02/14] feat(lite): carry unverified timestamps so receivers check verified tracks lite-07 TRACK_INFO gains `Unverified (8)`. A timed track received over lite-05/06 or moq-transport is unverified, and relays preserve the mark, so the rewind check runs on received groups only where every hop could vouch for them. Replaces the session-level received mark in Rust and js/net, and completes quest/m1/lite-time-provenance. Co-Authored-By: Claude Opus 5.5 --- doc/concept/moq-lite.md | 5 + drafts/draft-lcurley-moq-lite.md | 15 +- js/net/MIGRATE.md | 1 + js/net/src/ietf/subscriber.ts | 3 +- js/net/src/internal.ts | 9 -- js/net/src/lite/publisher.ts | 1 + js/net/src/lite/subscriber.test.ts | 113 +++++++++------ js/net/src/lite/subscriber.ts | 2 +- js/net/src/lite/track.test.ts | 32 +++++ js/net/src/lite/track.ts | 33 ++++- js/net/src/lite/version.ts | 17 +++ js/net/src/track.ts | 32 +++-- quest/m1/README.md | 1 - quest/m1/lite-time-provenance.md | 38 ----- quest/m1/public-model/ingest.md | 11 +- rs/moq-net/MIGRATE.md | 2 +- rs/moq-net/src/fuzz.rs | 1 + rs/moq-net/src/ietf/subscriber.rs | 19 ++- rs/moq-net/src/lite/publisher.rs | 14 +- rs/moq-net/src/lite/subscriber.rs | 53 ++++--- rs/moq-net/src/lite/track.rs | 132 ++++++++++++++++-- rs/moq-net/src/lite/tracks.rs | 1 + rs/moq-net/src/lite/version.rs | 12 ++ rs/moq-net/src/model/track.rs | 28 ++-- rs/moq-net/src/test_interop.rs | 1 + .../tests/integration/received_rewind.rs | 6 +- 26 files changed, 402 insertions(+), 180 deletions(-) delete mode 100644 quest/m1/lite-time-provenance.md diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index d7a9f893ca2..e55a178250e 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -280,6 +280,11 @@ untimed. moq-lite 05 and 06 cannot declare an untimed track, so there its frames go out stamped with their send time; the work-in-progress 07 carries it untimed. +A publisher refuses a group that starts before a lower one. A receiver checks +received groups the same way only when every hop could vouch for them: a timed +track received over moq-lite 05 or 06, or over moq-transport, is unverified, and +the work-in-progress 07 says so as it passes through relays. + A relay cancels its upstream subscription once nobody subscribes, but keeps its copy of the track for 30 seconds after the last reader leaves, so a returning reader or the next fetch finds it. That copy is not live meanwhile: readers get diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 3ecfd1154dd..108422e6105 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -176,7 +176,8 @@ Each frame of a timed Track carries a presentation timestamp expressed in the Tr A Track without a `Timescale` is untimed: its frames and datagrams carry no timestamp. A publisher MUST NOT start a timed Group's frame 0 earlier than that of any lower-sequence Group it publishes. lite-05 and lite-06 cannot encode an untimed Track, so a publisher on those versions declares a `Timescale` of 1000 for it and stamps every frame and datagram with its send time in milliseconds. -A receiver cannot tell such send times from presentation timestamps, and they need not rise with sequence, so it does not check received Groups against this rule. +A receiver cannot tell such send times from presentation timestamps, and they need not rise with sequence, so it treats every timed Track received on those versions as unverified (see `Unverified` in [TRACK_INFO](#track-info)). +A receiver MAY check a Group of a verified Track against this rule, refusing a violating Group's stream (STOP_SENDING) with TIMESTAMP_MISMATCH, and MUST NOT check one of an unverified Track. ## Positions {#positions} A Position is a (Group Sequence, Frame Index) pair identifying one frame within a Track. @@ -357,7 +358,7 @@ Sent when resetting a stream (RESET_STREAM), or when refusing to receive one (ST | ------- | ------------- | ----------- | | 0x38 | FRAME_TOO_LARGE | A frame declared a payload larger than the receiver accepts. | | ------- | ------------- | ----------- | -| 0x39 | TIMESTAMP_MISMATCH | A frame's timestamp does not match its track's timescale. | +| 0x39 | TIMESTAMP_MISMATCH | A frame's timestamp does not fit its track: it is present on an untimed track, absent on a timed one, or starts a verified Group before a lower-sequence one. | | ------- | ------------- | ----------- | | 0x3A | NOT_FETCHABLE | The FETCH named a group delivered only as a datagram, which is never cached (see [Datagrams](#datagrams)). It has no moq-transport value; a bridge refuses the FETCH with DOES_NOT_EXIST. | | ------- | ------------- | ----------- | @@ -1306,6 +1307,7 @@ TRACK_INFO Message { Publisher Max Age (i) Timescale (i) Refresh (i) + Unverified (8) } ~~~ @@ -1345,6 +1347,13 @@ Joins coalesce until the requested group arrives. A relay preserves the ratio and evaluates joins against its own cache, forwarding demand in SUBSCRIBE_UPDATE. Versions before lite-07 omit Refresh and Wanted Group; a Track received on those versions never refreshes. +**Unverified**: +1 when the Track's timestamps may break the rule that no Group starts before a lower-sequence one (see [Frame](#frame)), 0 otherwise. +A publisher sets it when a hop stamped the timestamps (a send or arrival time) or it received the Track on a wire that cannot say otherwise: a timed Track received over lite-05, lite-06, or MoqTransport, or with `Unverified` set to 1. +A relay preserves it, so a receiver checks a Track only when every hop since its publisher could. +It MUST be 0 on an untimed Track; a receiver MUST treat any other value as a protocol violation. +Versions before lite-07 omit Unverified. + ## TRACKS {#tracks} TRACKS is sent by a subscriber to list a Broadcast's Tracks. It is the first message on a Tracks Stream (0x8). @@ -1731,7 +1740,7 @@ The `Message Length` describes the payload size on the wire. - Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery. -- A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's. Receivers do not check it, since a received timestamp may be a send time. +- A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's. Added TRACK_INFO `Unverified`, set when a hop stamped the timestamps or received them over lite-05/06 or MoqTransport, which cannot say they weren't. A receiver may check only a verified Track. - Added the reliable ordered RTCDataChannel binding, with one Qmux Record per binary message and no datagrams. - Added `Group Order` to SUBSCRIBE and SUBSCRIBE_UPDATE: descending (0x0, the previous fixed order) or ascending (0x1). A relay requests descending upstream unless every subscription it serves is ascending. diff --git a/js/net/MIGRATE.md b/js/net/MIGRATE.md index 4d7076305f6..31e758145ab 100644 --- a/js/net/MIGRATE.md +++ b/js/net/MIGRATE.md @@ -46,6 +46,7 @@ A new group or datagram raises it at frame 0 before any frame lands, and it can ### A timed track refuses a group that rewinds (#5803) A local write whose first frame starts before a lower-sequence group's, or after a higher one's, throws `TimestampMismatch`. +A received group is checked too when every hop vouched for it over moq-lite-07. One received over moq-lite-05/06 or moq-transport is accepted as is. A publisher whose timeline restarts publishes a new broadcast instead. ### `Auth.Auth` is `Auth.Handle`, and `Auth.Token` is `Auth.Auth` (#5695) diff --git a/js/net/src/ietf/subscriber.ts b/js/net/src/ietf/subscriber.ts index bf621937bc1..a21e8a6f773 100644 --- a/js/net/src/ietf/subscriber.ts +++ b/js/net/src/ietf/subscriber.ts @@ -801,7 +801,6 @@ export class Subscriber { // Keep the request pending until SUBSCRIBE_OK supplies immutable track metadata. // Group streams already wait on the alias, so early data stays behind this response. const producer = hooks.pendingTrackProducer(request); - hooks.receiveTrack(producer); const subscription: Subscription = { track: producer, tail: new Tail() }; // Open the stream and wait for SUBSCRIBE_OK under a timeout. State @@ -1205,6 +1204,8 @@ export class Subscriber { timescale: ok.properties.timescale, priority: fromWire(ok.properties.priority ?? 128), maxAge: maxCacheDuration === undefined ? undefined : Milli(Number(maxCacheDuration)), + // A moq-transport peer may stamp arrival times, and nothing on the wire says it didn't. + unverified: ok.properties.timescale != null, }); if (ok.largest && ok.largest.groupId > BigInt(Number.MAX_SAFE_INTEGER)) { throw new RangeError("largest group exceeds safe integers"); diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index c342ed8218a..d94c86b1f32 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -365,12 +365,6 @@ export const hooks: { * `track.ts`. */ startTrack: (producer: Producer, served: { start: number; requested?: number }) => void; - // pending: /quest/m1/public-model/ingest.md - /** - * Mark a track as a session's copy of a remote one: its groups are never checked for a - * rewind, since a received timestamp may not be media time. - */ - receiveTrack: (producer: Producer) => void; /** Mint a track {@link Request}; assigned by `track.ts`. */ makeRequest: (options: TrackRequestOptions) => Request; // pending: /quest/m1/public-model/ingest.md @@ -497,9 +491,6 @@ export const hooks: { /** Name the epoch of the route an origin resolved a broadcast handle through. */ stampEpoch: (target: BroadcastConsumer, epoch: Epoch.Valid | undefined) => void; } = { - receiveTrack: () => { - throw new Error("track.ts not loaded"); - }, endTrack: () => { throw new Error("track.ts not loaded"); }, diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index c9e64cc4336..96f55e43a7d 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -1442,6 +1442,7 @@ export class Publisher { maxAge: info.maxAge, // `#serveGroup` emits each frame converted to it. timescale: info.timescale ?? undefined, + unverified: info.unverified, }); })(); diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index c98388f23ea..794e545f124 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -4,7 +4,16 @@ import { Signal } from "@moq/signals"; import type { Grant } from "../auth.ts"; import type { Probe as ProbeStats } from "../connection/stats.ts"; import * as Epoch from "../epoch.ts"; -import { error, fromTransport, ProtocolViolation, reason, SessionCode, StreamCode, StreamError } from "../error.ts"; +import { + error, + fromTransport, + ProtocolViolation, + reason, + SessionCode, + StreamCode, + StreamError, + TimestampMismatch, +} from "../error.ts"; import { HopSchema, isAnonymous, MAX_HOPS, Route, UNKNOWN_HOP } from "../hop.ts"; import * as Path from "../path.ts"; import { Reader, Writer } from "../stream.ts"; @@ -947,13 +956,14 @@ async function answerTrackInfo( stream: FakeStream, version: Version = Version.DRAFT_05, timescale?: number, + unverified = false, ): Promise { const chunks: Uint8Array[] = []; const writer = new Writer( new WritableStream({ write: (chunk) => void chunks.push(new Uint8Array(chunk)) }), version, ); - await new TrackInfo({ timescale }).encode(writer, version); + await new TrackInfo({ timescale, unverified }).encode(writer, version); for (const chunk of chunks) stream.inbound.enqueue(chunk); stream.inbound.close(); } @@ -1718,44 +1728,69 @@ async function groupStream(timestamp: number, version: Version = Version.DRAFT_0 ); } -// A received timestamp may not be media time (lite-05/06 stamp an untimed track's frames with -// their send time), so a received group that rewinds or lands out of order is accepted at every -// version. Only a local publisher's writes refuse a rewind. -test.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])( - "version %p accepts a received rewind", - async (version) => { - const { quic, streams, opened } = fakeSession(); - const subscriber = new Subscriber(quic, version, HopSchema.parse(1n)); - const broadcast = subscriber.consume(Path.from("room")); - // Old enough to keep every group, however far behind the newest start. - const track = broadcast.track("video").subscribe({ maxDelay: Milli(60_000) }); - try { - await opened(0); - await answerTrackInfo(streams[0], version, 1000); - await (await opened(1)).reading; - await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 4 }), await groupStream(1000, version)); - // Starts before the group below it. - await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 5 }), await groupStream(500, version)); - // Lands below group 5 but starts after it. - await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 3 }), await groupStream(2000, version)); - const got = []; - for (let i = 0; i < 3; i++) { - const group = await track.recvGroup(); - got.push([group?.sequence, (await group?.readFrame())?.payload.length]); - } - expect(got).toEqual([ - [3, 1], - [4, 1], - [5, 1], - ]); - expect(track.closed.peek()).toBeUndefined(); - } finally { - track.close(); - broadcast.close(); - subscriber.close(); +// A received group that rewinds or lands out of order is accepted wherever its timestamps are +// unverified: always on lite-05/06, which stamp an untimed track's frames with their send time, and +// on lite-07 when the publisher says so. +test.each([ + [Version.DRAFT_05, false], + [Version.DRAFT_06, false], + [Version.DRAFT_07, true], +])("version %p accepts an unverified rewind", async (version, unverified) => { + const { quic, streams, opened } = fakeSession(); + const subscriber = new Subscriber(quic, version, HopSchema.parse(1n)); + const broadcast = subscriber.consume(Path.from("room")); + // Old enough to keep every group, however far behind the newest start. + const track = broadcast.track("video").subscribe({ maxDelay: Milli(60_000) }); + try { + await opened(0); + await answerTrackInfo(streams[0], version, 1000, unverified); + await (await opened(1)).reading; + await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 4 }), await groupStream(1000, version)); + // Starts before the group below it. + await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 5 }), await groupStream(500, version)); + // Lands below group 5 but starts after it. + await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 3 }), await groupStream(2000, version)); + const got = []; + for (let i = 0; i < 3; i++) { + const group = await track.recvGroup(); + got.push([group?.sequence, (await group?.readFrame())?.payload.length]); } - }, -); + expect(got).toEqual([ + [3, 1], + [4, 1], + [5, 1], + ]); + expect(track.closed.peek()).toBeUndefined(); + } finally { + track.close(); + broadcast.close(); + subscriber.close(); + } +}); + +// A lite-07 track whose publisher vouches for its timestamps is checked like a local write. +test("a verified rewind is refused", async () => { + const version = Version.DRAFT_07; + const { quic, streams, opened } = fakeSession(); + const subscriber = new Subscriber(quic, version, HopSchema.parse(1n)); + const broadcast = subscriber.consume(Path.from("room")); + const track = broadcast.track("video").subscribe({ maxDelay: Milli(60_000) }); + try { + await opened(0); + await answerTrackInfo(streams[0], version, 1000); + await (await opened(1)).reading; + await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 4 }), await groupStream(1000, version)); + // Starts before the group below it, so its stream is stopped with the refusal. + const rewound = await groupStream(500, version); + const stop = spyOn(rewound, "stop"); + await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 5 }), rewound); + expect(stop.mock.calls[0]?.[0]).toBeInstanceOf(TimestampMismatch); + } finally { + track.close(); + broadcast.close(); + subscriber.close(); + } +}); test("a publisher FIN without SUBSCRIBE_END still violates an open subscription", async () => { const { quic, streams, opened, closes } = fakeSession(); diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 4a1cacc8009..856aef3562f 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -863,7 +863,6 @@ export class Subscriber { timescale.set(0); drainOk = true; } - hooks.receiveTrack(producer); // Register before opening SUBSCRIBE so a racing GROUP stream finds the entry. const entry: SubscribeEntry = { @@ -966,6 +965,7 @@ export class Subscriber { maxAge: info.maxAge === undefined ? undefined : Milli(info.maxAge), priority: info.priority, refresh: info.refresh, + unverified: info.unverified, }; } diff --git a/js/net/src/lite/track.test.ts b/js/net/src/lite/track.test.ts index e0c776bf829..ebcbeb3bb7c 100644 --- a/js/net/src/lite/track.test.ts +++ b/js/net/src/lite/track.test.ts @@ -212,3 +212,35 @@ test("refresh ratio survives lite-07 and disappears on legacy wires", async () = } } }); + +test("unverified survives lite-07, and every timed track arrives unverified before it", async () => { + for (const version of [Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07]) { + for (const unverified of [false, true]) { + const info = new TrackInfo({ priority: 7, timescale: 1000, unverified }); + const encoded = await bytes((w) => info.encode(w, version), version); + const decoded = await TrackInfo.decode(new Reader(undefined, encoded, version), version); + expect(decoded.unverified).toBe(version === Version.DRAFT_07 ? unverified : true); + } + } +}); + +test("an unverified timed TrackInfo matches cross-language wire bytes on lite-07", async () => { + const info = new TrackInfo({ priority: 1, timescale: 1000, unverified: true }); + expect(await bytes((w) => info.encode(w, Version.DRAFT_07), Version.DRAFT_07)).toEqual( + new Uint8Array([0x06, 0x01, 0x00, 0x83, 0xe8, 0x00, 0x01]), + ); +}); + +test("an untimed track can't be unverified", async () => { + expect(() => infoDefaults({ unverified: true })).toThrow(RangeError); + const info = new TrackInfo({ priority: 1, unverified: true }); + await expect(bytes((w) => info.encode(w, Version.DRAFT_07), Version.DRAFT_07)).rejects.toThrow(RangeError); + + const encoded = await bytes((w) => new TrackInfo({ priority: 1 }).encode(w, Version.DRAFT_07), Version.DRAFT_07); + for (const flag of [1, 2]) { + encoded[encoded.length - 1] = flag; + await expect( + TrackInfo.decode(new Reader(undefined, encoded, Version.DRAFT_07), Version.DRAFT_07), + ).rejects.toThrow(RangeError); + } +}); diff --git a/js/net/src/lite/track.ts b/js/net/src/lite/track.ts index e68210b4565..71d016a0522 100644 --- a/js/net/src/lite/track.ts +++ b/js/net/src/lite/track.ts @@ -4,7 +4,7 @@ import type { Reader, Writer } from "../stream.ts"; import { Timescale } from "../time.ts"; import { decodeEpoch, encodeEpoch } from "./epoch.ts"; import * as Message from "./message.ts"; -import { hasGroupOrder, hasRefresh, hasTrackStream, Version, wireTimescale } from "./version.ts"; +import { hasGroupOrder, hasRefresh, hasTrackStream, hasUnverified, Version, wireTimescale } from "./version.ts"; function guardTrack(version: Version) { if (!hasTrackStream(version)) throw new Error("track stream not supported for this version"); @@ -71,17 +71,24 @@ export class TrackInfo { * out as milliseconds (see {@link wireTimescale}). */ timescale?: Timescale; + /** + * Whether a timed track's timestamps may rewind, so receivers don't check them. Carried on + * lite-07; every timed track received over lite-05/06 is unverified. + */ + unverified: boolean; constructor({ priority = 0, maxAge, timescale, refresh, + unverified = false, }: { priority?: number; maxAge?: number; timescale?: number; refresh?: number; + unverified?: boolean; }) { if (!Number.isInteger(priority) || priority < 0 || priority > 255) { throw new RangeError(`priority must be an integer in 0..=255: ${priority}`); @@ -95,6 +102,7 @@ export class TrackInfo { this.priority = priority; this.maxAge = maxAge; this.timescale = timescale === undefined ? undefined : Timescale(timescale); + this.unverified = unverified; } async #encode(w: Writer, version: Version) { @@ -106,6 +114,7 @@ export class TrackInfo { await w.u62(legacy ? (age ?? BigInt(Number.MAX_SAFE_INTEGER)) : age === undefined ? 0n : age + 1n); await w.u53(wireTimescale(version, this.timescale) ?? 0); if (hasRefresh(version)) await w.u53(this.refresh === undefined ? 0 : this.refresh + 1); + if (hasUnverified(version)) await w.bool(this.unverified); } static async #decode(r: Reader, version: Version): Promise { @@ -127,7 +136,25 @@ export class TrackInfo { const scale = await r.u53(); const timescale = scale === 0 && !legacy ? undefined : Timescale(scale); const refresh = hasRefresh(version) ? (await r.u53()) - 1 : undefined; - return new TrackInfo({ priority, maxAge, timescale, refresh: refresh === -1 ? undefined : refresh }); + let unverified: boolean; + if (hasUnverified(version)) { + const flag = await r.u8(); + // Only a timed track has timestamps to doubt. + if (flag > 1 || (flag === 1 && timescale === undefined)) + throw new RangeError(`invalid unverified flag: ${flag}`); + unverified = flag === 1; + } else { + // Lite-05/06 always carry a timescale, and an untimed track's send times can't be told + // from media time. + unverified = timescale !== undefined; + } + return new TrackInfo({ + priority, + maxAge, + timescale, + refresh: refresh === -1 ? undefined : refresh, + unverified, + }); } async encode(w: Writer, version: Version): Promise { @@ -146,6 +173,8 @@ export class TrackInfo { ) throw new RangeError("refresh exceeds u32"); if (this.timescale !== undefined) Timescale(this.timescale); + if (this.unverified && this.timescale === undefined) + throw new RangeError("an untimed track can't be unverified"); return Message.encode(w, (w) => this.#encode(w, version)); } diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index 7ad0208cba6..63b61139737 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -404,6 +404,23 @@ export function hasRefresh(version: Version): boolean { } } +/** Whether TRACK_INFO says if a timed track's timestamps are unverified. Added in lite-07; earlier, + * a receiver cannot tell an untimed track's send times from media time, so every timed track is. */ +export function hasUnverified(version: Version): boolean { + // Explicitly list older versions so future versions keep the lite-07+ behavior. + switch (version) { + case Version.DRAFT_01: + case Version.DRAFT_02: + case Version.DRAFT_03: + case Version.DRAFT_04: + case Version.DRAFT_05: + case Version.DRAFT_06: + return false; + default: + return true; + } +} + /// The WebTransport subprotocol identifier for moq-lite. /// Version negotiation still happens via SETUP when this is used. export const ALPN = "moql"; diff --git a/js/net/src/track.ts b/js/net/src/track.ts index d6f11592e30..d6bb86b117d 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -106,6 +106,15 @@ export interface Info { * snapshot or keyframe. `0` asks on any join after a delta. An integer in `0..2^32`. */ refresh?: number; + // pending: /quest/m1/public-model/ingest.md + /** + * Whether a hop stamped this timed track's timestamps, or received them over a wire that + * cannot say it didn't, so they may rewind and no group is checked for it. A local + * publisher's writes are always checked. Mirrors the Rust `track::Info::unverified`. + * + * @internal + */ + unverified?: boolean; } // Normalize a duration for the wire, which carries it as an unsigned varint. @@ -134,11 +143,15 @@ function priorityByte(value: number): number { /** Fill in any unset {@link Info} fields with their defaults. */ export function infoDefaults(info: Partial = {}): Info { + const timescale = info.timescale == null ? undefined : Timescale(info.timescale); + // Only a timed track has timestamps to doubt. + if (info.unverified && timescale === undefined) throw new RangeError("an untimed track can't be unverified"); return { - timescale: info.timescale == null ? undefined : Timescale(info.timescale), + timescale, maxAge: info.maxAge === undefined ? undefined : wireMillis("maxAge", info.maxAge), priority: priorityByte(info.priority ?? DEFAULT_PRIORITY), refresh: info.refresh == null ? undefined : refreshRatio(info.refresh), + ...(info.unverified ? { unverified: true } : {}), }; } @@ -673,9 +686,6 @@ export class Producer { // group above it is checked against it as well as its cached neighbors; one at or below // it may have lost its lower bound. One entry, so nothing grows with the cache. #evicted?: Start; - // A session's copy of a remote track, whose groups are never checked for a rewind: a - // received timestamp may not be media time (a lite-05/06 untimed track carries send times). - #remote = false; // When the cache was last scanned. See #prune. #pruned = Number.NEGATIVE_INFINITY; // When the track closed, which restarts the former newest group's idle window. See #prune. @@ -730,9 +740,6 @@ export class Producer { producer.#served = served; producer.#owe(); }; - hooks.receiveTrack = (producer) => { - producer.#remote = true; - }; // Anything written before binding still counts, so the namespace never goes backwards. bindProducerSequence = (producer, sequence) => { const own = producer.#sequence; @@ -1199,8 +1206,8 @@ export class Producer { this.#drop(group.sequence, group.sequence + 1); }); const sequence = group.sequence; - if (this.#remote) { - // Only a local publisher's writes are checked. + if (this.#unverified()) { + // Its timestamps may rewind, so they are not checked. } else if (start) { this.#record(entry, start, late); } else { @@ -1333,12 +1340,17 @@ export class Producer { } // A group written before it joined is checked here, before it is cached or fanned out. - const start = this.#remote ? undefined : hooks.groupStart(group); + const start = this.#unverified() ? undefined : hooks.groupStart(group); const late = start !== undefined && this.#admit(group.sequence, start); this.#publish(group, start, late); } + // Whether this track's timestamps may rewind (see {@link Info.unverified}). + #unverified(): boolean { + return this.#state.info.peek()?.unverified === true; + } + // Fan a datagram out to every live subscriber, dropping the oldest once the ring is full. // Late subscribers do NOT replay old datagrams (best-effort, unlike the group cache). #publishDatagram(datagram: Datagram): void { diff --git a/quest/m1/README.md b/quest/m1/README.md index 239d3f44c0c..91dca9f620d 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -102,7 +102,6 @@ lands before the cut. - [Resumed in-flight groups](/quest/m1/resume-in-flight.md) - a replaced route copy stays while a group below its bound is in flight or it backs a handed-out group late, and no longer than the reader's budget - [Resume through an outage](/quest/m1/resume-outage.md) - a same-epoch resume after an unclosed route dies delivers the backlog its max age still holds, pinned by a regression test - [Resumed datagrams](/quest/m1/resume-datagrams.md) - a resumed track skips only datagrams it already delivered, not every lower sequence -- [Received timestamps are media time](/quest/m1/lite-time-provenance.md) - a received timed track's timestamps are media time, so the rewind check can run across hops - [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS - [A failed HLS export answers with an error](/quest/m1/hls-failure-signal.md) - playlists answer a no-store 5xx once their timeline fails, and so does the DASH manifest for a shared reference failure, so a truncated recording never looks complete - [Close tail drain](/quest/m1/close-tail-drain.md) - after a peer close, subscribers read the group streams their transport already delivered instead of aborting them, so a finished tail arrives on every version diff --git a/quest/m1/lite-time-provenance.md b/quest/m1/lite-time-provenance.md deleted file mode 100644 index 586562c9b38..00000000000 --- a/quest/m1/lite-time-provenance.md +++ /dev/null @@ -1,38 +0,0 @@ -# [M] Received timestamps are media time - -## Goal - -A receiver can trust a received timed track's timestamps as media time, so -the rewind check that today runs only on a local publisher's writes can run -across hops too. - -## Plan - -Split from [#5803](https://github.com/moq-dev/moq/pull/5803), which refuses a -rewind only on a local publisher's writes: a relay hop that checked received -groups closed tracks whose timestamps were never media time. - -Where non-media timestamps come from today: - -- Released clients stamp wall-clock write time on data tracks: the moq-mux - catalog, moq-json, moq-stats, moq-room chat, js/json, js/binary, and - js/net's `writeString`, `writeJson` and `writeBool`. -- Released relays stamp arrival time on input that carries no timestamps - (IETF, and lite-04 or older) and forward it as media time. -- Unreleased main stamps send time on untimed tracks over lite-05/06, which - cannot encode an untimed track - ([#4822](https://github.com/moq-dev/moq/pull/4822), - [#4968](https://github.com/moq-dev/moq/pull/4968)). -- Media tracks carry real presentation times. - -Options weighed: - -1. Fix our unreleased publishers and relays so only media time reaches the - wire, and stop stamping data tracks. Recommended. -2. Reserve an untimed marker value on lite-05/06. -3. Treat a 1000 timescale as untimed. -4. Distrust every lite-05/06 timestamp. -5. A lite-07 TRACK_INFO marker for timestamp provenance. - -Released peers still send wall-clock stamps, so whichever option lands must -say which received tracks the check may trust. diff --git a/quest/m1/public-model/ingest.md b/quest/m1/public-model/ingest.md index 3c328a23b16..8057f2b57af 100644 --- a/quest/m1/public-model/ingest.md +++ b/quest/m1/public-model/ingest.md @@ -170,11 +170,12 @@ Rust items (`rs/moq-net/src/model/`), as they stood at b4eab0db1: `track::TrackState`, `track::FetchOutcome`, `finish_track_at`, `poll_landed`): moved to [the deferred quest](/quest/m1/public-model/deferred.md), which waits on Model ranges. -- `track::Producer::received` (lite and IETF subscribers; JS - `hooks.receiveTrack`) marks a session's copy so the local-write rewind - check skips it (#5803). It goes public with the other ingest calls, unless - [time provenance](/quest/m1/lite-time-provenance.md) makes received - timestamps checkable first. +- `track::Info::unverified` (JS `Info.unverified`, `@internal`) marks a + track whose timestamps may rewind, so no group is checked. Sessions set + it on ingress over moq-lite-05/06 and moq-transport, and lite-07 carries + it in TRACK_INFO. A custom session's copies stay verified, so they are + checked; expose it with the other ingest calls only if one must ingest + send or arrival times. - Passthrough, stay private and marked: `group::Producer::wire`, `set_wire`, `mark_gap`, `set_end_object`, `group::Wire`, the `object` and `datagram` modules, `track::Producer::insert_wire_datagram`, diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index 8009db2c319..6a2928d635b 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -103,7 +103,7 @@ let rate = bandwidth.as_ref().and_then(|b| b.peek()); ### A timed track refuses a group that rewinds (#5803) A local write whose first frame starts before a lower-sequence group's, or after a higher one's, fails with `TimestampMismatch`; `@moq/net` throws it the same way. -Received groups are accepted as is, since their timestamps may be send times. +A received group is checked too when every hop vouched for it over moq-lite-07. One received over moq-lite-05/06 or moq-transport is accepted as is, since its timestamps may be send or arrival times. A publisher whose timeline restarts, such as an HLS `EXT-X-DISCONTINUITY` that resets decode times, publishes a new broadcast instead. ### `broadcast::Producer::finish` is back, and lingers (#5795) diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index 29abae75ae5..a5e884970dd 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -645,6 +645,7 @@ impl Default for Messages { priority: 1, max_age: Some(Duration::from_secs(10)), timescale: Some(crate::Timescale::MICRO), + unverified: false, }, lite_group: lite::Group { subscribe: 7, diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index d590a411019..1f00cd93ea7 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -477,6 +477,8 @@ fn accepted_info(properties: &ietf::Properties) -> track::Info { .with_refresh(properties.dynamic_groups.then_some(0)); info.properties = Some(std::sync::Arc::new(properties.unknown.clone())).filter(|unknown| !unknown.values.is_empty()); + // A moq-transport peer may stamp arrival times, and nothing on the wire says it didn't. + info.unverified = info.timescale.is_some(); info } @@ -3006,7 +3008,6 @@ where (request.accept(info), dynamic) } }; - track.received(); // A live join starts at the publisher's edge; an absolute one where it asked. With // no Largest the join skips nothing, so every group is owed: a lower one's stream // may still arrive after a higher one's. @@ -3868,7 +3869,6 @@ where tracing::debug!(broadcast = %self.origin.absolute(broadcast_path), track = %track_name, "track status known"); let dynamic = request.dynamic(); let mut track = request.accept(accepted_info(&ok.properties)); - track.received(); // Nothing feeds this copy live: what it caches is fetched, never the live edge. track.set_idle(); Some(Idle { @@ -12007,12 +12007,11 @@ mod stitch_tests { ); // The subscriber accepts a timed track at microseconds, matching `run_subscribe`. - let track = track::Producer::new( - std::sync::Arc::new(crate::broadcast::Info::default()), - "video", - track::Info::default().with_timescale(timescale.map(|_| Timescale::MICRO)), - ); - track.received(); + let info = accepted_info(&ietf::Properties { + timescale: timescale.map(|_| Timescale::MICRO), + ..Default::default() + }); + let track = track::Producer::new(std::sync::Arc::new(crate::broadcast::Info::default()), "video", info); let fill = kio::Producer::new(fill); { @@ -14041,8 +14040,8 @@ mod stitch_tests { assert!(h.session.log.closes().is_empty()); } - /// A received group whose first object rewinds past the group before it is accepted: - /// only a local publisher's writes refuse a rewind. + /// A received group whose first object rewinds past the group before it is accepted: a + /// moq-transport peer may stamp arrival times, so its timed tracks are unverified. #[moq_clock::test] async fn a_received_rewind_is_accepted() { use futures::FutureExt; diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 63504585370..32be630bc12 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -1957,12 +1957,7 @@ impl Request for TrackInfoServe { let info = ready!(self.querying.poll_ok(waiter))?; // TRACK_INFO only flows on Lite05+ (the encode errors otherwise). - writer.buffer(&lite::TrackInfo { - refresh: info.refresh, - priority: info.priority, - max_age: info.max_age, - timescale: info.timescale, - })?; + writer.buffer(&lite::TrackInfo::from(&info))?; if ietf { writer.buffer(&lite::ietf::TrackProperties(info.properties.clone()))?; } @@ -2019,12 +2014,7 @@ impl Request for TracksServe { let event = match ready!(self.tracks.poll_next(waiter))? { Some(crate::broadcast::TracksEvent::Added { name, info }) => lite::TracksEvent::Added { name: name.to_string().into(), - info: lite::TrackInfo { - refresh: info.refresh, - priority: info.priority, - max_age: info.max_age, - timescale: info.timescale, - }, + info: lite::TrackInfo::from(&info), }, Some(crate::broadcast::TracksEvent::Removed { name }) => lite::TracksEvent::Removed { name: name.to_string().into(), diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 2254f3c7014..76a50a59c9a 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -1780,12 +1780,7 @@ impl TracksRecv { return Poll::Ready(Err(Error::ProtocolViolation)); } let slot = source.subscriber.announces.acquire()?; - let info = track::Info::default() - .with_timescale(info.timescale) - .with_max_age(info.max_age) - .with_priority(info.priority) - .with_refresh(info.refresh); - source.lister.add(name.clone(), info)?; + source.lister.add(name.clone(), info.into())?; self.listed.insert(name, slot); } lite::TracksEvent::Removed { name } => { @@ -2828,8 +2823,9 @@ mod tests { } /// Receive group 5 starting at 500ms over `version`, on a received timed track whose - /// group 4 starts at 1000ms, returning the stream's result and the track. - async fn receive_rewind(version: Version) -> (Result<(), Error>, track::Producer) { + /// group 4 starts at 1000ms and whose publisher's TRACK_INFO said `unverified`, returning + /// the stream's result and the track. + async fn receive_rewind(version: Version, unverified: bool) -> (Result<(), Error>, track::Producer) { use crate::transport::poll::Session as _; let mut script = Vec::new(); @@ -2858,12 +2854,18 @@ mod tests { auth: crate::auth::Handle::new(false), }); + let sent = lite::TrackInfo { + unverified, + ..lite::TrackInfo::from(&crate::track::Info::timed()) + } + .encode_bytes(version) + .unwrap(); + let info = lite::TrackInfo::decode_slice(&sent, version).unwrap().0; let track = track::Producer::new( std::sync::Arc::new(crate::broadcast::Info::default()), "video", - crate::track::Info::timed(), + track::Info::from(info), ); - track.received(); let mut before = track.create_group(4u64.into()).unwrap(); before .write_frame(Timestamp::from_millis(1000).unwrap(), &b"before"[..]) @@ -2884,18 +2886,30 @@ mod tests { (res, track) } - /// A received timestamp may not be media time (lite-05/06 untimed tracks carry send - /// times), so a received group that rewinds is accepted at every version and the track - /// stays open. Only a local publisher's writes refuse a rewind. + /// A received group that rewinds is accepted, and the track stays open, wherever its + /// timestamps are unverified: always on lite-05/06, whose untimed tracks carry send times, + /// and on lite-07 when the publisher says so. #[moq_clock::test] - async fn a_received_rewind_is_accepted_at_every_version() { - for version in [Version::Lite05, Version::Lite06, Version::Lite07] { - let (res, track) = receive_rewind(version).await; + async fn an_unverified_rewind_is_accepted() { + for (version, unverified) in [ + (Version::Lite05, false), + (Version::Lite06, false), + (Version::Lite07, true), + ] { + let (res, track) = receive_rewind(version, unverified).await; res.unwrap_or_else(|err| panic!("{version}: {err:?}")); assert!(track.closed().now_or_never().is_none(), "{version}"); } } + /// A lite-07 track whose publisher vouches for its timestamps is checked like a local + /// write, so a received group that rewinds is refused. + #[moq_clock::test] + async fn a_verified_rewind_is_refused() { + let (res, _track) = receive_rewind(Version::Lite07, false).await; + assert!(matches!(res, Err(Error::TimestampMismatch)), "{res:?}"); + } + /// An idle copy that resubscribes gets its newest cached group again. That stream only /// duplicates the cache, but lite-07's SUBSCRIBE_END still counts it, so the tail must /// too, or the subscription waits out the grace before it ends. @@ -6395,11 +6409,7 @@ impl TrackInfoFetch { // window matches what the upstream advertises (relays re-serve with // the same bound). `broadcast` is left at its default here; // `track::Request::accept` stamps the track's real broadcast. - let model = track::Info::default() - .with_timescale(info.timescale) - .with_max_age(info.max_age) - .with_priority(info.priority) - .with_refresh(info.refresh); + let model = track::Info::from(info); if !ietf { return Poll::Ready(Ok((model, HeldTrack(stream)))); } @@ -6666,7 +6676,6 @@ impl ServeLoop { false => request, }; let serving = request.accept(info); - serving.received(); Self { largest: None, demand: serving.demand(), diff --git a/rs/moq-net/src/lite/track.rs b/rs/moq-net/src/lite/track.rs index a2a5f5ca5f1..2f8cdf2d0b1 100644 --- a/rs/moq-net/src/lite/track.rs +++ b/rs/moq-net/src/lite/track.rs @@ -70,6 +70,34 @@ pub struct TrackInfo { /// Lite07 encodes `None` as zero; Lite05/06 have no untimed encoding, so `None` /// goes out as milliseconds (see [`Version::wire_timescale`]). pub timescale: Option, + /// Whether a timed track's timestamps may rewind, so receivers don't check them (see + /// [`crate::track::Info::unverified`]). Carried on lite-07; every timed track received + /// over lite-05/06 is unverified. + pub unverified: bool, +} + +impl From<&crate::track::Info> for TrackInfo { + fn from(info: &crate::track::Info) -> Self { + Self { + refresh: info.refresh, + priority: info.priority, + max_age: info.max_age, + timescale: info.timescale, + unverified: info.unverified, + } + } +} + +impl From for crate::track::Info { + fn from(info: TrackInfo) -> Self { + let mut model = Self::default() + .with_timescale(info.timescale) + .with_max_age(info.max_age) + .with_priority(info.priority) + .with_refresh(info.refresh); + model.unverified = info.unverified; + model + } } impl Message for TrackInfo { @@ -101,11 +129,24 @@ impl Message for TrackInfo { false => None, }; + let unverified = match version.has_unverified() { + true => match r.u8()? { + 0 => false, + // Only a timed track has timestamps to doubt. + 1 if timescale.is_some() => true, + _ => return Err(DecodeError::InvalidValue), + }, + // lite-05/06 always carry a timescale, and an untimed track's send times can't be + // told from media time. + false => timescale.is_some(), + }; + Ok(Self { priority, refresh, max_age, timescale, + unverified, }) } @@ -127,6 +168,12 @@ impl Message for TrackInfo { if version.has_refresh() { w.varint(self.refresh.map_or(0, |ratio| u64::from(ratio) + 1))?; } + if version.has_unverified() { + if self.unverified && self.timescale.is_none() { + return Err(EncodeError::InvalidState); + } + w.u8(self.unverified.into()); + } Ok(()) } } @@ -141,6 +188,7 @@ mod test { priority: 7, max_age: Some(Duration::from_millis(2000)), timescale: Some(Timescale::MICRO), + unverified: false, } } @@ -237,14 +285,8 @@ mod test { #[test] fn track_info_defaults_match_cross_language_wire_bytes() { - let info = crate::track::Info::default(); - let info = TrackInfo { - refresh: None, - priority: info.priority, - max_age: info.max_age, - // The default track is untimed, which lite-05 declares as milliseconds. - timescale: info.timescale, - }; + // The default track is untimed, which lite-05 declares as milliseconds. + let info = TrackInfo::from(&crate::track::Info::default()); let mut buf = Vec::new(); info.encode(&mut Encoder::new(&mut buf, Version::Lite05.into()), Version::Lite05) .unwrap(); @@ -268,6 +310,77 @@ mod test { assert_eq!(info_roundtrip(Version::Lite07, &info).timescale, None); } + #[test] + fn lite07_carries_unverified() { + for unverified in [false, true] { + let info = TrackInfo { + unverified, + ..info_sample() + }; + let buf = info.encode_bytes(Version::Lite07).unwrap(); + assert_eq!(buf.last(), Some(&u8::from(unverified))); + assert_eq!(info_roundtrip(Version::Lite07, &info).unverified, unverified); + } + } + + #[test] + fn unverified_matches_cross_language_wire_bytes_on_lite07() { + let info = TrackInfo { + refresh: None, + priority: 1, + max_age: None, + timescale: Some(Timescale::MILLI), + unverified: true, + }; + let buf = info.encode_bytes(Version::Lite07).unwrap(); + assert_eq!(buf.as_ref(), [0x06, 0x01, 0x00, 0x83, 0xe8, 0x00, 0x01]); + } + + /// lite-05/06 can't say, and an untimed track's send times look like media time there. + #[test] + fn lite06_timed_tracks_arrive_unverified() { + for version in [Version::Lite05, Version::Lite06] { + for timescale in [None, Some(Timescale::MICRO)] { + let info = TrackInfo { + timescale, + ..info_sample() + }; + assert!(info_roundtrip(version, &info).unverified, "{version}"); + } + } + } + + #[test] + fn lite07_refuses_unverified_on_an_untimed_track() { + let info = TrackInfo { + timescale: None, + unverified: true, + ..info_sample() + }; + assert!(matches!( + info.encode_bytes(Version::Lite07), + Err(EncodeError::InvalidState) + )); + + let mut raw = TrackInfo { + timescale: None, + ..info_sample() + } + .encode_bytes(Version::Lite07) + .unwrap() + .to_vec(); + *raw.last_mut().unwrap() = 1; + assert!(matches!( + TrackInfo::decode_slice(&raw, Version::Lite07), + Err(DecodeError::InvalidValue) + )); + *raw.last_mut().unwrap() = 2; + assert!(matches!( + TrackInfo::decode_slice(&raw, Version::Lite07), + Err(DecodeError::InvalidValue) + )); + } + #[test] fn lite06_declares_an_untimed_track_as_milliseconds() { for version in [Version::Lite05, Version::Lite06] { @@ -312,6 +425,7 @@ mod test { priority: 255, max_age: None, timescale: Some(Timescale::new((1u64 << 62) - 1).unwrap()), + unverified: false, }; let got = info_roundtrip(Version::Lite05, &info); assert_eq!(got.priority, 255); @@ -326,6 +440,7 @@ mod test { priority: 0, max_age: Some(Duration::from_nanos(999_999)), timescale: Some(Timescale::MILLI), + unverified: false, }; let got = info_roundtrip(Version::Lite05, &info); assert_eq!(got.max_age, Some(Duration::ZERO)); @@ -339,6 +454,7 @@ mod test { priority: 7, max_age: Some(Duration::from_millis(u64::MAX)), timescale: Some(Timescale::MILLI), + unverified: false, }; // The partial bytes are the Writer's to drop; see `a_failed_encode_leaves_no_partial_bytes`. assert!(matches!( diff --git a/rs/moq-net/src/lite/tracks.rs b/rs/moq-net/src/lite/tracks.rs index c39fdeb40cb..650c9d245c3 100644 --- a/rs/moq-net/src/lite/tracks.rs +++ b/rs/moq-net/src/lite/tracks.rs @@ -131,6 +131,7 @@ mod test { priority: 3, max_age: Some(Duration::from_secs(2)), timescale: Some(Timescale::MICRO), + unverified: true, }; let TracksEvent::Added { name, info: got } = roundtrip(&TracksEvent::Added { name: "video".into(), diff --git a/rs/moq-net/src/lite/version.rs b/rs/moq-net/src/lite/version.rs index 543028c8419..7e28ba9cab1 100644 --- a/rs/moq-net/src/lite/version.rs +++ b/rs/moq-net/src/lite/version.rs @@ -320,6 +320,18 @@ impl Version { } } + /// Whether TRACK_INFO says if a timed track's timestamps are unverified. Added in + /// lite-07; earlier, a receiver cannot tell an untimed track's send times from media + /// time, so it treats every timed track as unverified. + #[allow(clippy::match_like_matches_macro)] + pub(crate) fn has_unverified(self) -> bool { + // Match form so future versions default forward (AGENTS.md convention). + match self { + Self::Lite01 | Self::Lite02 | Self::Lite03 | Self::Lite04 | Self::Lite05 | Self::Lite06 => false, + _ => true, + } + } + /// Whether discovery streams end their initial set with SYNCED rather than a count. #[allow(clippy::match_like_matches_macro)] pub fn has_synced(self) -> bool { diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index db1942a7c40..ad143f4b2ea 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -157,6 +157,16 @@ pub struct Info { #[cfg_attr(not(feature = "async"), allow(dead_code))] // private: passthrough pub(crate) properties: Option>, + /// Whether a hop stamped this timed track's timestamps, or received them over a wire + /// that cannot say they weren't, so they may rewind and no group is checked for it. + /// + /// A local publisher's writes are always checked. A received track is unverified when it + /// arrived over moq-lite 05/06 (an untimed track there carries send times) or + /// moq-transport (a peer may stamp arrival times), or over lite-07 from a hop that says + /// so. Never set on an untimed track. + #[cfg_attr(not(feature = "async"), allow(dead_code))] + // pending: /quest/m1/public-model/ingest.md + pub(crate) unverified: bool, } impl Default for Info { @@ -167,6 +177,7 @@ impl Default for Info { priority: DEFAULT_PRIORITY, refresh: None, properties: None, + unverified: false, } } } @@ -411,9 +422,6 @@ pub(crate) struct TrackState { // [`Subscriber::poll_start`]) wait on it; everything else treats the floor as usual. start_pending: bool, - // A session's copy of a remote track. Its groups skip the rewind check: a received - // timestamp may not be media time (a lite-05/06 untimed track carries send times). - received: bool, // Sequences the upstream publisher gave up: an incoming SUBSCRIBE_DROP or a producer // cut. Kept apart from the cache so reclaiming a slot cannot erase a disposition a // relay must forward, and as ranges so a wide DROP costs one entry. @@ -774,9 +782,9 @@ impl TrackState { /// so the watermark caps it instead. /// /// Usually one group lookup each way; only unstarted or partial neighbors are skipped. - /// `None` on a received track, which only a local publisher's writes are checked on. + /// `None` on an [unverified](Info::unverified) track, whose timestamps may rewind. pub(super) fn neighbors(&self, sequence: u64) -> Option { - if self.received { + if self.info.as_ref().is_some_and(|info| info.unverified) { return None; } let bounds = self.bounds.lock().unwrap_or_else(PoisonError::into_inner); @@ -2810,16 +2818,6 @@ impl Producer { Ok(()) } - /// Mark this track as a session's copy of a remote one: its groups are never checked - /// for a rewind, which only a local publisher's writes refuse. - #[cfg_attr(not(feature = "async"), allow(dead_code))] - // pending: /quest/m1/public-model/ingest.md - pub(crate) fn received(&self) { - if let Ok(mut state) = self.modify() { - state.received = true; - } - } - /// Record sequences the upstream publisher explicitly gave up. #[cfg_attr(not(feature = "async"), allow(dead_code))] // pending: /quest/m1/public-model/ingest.md diff --git a/rs/moq-net/src/test_interop.rs b/rs/moq-net/src/test_interop.rs index a72b0dd1480..d53ae3018a0 100644 --- a/rs/moq-net/src/test_interop.rs +++ b/rs/moq-net/src/test_interop.rs @@ -174,6 +174,7 @@ fn lite_varint_interop() { priority: 1, max_age: None, timescale: None, + unverified: false, } .encode_bytes(version) .unwrap(); diff --git a/rs/moq-net/tests/integration/received_rewind.rs b/rs/moq-net/tests/integration/received_rewind.rs index 5cc999f7143..73eb0f3907f 100644 --- a/rs/moq-net/tests/integration/received_rewind.rs +++ b/rs/moq-net/tests/integration/received_rewind.rs @@ -1,11 +1,11 @@ #![cfg(feature = "async")] -//! A relay re-serves what it received without checking it for a rewind. +//! A relay re-serves unverified timestamps as unverified. //! //! Lite-06 stamps an untimed track's frames with their send time and declares //! milliseconds, so the relay's copy reads as timed and re-advertises that scale. A group -//! sent after a higher one carries the later stamp, which the next hop must not refuse: -//! only a local publisher's writes are checked. +//! sent after a higher one carries the later stamp, which no hop may refuse: the relay's +//! copy is unverified, and it says so to a lite-07 subscriber. use crate::support; From 62e8288a9dd8649a93ebd4b04a2f87a3da16a9ef Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 22:31:39 -0700 Subject: [PATCH 03/14] feat(lite): drop lite-05/06 timestamps instead of flagging them A receiver drops every timestamp it gets over moq-lite-05/06, whose send times look like media time, so those tracks arrive and are forwarded untimed. The lite-07 TRACK_INFO Unverified byte is gone. Timed tracks received over moq-transport stay unchecked and are served to moq-lite untimed. Co-Authored-By: Claude Opus 5.5 --- doc/concept/moq-lite.md | 10 +- doc/setup/upgrade.md | 6 +- drafts/draft-lcurley-moq-lite.md | 17 +-- js/e2ee/src/transport.test.ts | 4 +- js/net/MIGRATE.md | 4 +- js/net/src/ietf/subscriber.ts | 2 +- js/net/src/integration.test.ts | 16 +-- js/net/src/lite/group.ts | 18 ++- js/net/src/lite/publisher.ts | 9 +- js/net/src/lite/subscriber.test.ts | 31 +++-- js/net/src/lite/subscriber.ts | 14 ++- js/net/src/lite/track.test.ts | 32 ------ js/net/src/lite/track.ts | 33 +----- js/net/src/lite/track_stream.test.ts | 32 ++++++ js/net/src/lite/version.ts | 7 +- js/net/src/track.ts | 30 +++-- js/net/src/untimed.test.ts | 46 +++----- quest/m1/public-model/ingest.md | 11 +- rs/moq-net/MIGRATE.md | 4 +- rs/moq-net/src/fuzz.rs | 1 - rs/moq-net/src/ietf/subscriber.rs | 55 ++++----- rs/moq-net/src/lite/publisher.rs | 20 ++-- rs/moq-net/src/lite/subscriber.rs | 82 +++++++------ rs/moq-net/src/lite/track.rs | 108 +----------------- rs/moq-net/src/lite/tracks.rs | 1 - rs/moq-net/src/lite/version.rs | 8 +- rs/moq-net/src/model/group.rs | 8 ++ rs/moq-net/src/model/track.rs | 26 +++-- rs/moq-net/src/test_interop.rs | 1 - rs/moq-net/tests/integration/datagram.rs | 3 +- .../tests/integration/received_rewind.rs | 57 ++++++--- rs/moq-net/tests/integration/untimed.rs | 31 ++--- rs/moq-net/tests/integration/wide_sequence.rs | 6 +- 33 files changed, 323 insertions(+), 410 deletions(-) diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index e55a178250e..b8fba9cf252 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -280,10 +280,12 @@ untimed. moq-lite 05 and 06 cannot declare an untimed track, so there its frames go out stamped with their send time; the work-in-progress 07 carries it untimed. -A publisher refuses a group that starts before a lower one. A receiver checks -received groups the same way only when every hop could vouch for them: a timed -track received over moq-lite 05 or 06, or over moq-transport, is unverified, and -the work-in-progress 07 says so as it passes through relays. +A publisher refuses a group that starts before a lower one, and so does a +receiver. Since moq-lite 05 and 06 send times look like media time, a receiver +drops every timestamp it gets over them and the track arrives untimed. +Timestamps received over moq-transport, whose peers may stamp arrival times, +are not checked and are only passed on over moq-transport: a relay serves that +track to moq-lite untimed. A relay cancels its upstream subscription once nobody subscribes, but keeps its copy of the track for 30 seconds after the last reader leaves, so a returning diff --git a/doc/setup/upgrade.md b/doc/setup/upgrade.md index e5163fe8bfb..45797fc89c1 100644 --- a/doc/setup/upgrade.md +++ b/doc/setup/upgrade.md @@ -65,8 +65,10 @@ These land with the next breaking release. - **A track is timed or untimed, and nothing fills in a timestamp on receive.** Tracks from moq-lite before 05, and moq-transport tracks without `TIMESCALE` (every track on drafts 14 to 16), arrive untimed instead of - stamped with their arrival time. moq-lite 05 and later can't encode absence - yet and carry a send time instead; see + stamped with their arrival time. moq-lite 05 and 06 can't encode absence and + carry a send time instead, so their receivers drop every timestamp and those + tracks arrive untimed too: read media time from the payload. moq-transport + timestamps are served to moq-lite untimed; see [untimed tracks](/concept/moq-lite#subscriptions). A new subscriber with no start on an untimed track starts at the latest group. - **Max age and max delay count wall-clock time too.** A superseded group is diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 108422e6105..5f3e99c24eb 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -176,8 +176,9 @@ Each frame of a timed Track carries a presentation timestamp expressed in the Tr A Track without a `Timescale` is untimed: its frames and datagrams carry no timestamp. A publisher MUST NOT start a timed Group's frame 0 earlier than that of any lower-sequence Group it publishes. lite-05 and lite-06 cannot encode an untimed Track, so a publisher on those versions declares a `Timescale` of 1000 for it and stamps every frame and datagram with its send time in milliseconds. -A receiver cannot tell such send times from presentation timestamps, and they need not rise with sequence, so it treats every timed Track received on those versions as unverified (see `Unverified` in [TRACK_INFO](#track-info)). -A receiver MAY check a Group of a verified Track against this rule, refusing a violating Group's stream (STOP_SENDING) with TIMESTAMP_MISMATCH, and MUST NOT check one of an unverified Track. +A receiver cannot tell such send times from presentation timestamps, and they need not rise with sequence, so it discards every timestamp received on those versions and treats the Track as untimed. +On later versions a receiver MAY check a received Group against this rule, refusing a violating Group's stream (STOP_SENDING) with TIMESTAMP_MISMATCH. +A relay that cannot vouch for a Track's timestamps, such as ones received over another protocol, MUST serve the Track untimed. ## Positions {#positions} A Position is a (Group Sequence, Frame Index) pair identifying one frame within a Track. @@ -358,7 +359,7 @@ Sent when resetting a stream (RESET_STREAM), or when refusing to receive one (ST | ------- | ------------- | ----------- | | 0x38 | FRAME_TOO_LARGE | A frame declared a payload larger than the receiver accepts. | | ------- | ------------- | ----------- | -| 0x39 | TIMESTAMP_MISMATCH | A frame's timestamp does not fit its track: it is present on an untimed track, absent on a timed one, or starts a verified Group before a lower-sequence one. | +| 0x39 | TIMESTAMP_MISMATCH | A frame's timestamp does not fit its track: it is present on an untimed track, absent on a timed one, or starts a Group before a lower-sequence one. | | ------- | ------------- | ----------- | | 0x3A | NOT_FETCHABLE | The FETCH named a group delivered only as a datagram, which is never cached (see [Datagrams](#datagrams)). It has no moq-transport value; a bridge refuses the FETCH with DOES_NOT_EXIST. | | ------- | ------------- | ----------- | @@ -1307,7 +1308,6 @@ TRACK_INFO Message { Publisher Max Age (i) Timescale (i) Refresh (i) - Unverified (8) } ~~~ @@ -1347,13 +1347,6 @@ Joins coalesce until the requested group arrives. A relay preserves the ratio and evaluates joins against its own cache, forwarding demand in SUBSCRIBE_UPDATE. Versions before lite-07 omit Refresh and Wanted Group; a Track received on those versions never refreshes. -**Unverified**: -1 when the Track's timestamps may break the rule that no Group starts before a lower-sequence one (see [Frame](#frame)), 0 otherwise. -A publisher sets it when a hop stamped the timestamps (a send or arrival time) or it received the Track on a wire that cannot say otherwise: a timed Track received over lite-05, lite-06, or MoqTransport, or with `Unverified` set to 1. -A relay preserves it, so a receiver checks a Track only when every hop since its publisher could. -It MUST be 0 on an untimed Track; a receiver MUST treat any other value as a protocol violation. -Versions before lite-07 omit Unverified. - ## TRACKS {#tracks} TRACKS is sent by a subscriber to list a Broadcast's Tracks. It is the first message on a Tracks Stream (0x8). @@ -1740,7 +1733,7 @@ The `Message Length` describes the payload size on the wire. - Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery. -- A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's. Added TRACK_INFO `Unverified`, set when a hop stamped the timestamps or received them over lite-05/06 or MoqTransport, which cannot say they weren't. A receiver may check only a verified Track. +- A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's, and a receiver may refuse a Group that does. A receiver discards the timestamps it receives over lite-05/06, which may be send times, and a relay serves a Track whose timestamps it cannot vouch for untimed. - Added the reliable ordered RTCDataChannel binding, with one Qmux Record per binary message and no datagrams. - Added `Group Order` to SUBSCRIBE and SUBSCRIBE_UPDATE: descending (0x0, the previous fixed order) or ascending (0x1). A relay requests descending upstream unless every subscription it serves is ascending. diff --git a/js/e2ee/src/transport.test.ts b/js/e2ee/src/transport.test.ts index acd755d131d..84588ae7242 100644 --- a/js/e2ee/src/transport.test.ts +++ b/js/e2ee/src/transport.test.ts @@ -48,10 +48,10 @@ async function active(origin: Origin.Producer, path: Path.Valid): Promise BigInt(Number.MAX_SAFE_INTEGER)) { throw new RangeError("largest group exceeds safe integers"); diff --git a/js/net/src/integration.test.ts b/js/net/src/integration.test.ts index 213c5667df9..de3acfd7ecf 100644 --- a/js/net/src/integration.test.ts +++ b/js/net/src/integration.test.ts @@ -984,10 +984,10 @@ test("integration: a peer is served the cheaper route it was offered, not the lo origin.close(); }); -test("integration: lite draft-05 fetches a cached group", async () => { +test("integration: lite draft-07 fetches a cached group", async () => { const enc = new TextEncoder(); const dec = new TextDecoder(); - const pair = createMockTransportPair(Lite.ALPN_05); + const pair = createMockTransportPair(Lite.ALPN_07_WIP); const origin = new OriginProducer(); const [client, server] = await Promise.all([ @@ -1160,10 +1160,10 @@ test("integration: lite draft-05 coalesces concurrent fetches of one group", asy server.abort(); }); -test("integration: lite draft-05 fetches an in-progress group", async () => { +test("integration: lite draft-07 fetches an in-progress group", async () => { const enc = new TextEncoder(); const dec = new TextDecoder(); - const pair = createMockTransportPair(Lite.ALPN_05); + const pair = createMockTransportPair(Lite.ALPN_07_WIP); const origin = new OriginProducer(); const [client, server] = await Promise.all([ @@ -1202,8 +1202,8 @@ test("integration: lite draft-05 fetches an in-progress group", async () => { // The publisher caches TRACK_INFO so it only asks the application once per track. The cache // has to expire with the broadcast that answered it: a republish puts a different producer // on the path, and its immutable properties are its own. -test("integration: lite draft-05 track info follows a republished broadcast", async () => { - const pair = createMockTransportPair(Lite.ALPN_05); +test("integration: lite draft-07 track info follows a republished broadcast", async () => { + const pair = createMockTransportPair(Lite.ALPN_07_WIP); const origin = new OriginProducer(); const [client, server] = await Promise.all([ @@ -1240,9 +1240,9 @@ test("integration: lite draft-05 track info follows a republished broadcast", as // FETCH reads the same cache to decide the timescale it serves frames in, so a stale entry // quantizes the successor's timestamps to the predecessor's resolution. -test("integration: lite draft-05 fetch uses the republished track's timescale", async () => { +test("integration: lite draft-07 fetch uses the republished track's timescale", async () => { const enc = new TextEncoder(); - const pair = createMockTransportPair(Lite.ALPN_05); + const pair = createMockTransportPair(Lite.ALPN_07_WIP); const origin = new OriginProducer(); const [client, server] = await Promise.all([ diff --git a/js/net/src/lite/group.ts b/js/net/src/lite/group.ts index b7e57567db1..90b4bf4ecbe 100644 --- a/js/net/src/lite/group.ts +++ b/js/net/src/lite/group.ts @@ -76,7 +76,7 @@ function unzigzag(v: bigint): bigint { * FRAME format), decoded into a Timestamp at that scale. Scale 0 (pre-lite-05) carries no * timestamp, so frames arrive untimed. */ -export function frameDecoder(scale: number): (c: Cursor) => netGroup.Frame { +export function frameDecoder(scale: number, timed = true): (c: Cursor) => netGroup.Frame { if (scale === 0) { return (c) => ({ payload: c.read(c.u53()) }); } @@ -88,13 +88,23 @@ export function frameDecoder(scale: number): (c: Cursor) => netGroup.Frame { const payload = c.read(c.u53()); // After the last read, so a decode that ran short and gets retried adds the delta once. prevTs += delta; + // Read to stay in step, but dropped where it may be a send time. + if (!timed) return { payload }; return { payload, timestamp: new Time.Timestamp(Number(prevTs), timescale) }; }; } -/** Write a group stream's frames into `producer` until the stream ends or the producer closes. */ -export async function readFrames(stream: Reader, producer: netGroup.Producer, scale: number): Promise { - const decode = frameDecoder(scale); +/** + * Write a group stream's frames into `producer` until the stream ends or the producer closes. + * Each frame's timestamp is read at `scale` and kept only when `timed`. + */ +export async function readFrames( + stream: Reader, + producer: netGroup.Producer, + scale: number, + timed = true, +): Promise { + const decode = frameDecoder(scale, timed); for (;;) { // Every frame already buffered is written without an await, so the reader wakes once diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index 96f55e43a7d..f9566fff651 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -17,6 +17,7 @@ import { Priority, sendOrder } from "../priority.ts"; import { type Reader, type Stream, Writer } from "../stream.ts"; import { type Timescale, Timestamp } from "../time.ts"; import type * as track from "../track.ts"; +import { liteTimescale } from "../track.ts"; import { untilAborted } from "../util/abort.ts"; import { Interests, LIMITS, type Limits, Slots } from "../util/limits.ts"; import { nextTask } from "../util/yield.ts"; @@ -867,7 +868,7 @@ export class Publisher { // (accept never called, track closed) as an error here, which resets the // stream. const info = await untilAborted(track.info(), left.signal); - timescale = wireTimescale(this.version, info.timescale); + timescale = wireTimescale(this.version, liteTimescale(info)); } else { // Older drafts acknowledge with SUBSCRIBE_OK and stream frames verbatim. const ok = new SubscribeOk({ @@ -1440,9 +1441,9 @@ export class Publisher { // Publisher Max Age: the publisher's retention bound, advertised so // relays re-serve with the same window. maxAge: info.maxAge, - // `#serveGroup` emits each frame converted to it. - timescale: info.timescale ?? undefined, - unverified: info.unverified, + // `#serveGroup` emits each frame converted to it. Unchecked timestamps never + // enter moq-lite, whose receivers check them. + timescale: liteTimescale(info), }); })(); diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index 794e545f124..fc8efae24e2 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -956,14 +956,13 @@ async function answerTrackInfo( stream: FakeStream, version: Version = Version.DRAFT_05, timescale?: number, - unverified = false, ): Promise { const chunks: Uint8Array[] = []; const writer = new Writer( new WritableStream({ write: (chunk) => void chunks.push(new Uint8Array(chunk)) }), version, ); - await new TrackInfo({ timescale, unverified }).encode(writer, version); + await new TrackInfo({ timescale }).encode(writer, version); for (const chunk of chunks) stream.inbound.enqueue(chunk); stream.inbound.close(); } @@ -1728,14 +1727,10 @@ async function groupStream(timestamp: number, version: Version = Version.DRAFT_0 ); } -// A received group that rewinds or lands out of order is accepted wherever its timestamps are -// unverified: always on lite-05/06, which stamp an untimed track's frames with their send time, and -// on lite-07 when the publisher says so. -test.each([ - [Version.DRAFT_05, false], - [Version.DRAFT_06, false], - [Version.DRAFT_07, true], -])("version %p accepts an unverified rewind", async (version, unverified) => { +// Lite-05/06 stamp an untimed track's frames with their send time, which a receiver cannot tell +// from media time, so it drops every timestamp there: a group that rewinds or lands out of order is +// accepted untimed. +test.each([Version.DRAFT_05, Version.DRAFT_06])("version %p drops received timestamps", async (version) => { const { quic, streams, opened } = fakeSession(); const subscriber = new Subscriber(quic, version, HopSchema.parse(1n)); const broadcast = subscriber.consume(Path.from("room")); @@ -1743,7 +1738,7 @@ test.each([ const track = broadcast.track("video").subscribe({ maxDelay: Milli(60_000) }); try { await opened(0); - await answerTrackInfo(streams[0], version, 1000, unverified); + await answerTrackInfo(streams[0], version, 1000); await (await opened(1)).reading; await subscriber.runGroup(new Group({ subscribe: 0n, sequence: 4 }), await groupStream(1000, version)); // Starts before the group below it. @@ -1753,13 +1748,15 @@ test.each([ const got = []; for (let i = 0; i < 3; i++) { const group = await track.recvGroup(); - got.push([group?.sequence, (await group?.readFrame())?.payload.length]); + const frame = await group?.readFrame(); + got.push([group?.sequence, frame?.payload.length, frame?.timestamp]); } expect(got).toEqual([ - [3, 1], - [4, 1], - [5, 1], + [3, 1, undefined], + [4, 1, undefined], + [5, 1, undefined], ]); + expect((await track.info()).timescale).toBeUndefined(); expect(track.closed.peek()).toBeUndefined(); } finally { track.close(); @@ -1768,8 +1765,8 @@ test.each([ } }); -// A lite-07 track whose publisher vouches for its timestamps is checked like a local write. -test("a verified rewind is refused", async () => { +// Lite-07 timestamps are media time, so a received group is checked like a local write. +test("a lite-07 rewind is refused", async () => { const version = Version.DRAFT_07; const { quic, streams, opened } = fakeSession(); const subscriber = new Subscriber(quic, version, HopSchema.parse(1n)); diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 856aef3562f..e764671c057 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -55,6 +55,7 @@ import { hasAnnounceOk, hasDatagrams, hasFetchMaxDelay, + hasMediaTime, hasOrder, hasProbeRtt, hasRefresh, @@ -959,13 +960,13 @@ export class Subscriber { // Map the wire TRACK_INFO onto the model track.Info a producer/consumer holds. #toModelInfo(info: TrackInfo): track.Info { return { - timescale: info.timescale, + // Lite-05/06 timestamps may be send times, so the copy is untimed and the ingest drops them. + timescale: hasMediaTime(this.version) ? info.timescale : undefined, // Publisher Max Age rides on the wire, so the local media-time budget // matches what the upstream advertises (relays re-serve with the same bound). maxAge: info.maxAge === undefined ? undefined : Milli(info.maxAge), priority: info.priority, refresh: info.refresh, - unverified: info.unverified, }; } @@ -1159,7 +1160,7 @@ export class Subscriber { // the gap. async #runFetchResponse(stream: Stream, group: netGroup.Producer, timescale: number): Promise { try { - const decode = frameDecoder(timescale); + const decode = frameDecoder(timescale, hasMediaTime(this.version)); // Serve until the stream FINs, the group closes, or every reader leaves. A group can // stay open indefinitely (a catalog or JSON stream), so an abandoned fetch is stopped by @@ -1449,7 +1450,7 @@ export class Subscriber { scale = timescale.peek(); } - await readFrames(stream, producer, scale); + await readFrames(stream, producer, scale, hasMediaTime(this.version)); producer.close(); stream.stop(StreamCode.Cancel); @@ -1525,8 +1526,11 @@ export class Subscriber { const scale = entry.timescale.peek(); if (scale === undefined) return; + // Dropped where it may be a send time, like a frame's. const timestamp = - dg.timestamp === undefined ? undefined : new Time.Timestamp(dg.timestamp, Time.Timescale(scale)); + dg.timestamp === undefined || !hasMediaTime(this.version) + ? undefined + : new Time.Timestamp(dg.timestamp, Time.Timescale(scale)); // A datagram's sequence is never owed a stream, so it never holds the tail open. entry.tail.account(dg.sequence, dg.sequence + 1); entry.track.insertDatagram(dg.sequence, timestamp, dg.payload); diff --git a/js/net/src/lite/track.test.ts b/js/net/src/lite/track.test.ts index ebcbeb3bb7c..e0c776bf829 100644 --- a/js/net/src/lite/track.test.ts +++ b/js/net/src/lite/track.test.ts @@ -212,35 +212,3 @@ test("refresh ratio survives lite-07 and disappears on legacy wires", async () = } } }); - -test("unverified survives lite-07, and every timed track arrives unverified before it", async () => { - for (const version of [Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07]) { - for (const unverified of [false, true]) { - const info = new TrackInfo({ priority: 7, timescale: 1000, unverified }); - const encoded = await bytes((w) => info.encode(w, version), version); - const decoded = await TrackInfo.decode(new Reader(undefined, encoded, version), version); - expect(decoded.unverified).toBe(version === Version.DRAFT_07 ? unverified : true); - } - } -}); - -test("an unverified timed TrackInfo matches cross-language wire bytes on lite-07", async () => { - const info = new TrackInfo({ priority: 1, timescale: 1000, unverified: true }); - expect(await bytes((w) => info.encode(w, Version.DRAFT_07), Version.DRAFT_07)).toEqual( - new Uint8Array([0x06, 0x01, 0x00, 0x83, 0xe8, 0x00, 0x01]), - ); -}); - -test("an untimed track can't be unverified", async () => { - expect(() => infoDefaults({ unverified: true })).toThrow(RangeError); - const info = new TrackInfo({ priority: 1, unverified: true }); - await expect(bytes((w) => info.encode(w, Version.DRAFT_07), Version.DRAFT_07)).rejects.toThrow(RangeError); - - const encoded = await bytes((w) => new TrackInfo({ priority: 1 }).encode(w, Version.DRAFT_07), Version.DRAFT_07); - for (const flag of [1, 2]) { - encoded[encoded.length - 1] = flag; - await expect( - TrackInfo.decode(new Reader(undefined, encoded, Version.DRAFT_07), Version.DRAFT_07), - ).rejects.toThrow(RangeError); - } -}); diff --git a/js/net/src/lite/track.ts b/js/net/src/lite/track.ts index 71d016a0522..e68210b4565 100644 --- a/js/net/src/lite/track.ts +++ b/js/net/src/lite/track.ts @@ -4,7 +4,7 @@ import type { Reader, Writer } from "../stream.ts"; import { Timescale } from "../time.ts"; import { decodeEpoch, encodeEpoch } from "./epoch.ts"; import * as Message from "./message.ts"; -import { hasGroupOrder, hasRefresh, hasTrackStream, hasUnverified, Version, wireTimescale } from "./version.ts"; +import { hasGroupOrder, hasRefresh, hasTrackStream, Version, wireTimescale } from "./version.ts"; function guardTrack(version: Version) { if (!hasTrackStream(version)) throw new Error("track stream not supported for this version"); @@ -71,24 +71,17 @@ export class TrackInfo { * out as milliseconds (see {@link wireTimescale}). */ timescale?: Timescale; - /** - * Whether a timed track's timestamps may rewind, so receivers don't check them. Carried on - * lite-07; every timed track received over lite-05/06 is unverified. - */ - unverified: boolean; constructor({ priority = 0, maxAge, timescale, refresh, - unverified = false, }: { priority?: number; maxAge?: number; timescale?: number; refresh?: number; - unverified?: boolean; }) { if (!Number.isInteger(priority) || priority < 0 || priority > 255) { throw new RangeError(`priority must be an integer in 0..=255: ${priority}`); @@ -102,7 +95,6 @@ export class TrackInfo { this.priority = priority; this.maxAge = maxAge; this.timescale = timescale === undefined ? undefined : Timescale(timescale); - this.unverified = unverified; } async #encode(w: Writer, version: Version) { @@ -114,7 +106,6 @@ export class TrackInfo { await w.u62(legacy ? (age ?? BigInt(Number.MAX_SAFE_INTEGER)) : age === undefined ? 0n : age + 1n); await w.u53(wireTimescale(version, this.timescale) ?? 0); if (hasRefresh(version)) await w.u53(this.refresh === undefined ? 0 : this.refresh + 1); - if (hasUnverified(version)) await w.bool(this.unverified); } static async #decode(r: Reader, version: Version): Promise { @@ -136,25 +127,7 @@ export class TrackInfo { const scale = await r.u53(); const timescale = scale === 0 && !legacy ? undefined : Timescale(scale); const refresh = hasRefresh(version) ? (await r.u53()) - 1 : undefined; - let unverified: boolean; - if (hasUnverified(version)) { - const flag = await r.u8(); - // Only a timed track has timestamps to doubt. - if (flag > 1 || (flag === 1 && timescale === undefined)) - throw new RangeError(`invalid unverified flag: ${flag}`); - unverified = flag === 1; - } else { - // Lite-05/06 always carry a timescale, and an untimed track's send times can't be told - // from media time. - unverified = timescale !== undefined; - } - return new TrackInfo({ - priority, - maxAge, - timescale, - refresh: refresh === -1 ? undefined : refresh, - unverified, - }); + return new TrackInfo({ priority, maxAge, timescale, refresh: refresh === -1 ? undefined : refresh }); } async encode(w: Writer, version: Version): Promise { @@ -173,8 +146,6 @@ export class TrackInfo { ) throw new RangeError("refresh exceeds u32"); if (this.timescale !== undefined) Timescale(this.timescale); - if (this.unverified && this.timescale === undefined) - throw new RangeError("an untimed track can't be unverified"); return Message.encode(w, (w) => this.#encode(w, version)); } diff --git a/js/net/src/lite/track_stream.test.ts b/js/net/src/lite/track_stream.test.ts index 3b8f9ca6047..77579216486 100644 --- a/js/net/src/lite/track_stream.test.ts +++ b/js/net/src/lite/track_stream.test.ts @@ -382,4 +382,36 @@ describe.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])("%s", (ver broadcast.close(); origin.close(); }); + + // Timestamps received over moq-transport may be arrival times, so they never enter moq-lite, + // whose receivers check every timestamp: the track is served untimed (milliseconds of send + // time before lite-07). + test("an unchecked track is served untimed", async () => { + const pair = createMockTransportPair(ALPN_05); + const origin = new OriginProducer(); + const publisher = new Publisher(pair.server, version, randomHop(), origin.consume()); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + const serving = (async () => { + for (;;) { + const request = await wireOf(broadcast).requested(); + if (!request) return; + request.accept({ timescale: Timescale.MICRO, unchecked: true }); + } + })(); + + const client = await Stream.open(pair.client, { version }); + const server = await Stream.accept(pair.server, version); + if (!server) throw new Error("the publisher never accepted the TRACK stream"); + void publisher.runTrackInfo(new TrackMessage(Path.from("room"), "video"), server); + + const info = await TrackInfo.decode(client.reader, version); + expect(info.timescale).toBe(version === Version.DRAFT_07 ? undefined : Timescale.MILLI); + + client.writer.close(); + publisher.close(); + broadcast.close(); + await serving; + origin.close(); + }); }); diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index 63b61139737..cfd91ecc29c 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -404,9 +404,10 @@ export function hasRefresh(version: Version): boolean { } } -/** Whether TRACK_INFO says if a timed track's timestamps are unverified. Added in lite-07; earlier, - * a receiver cannot tell an untimed track's send times from media time, so every timed track is. */ -export function hasUnverified(version: Version): boolean { +/** Whether received frame timestamps are the publisher's media time. Lite-05/06 stamp an untimed + * track's payloads with their send time, which a receiver cannot tell from it, so it drops every + * timestamp received on them and the track arrives untimed. */ +export function hasMediaTime(version: Version): boolean { // Explicitly list older versions so future versions keep the lite-07+ behavior. switch (version) { case Version.DRAFT_01: diff --git a/js/net/src/track.ts b/js/net/src/track.ts index d6bb86b117d..c95b580059f 100644 --- a/js/net/src/track.ts +++ b/js/net/src/track.ts @@ -106,15 +106,15 @@ export interface Info { * snapshot or keyframe. `0` asks on any join after a delta. An integer in `0..2^32`. */ refresh?: number; - // pending: /quest/m1/public-model/ingest.md /** - * Whether a hop stamped this timed track's timestamps, or received them over a wire that - * cannot say it didn't, so they may rewind and no group is checked for it. A local - * publisher's writes are always checked. Mirrors the Rust `track::Info::unverified`. + * Whether this timed track's timestamps arrived over moq-transport, whose peers may stamp + * arrival times, so no group is checked for a rewind. A moq-lite hop serves the track + * untimed, since its receivers check every timestamp. Mirrors the Rust + * `track::Info::unchecked`. * * @internal */ - unverified?: boolean; + unchecked?: boolean; } // Normalize a duration for the wire, which carries it as an unsigned varint. @@ -141,17 +141,23 @@ function priorityByte(value: number): number { return value; } +// private: passthrough +/** The timescale a moq-lite hop serves a track at: none for an {@link Info.unchecked} one. @internal */ +export function liteTimescale(info: Info): Timescale | undefined { + return info.unchecked ? undefined : (info.timescale ?? undefined); +} + /** Fill in any unset {@link Info} fields with their defaults. */ export function infoDefaults(info: Partial = {}): Info { const timescale = info.timescale == null ? undefined : Timescale(info.timescale); // Only a timed track has timestamps to doubt. - if (info.unverified && timescale === undefined) throw new RangeError("an untimed track can't be unverified"); + if (info.unchecked && timescale === undefined) throw new RangeError("an untimed track can't be unchecked"); return { timescale, maxAge: info.maxAge === undefined ? undefined : wireMillis("maxAge", info.maxAge), priority: priorityByte(info.priority ?? DEFAULT_PRIORITY), refresh: info.refresh == null ? undefined : refreshRatio(info.refresh), - ...(info.unverified ? { unverified: true } : {}), + ...(info.unchecked ? { unchecked: true } : {}), }; } @@ -1206,7 +1212,7 @@ export class Producer { this.#drop(group.sequence, group.sequence + 1); }); const sequence = group.sequence; - if (this.#unverified()) { + if (this.#unchecked()) { // Its timestamps may rewind, so they are not checked. } else if (start) { this.#record(entry, start, late); @@ -1340,15 +1346,15 @@ export class Producer { } // A group written before it joined is checked here, before it is cached or fanned out. - const start = this.#unverified() ? undefined : hooks.groupStart(group); + const start = this.#unchecked() ? undefined : hooks.groupStart(group); const late = start !== undefined && this.#admit(group.sequence, start); this.#publish(group, start, late); } - // Whether this track's timestamps may rewind (see {@link Info.unverified}). - #unverified(): boolean { - return this.#state.info.peek()?.unverified === true; + // Whether this track's timestamps may rewind (see {@link Info.unchecked}). + #unchecked(): boolean { + return this.#state.info.peek()?.unchecked === true; } // Fan a datagram out to every live subscriber, dropping the oldest once the ring is full. diff --git a/js/net/src/untimed.test.ts b/js/net/src/untimed.test.ts index e839f281006..38c2f2a1835 100644 --- a/js/net/src/untimed.test.ts +++ b/js/net/src/untimed.test.ts @@ -18,23 +18,21 @@ interface Wire { name: string; protocol: string; version?: number; - /** Whether the wire can declare a track's timeline. */ + /** Whether a timed track arrives timed. Lite-05/06 receivers drop timestamps, which may be send times. */ declares: boolean; - /** Whether the wire carries a timestamp on every frame, a send time for an untimed one. */ - stamps: boolean; } const WIRES: Wire[] = [ - { name: "lite-01", protocol: "", version: Lite.Version.DRAFT_01, declares: false, stamps: false }, - { name: "lite-03", protocol: Lite.ALPN_03, declares: false, stamps: false }, - { name: "lite-04", protocol: Lite.ALPN_04, declares: false, stamps: false }, - { name: "lite-05", protocol: Lite.ALPN_05, declares: true, stamps: true }, - { name: "lite-06", protocol: Lite.ALPN_06, declares: true, stamps: true }, - { name: "lite-07-wip", protocol: Lite.ALPN_07_WIP, declares: true, stamps: false }, - { name: "ietf-14", protocol: "", version: Ietf.Version.DRAFT_14, declares: false, stamps: false }, - { name: "ietf-16", protocol: Ietf.ALPN.DRAFT_16, declares: false, stamps: false }, - { name: "ietf-17", protocol: Ietf.ALPN.DRAFT_17, declares: true, stamps: false }, - { name: "ietf-20", protocol: Ietf.ALPN.DRAFT_20, declares: true, stamps: false }, + { name: "lite-01", protocol: "", version: Lite.Version.DRAFT_01, declares: false }, + { name: "lite-03", protocol: Lite.ALPN_03, declares: false }, + { name: "lite-04", protocol: Lite.ALPN_04, declares: false }, + { name: "lite-05", protocol: Lite.ALPN_05, declares: false }, + { name: "lite-06", protocol: Lite.ALPN_06, declares: false }, + { name: "lite-07-wip", protocol: Lite.ALPN_07_WIP, declares: true }, + { name: "ietf-14", protocol: "", version: Ietf.Version.DRAFT_14, declares: false }, + { name: "ietf-16", protocol: Ietf.ALPN.DRAFT_16, declares: false }, + { name: "ietf-17", protocol: Ietf.ALPN.DRAFT_17, declares: true }, + { name: "ietf-20", protocol: Ietf.ALPN.DRAFT_20, declares: true }, ]; /** Serve one group of `timestamps` on a track with `timescale`, and read it back over `wire`. */ @@ -156,21 +154,14 @@ test("a forwarded untimed track stays untimed", async () => { }); for (const wire of WIRES) { - test(`${wire.name}: an untimed track arrives untimed, or with send times where the wire must stamp`, async () => { + test(`${wire.name}: an untimed track arrives untimed`, async () => { const { info, frames } = await roundTrip(wire, undefined, [undefined, undefined]); expect(frames.map((frame) => frame.payload[0])).toEqual([0, 1]); - - if (wire.stamps) { - // Lite-05 and lite-06 can't say a track is untimed, so it goes out timed at milliseconds. - expect(info.timescale).toBe(Timescale.MILLI); - for (const frame of frames) expect(frame.timestamp).toBeDefined(); - } else { - expect(info.timescale).toBeUndefined(); - for (const frame of frames) expect(frame.timestamp).toBeUndefined(); - } + expect(info.timescale).toBeUndefined(); + for (const frame of frames) expect(frame.timestamp).toBeUndefined(); }); - test(`${wire.name}: a timed track keeps its timestamps only where the wire declares units`, async () => { + test(`${wire.name}: a timed track keeps its timestamps only where the wire delivers them`, async () => { const { info, frames } = await roundTrip(wire, Timescale.MICRO, [ Timestamp.fromMicros(1_000), Timestamp.fromMicros(1_234), @@ -181,7 +172,7 @@ for (const wire of WIRES) { expect(info.timescale).toBe(Timescale.MICRO); expect(frames.map((frame) => frame.timestamp?.asMicros())).toEqual([1_000, 1_234]); } else { - // No units on the wire, and no arrival time made up in their place. + // No trusted units on the wire, and no arrival time made up in their place. expect(info.timescale).toBeUndefined(); for (const frame of frames) expect(frame.timestamp).toBeUndefined(); } @@ -189,7 +180,7 @@ for (const wire of WIRES) { } for (const wire of WIRES.filter((wire) => wire.name === "lite-06" || wire.name === "lite-07-wip")) { - test(`${wire.name}: an untimed datagram arrives untimed, or with a send time where the wire must stamp`, async () => { + test(`${wire.name}: an untimed datagram arrives untimed`, async () => { const pair = createMockTransportPair(wire.protocol); const origin = new OriginProducer(); const [client, server] = await Promise.all([ @@ -212,8 +203,7 @@ for (const wire of WIRES.filter((wire) => wire.name === "lite-06" || wire.name = const datagram = await subscriber.recvDatagram(); expect(datagram?.payload).toEqual(new Uint8Array([7])); - if (wire.stamps) expect(datagram?.timestamp).toBeDefined(); - else expect(datagram?.timestamp).toBeUndefined(); + expect(datagram?.timestamp).toBeUndefined(); broadcast.close(); subscriber.close(); diff --git a/quest/m1/public-model/ingest.md b/quest/m1/public-model/ingest.md index 8057f2b57af..2f7305125d3 100644 --- a/quest/m1/public-model/ingest.md +++ b/quest/m1/public-model/ingest.md @@ -170,12 +170,11 @@ Rust items (`rs/moq-net/src/model/`), as they stood at b4eab0db1: `track::TrackState`, `track::FetchOutcome`, `finish_track_at`, `poll_landed`): moved to [the deferred quest](/quest/m1/public-model/deferred.md), which waits on Model ranges. -- `track::Info::unverified` (JS `Info.unverified`, `@internal`) marks a - track whose timestamps may rewind, so no group is checked. Sessions set - it on ingress over moq-lite-05/06 and moq-transport, and lite-07 carries - it in TRACK_INFO. A custom session's copies stay verified, so they are - checked; expose it with the other ingest calls only if one must ingest - send or arrival times. +- `track::Info::unchecked` (JS `Info.unchecked`, `@internal`) marks a + track whose timestamps arrived over moq-transport, so no group is checked + and moq-lite serves it untimed. A custom session's copies stay checked; + expose it with the other ingest calls only if one must ingest arrival + times. - Passthrough, stay private and marked: `group::Producer::wire`, `set_wire`, `mark_gap`, `set_end_object`, `group::Wire`, the `object` and `datagram` modules, `track::Producer::insert_wire_datagram`, diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index 6a2928d635b..920a50b51e3 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -103,7 +103,9 @@ let rate = bandwidth.as_ref().and_then(|b| b.peek()); ### A timed track refuses a group that rewinds (#5803) A local write whose first frame starts before a lower-sequence group's, or after a higher one's, fails with `TimestampMismatch`; `@moq/net` throws it the same way. -A received group is checked too when every hop vouched for it over moq-lite-07. One received over moq-lite-05/06 or moq-transport is accepted as is, since its timestamps may be send or arrival times. +A group received over moq-lite-07 is checked the same way. +A track received over moq-lite-05/06 arrives untimed, since its timestamps may be send times: read times from the payload, such as the hang container's, instead of `frame::Frame::timestamp`. +One received over moq-transport keeps its timestamps unchecked, and a relay serves it to moq-lite untimed. A publisher whose timeline restarts, such as an HLS `EXT-X-DISCONTINUITY` that resets decode times, publishes a new broadcast instead. ### `broadcast::Producer::finish` is back, and lingers (#5795) diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index a5e884970dd..29abae75ae5 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -645,7 +645,6 @@ impl Default for Messages { priority: 1, max_age: Some(Duration::from_secs(10)), timescale: Some(crate::Timescale::MICRO), - unverified: false, }, lite_group: lite::Group { subscribe: 7, diff --git a/rs/moq-net/src/ietf/subscriber.rs b/rs/moq-net/src/ietf/subscriber.rs index 1f00cd93ea7..d3d49c23264 100644 --- a/rs/moq-net/src/ietf/subscriber.rs +++ b/rs/moq-net/src/ietf/subscriber.rs @@ -478,7 +478,7 @@ fn accepted_info(properties: &ietf::Properties) -> track::Info { info.properties = Some(std::sync::Arc::new(properties.unknown.clone())).filter(|unknown| !unknown.values.is_empty()); // A moq-transport peer may stamp arrival times, and nothing on the wire says it didn't. - info.unverified = info.timescale.is_some(); + info.unchecked = info.timescale.is_some(); info } @@ -12175,30 +12175,19 @@ mod stitch_tests { settled.await; } - /// Object numbering is relay metadata; plain readers still see consecutive frames. - async fn relay_passthrough( - script: Vec, - ingress: Version, - egress: Version, - fetch: Option<(u64, u64)>, - ) -> (Vec, crate::lite::test_transport::Log) { - relay_passthrough_timed(script, ingress, egress, fetch, None).await - } - - /// Relay `script` through a track the ingress times in `timescale`, which the cache - /// normalizes to microseconds like [`accepted_info`]. + /// Relay `script` through an untimed track. /// /// Also relays it through a second relay behind a moq-lite hop with the `ietf` - /// extension, which must not change a byte. - async fn relay_passthrough_timed( + /// extension, which must not change a byte. Object numbering is relay metadata; plain + /// readers still see consecutive frames. + async fn relay_passthrough( script: Vec, ingress: Version, egress: Version, fetch: Option<(u64, u64)>, - timescale: Option, ) -> (Vec, crate::lite::test_transport::Log) { - let direct = relay_passthrough_via(script.clone(), ingress, egress, fetch, timescale, false).await; - let hopped = relay_passthrough_via(script, ingress, egress, fetch, timescale, true).await; + let direct = relay_passthrough_via(script.clone(), ingress, egress, fetch, None, false).await; + let hopped = relay_passthrough_via(script, ingress, egress, fetch, None, true).await; let fetch_ok = |bytes: &[u8]| { let mut decoder = Decoder::new(bytes, egress.into()); (decoder.varint().ok()? == ietf::FetchOk::ID).then_some(())?; @@ -12240,7 +12229,9 @@ mod stitch_tests { direct } - /// [`relay_passthrough_timed`] through one relay, or two joined by a moq-lite `hop`. + /// Relay `script` through a track the ingress times in `timescale`, which the cache + /// normalizes to microseconds like [`accepted_info`], via one relay or two joined by a + /// moq-lite `hop`. async fn relay_passthrough_via( script: Vec, ingress: Version, @@ -12424,22 +12415,26 @@ mod stitch_tests { /// A relayed Timestamp leaves in the units the publisher declared: the cache holds the /// track on the publisher's grid, so a copy of the same instance over another route - /// matches it. + /// matches it. A moq-lite hop serves the unchecked track untimed, so the Timestamp + /// stops there and the other property survives. #[moq_clock::test] async fn passthrough_keeps_the_publishers_timestamp_units() { let version = Version::Draft22; // Timestamp 42 (ms), then an unknown property that must survive beside it. let properties = [0x10, 42, 0x70, 7]; let script = object_stream(version, true, &[(0, &properties, None)]); - let (bytes, log) = relay_passthrough_timed(script, version, version, None, Some(Timescale::MILLI)).await; - assert!(log.resets().is_empty()); - let mut decoder = Decoder::new(&bytes, version.into()); - ietf::GroupHeader::decode(&mut decoder, version).unwrap(); - assert_eq!(decoder.varint().unwrap(), 0); - let relayed = bytes::Bytes::copy_from_slice(decoder.bytes().unwrap()); - let decoded = super::super::object::read(&relayed, version, Some(Timescale::MILLI)).unwrap(); - assert_eq!(decoded.timestamp, Some(Timestamp::new(42, Timescale::MILLI).unwrap())); - assert_eq!(super::super::object::listed(&decoded.properties).len(), 1); + for (hop, timestamp) in [(false, Timestamp::new(42, Timescale::MILLI).ok()), (true, None)] { + let (bytes, log) = + relay_passthrough_via(script.clone(), version, version, None, Some(Timescale::MILLI), hop).await; + assert!(log.resets().is_empty(), "hop: {hop}"); + let mut decoder = Decoder::new(&bytes, version.into()); + ietf::GroupHeader::decode(&mut decoder, version).unwrap(); + assert_eq!(decoder.varint().unwrap(), 0); + let relayed = bytes::Bytes::copy_from_slice(decoder.bytes().unwrap()); + let decoded = super::super::object::read(&relayed, version, timestamp.map(|_| Timescale::MILLI)).unwrap(); + assert_eq!(decoded.timestamp, timestamp, "hop: {hop}"); + assert_eq!(super::super::object::listed(&decoded.properties).len(), 1, "hop: {hop}"); + } } fn object_stream(version: Version, first: bool, objects: &[(u64, &[u8], Option)]) -> Vec { @@ -14041,7 +14036,7 @@ mod stitch_tests { } /// A received group whose first object rewinds past the group before it is accepted: a - /// moq-transport peer may stamp arrival times, so its timed tracks are unverified. + /// moq-transport peer may stamp arrival times, so its timed tracks are unchecked. #[moq_clock::test] async fn a_received_rewind_is_accepted() { use futures::FutureExt; diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 32be630bc12..2ff30079d31 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -2127,7 +2127,7 @@ impl Request for SubscribeServe { // `None` omits the per-frame timestamp prefix: an untimed track on // Lite07+, or any track before Lite05 (whose frames still carry // timestamps in the model, just not on this wire). - let timescale = shared.version.wire_timescale(track.info().timescale); + let timescale = shared.version.wire_timescale(track.info().lite_timescale()); // Lite05+ accepts implicitly: no SUBSCRIBE_OK, the immutable properties // live in TRACK_INFO, and the resolved range arrives as @@ -2301,7 +2301,7 @@ impl Request for FetchServe { group.end_at(msg.end_frame.map_or(Bound::Unbounded, Bound::Included)); // Matches the timescale TRACK_INFO declared for the track. - let timescale = shared.version.wire_timescale(group.timescale()); + let timescale = shared.version.wire_timescale(group.lite_timescale()); let capped = msg.end_frame.is_some(); let records = lite::ietf::Records::start(writer, ietf, &group, capped, timescale, shared.version)?; @@ -3354,20 +3354,18 @@ fn buffer_frame_info( /// A frame or datagram timestamp as its raw value at the wire `timescale`. /// /// Lite05/06 cannot encode an untimed track, so its payloads carry their send time on -/// `runtime` instead, at the millisecond scale its TRACK_INFO declares. A timed payload -/// is already at the track's timescale. +/// `runtime` instead, at the millisecond scale its TRACK_INFO declares. An unchecked track +/// declares milliseconds there too, so its timestamps are converted to it. fn wire_timestamp( timestamp: Option, timescale: crate::Timescale, runtime: &crate::time::Deadlines, ) -> Result { - match timestamp { - Some(timestamp) => Ok(timestamp.value()), - None => crate::Timestamp::from(runtime.now()) - .convert(timescale) - .map(|now| now.value()) - .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded)), - } + timestamp + .unwrap_or_else(|| crate::Timestamp::from(runtime.now())) + .convert(timescale) + .map(|time| time.value()) + .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded)) } /// Buffer `curr` as a zigzag-mapped varint delta against `*prev`, then advance diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 76a50a59c9a..2807a80fd4c 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -524,7 +524,9 @@ impl Subscriber { .zip(entry.timescale) .map(|(value, scale)| Timestamp::new(value, scale)) .transpose() - .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded))?; + .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded))? + // Dropped where it may be a send time, like a frame's. + .filter(|_| self.version.has_media_time()); // A datagram is never owed a stream, so it never holds the subscription's end open. if let Ok(mut tail) = entry.tail.write() { @@ -1202,8 +1204,9 @@ impl FrameIngest { self.prev_ts = next; let timestamp = Timestamp::new(next, scale) .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded))?; + // Read to stay in step, but dropped where it may be a send time. self.phase = IngestPhase::Size { - timestamp: Some(timestamp), + timestamp: self.version.has_media_time().then_some(timestamp), }; } IngestPhase::Size { timestamp } => { @@ -2563,10 +2566,11 @@ mod tests { going_away: Default::default(), auth: crate::auth::Handle::new(false), }); + // The track is untimed where the subscriber drops received timestamps. let track = track::Producer::new( std::sync::Arc::new(crate::broadcast::Info::default()), "video", - crate::track::Info::timed(), + crate::track::Info::default().with_timescale(version.has_media_time().then_some(Timescale::MILLI)), ); let mut consumer = track.consume().subscribe(None).await.unwrap(); subscriber.subscribes.lock().insert( @@ -2822,10 +2826,9 @@ mod tests { ); } - /// Receive group 5 starting at 500ms over `version`, on a received timed track whose - /// group 4 starts at 1000ms and whose publisher's TRACK_INFO said `unverified`, returning - /// the stream's result and the track. - async fn receive_rewind(version: Version, unverified: bool) -> (Result<(), Error>, track::Producer) { + /// Receive group 5 stamped 500ms over `version` into the copy a subscriber makes there, + /// whose group 4 started at 1000ms, returning the stream's result and the track. + async fn receive_rewind(version: Version) -> (Result<(), Error>, track::Producer) { use crate::transport::poll::Session as _; let mut script = Vec::new(); @@ -2854,21 +2857,16 @@ mod tests { auth: crate::auth::Handle::new(false), }); - let sent = lite::TrackInfo { - unverified, - ..lite::TrackInfo::from(&crate::track::Info::timed()) - } - .encode_bytes(version) - .unwrap(); - let info = lite::TrackInfo::decode_slice(&sent, version).unwrap().0; - let track = track::Producer::new( - std::sync::Arc::new(crate::broadcast::Info::default()), - "video", - track::Info::from(info), - ); + // The copy is untimed where the wire's timestamps may be send times. + let timed = version.has_media_time(); + let info = match timed { + true => crate::track::Info::timed(), + false => crate::track::Info::default(), + }; + let track = track::Producer::new(std::sync::Arc::new(crate::broadcast::Info::default()), "video", info); let mut before = track.create_group(4u64.into()).unwrap(); before - .write_frame(Timestamp::from_millis(1000).unwrap(), &b"before"[..]) + .write_frame(timed.then(|| Timestamp::from_millis(1000).unwrap()), &b"before"[..]) .unwrap(); before.finish().unwrap(); subscriber.subscribes.lock().insert( @@ -2886,27 +2884,28 @@ mod tests { (res, track) } - /// A received group that rewinds is accepted, and the track stays open, wherever its - /// timestamps are unverified: always on lite-05/06, whose untimed tracks carry send times, - /// and on lite-07 when the publisher says so. + /// Lite-05/06 timestamps may be send times, so a received frame is read past its + /// timestamp and lands untimed: nothing rewinds, and the track stays open. #[moq_clock::test] - async fn an_unverified_rewind_is_accepted() { - for (version, unverified) in [ - (Version::Lite05, false), - (Version::Lite06, false), - (Version::Lite07, true), - ] { - let (res, track) = receive_rewind(version, unverified).await; + async fn lite06_timestamps_are_dropped() { + for version in [Version::Lite05, Version::Lite06] { + let (res, track) = receive_rewind(version).await; res.unwrap_or_else(|err| panic!("{version}: {err:?}")); assert!(track.closed().now_or_never().is_none(), "{version}"); + let mut consumer = track.subscribe(None); + let mut group = consumer.recv_group().now_or_never().unwrap().unwrap().unwrap(); + assert_eq!(group.sequence, 5, "{version}"); + let frame = group.read_frame().now_or_never().unwrap().unwrap().unwrap(); + assert_eq!(frame.payload.as_ref(), b"rewound", "{version}"); + assert_eq!(frame.timestamp, None, "{version}"); } } - /// A lite-07 track whose publisher vouches for its timestamps is checked like a local - /// write, so a received group that rewinds is refused. + /// Lite-07 timestamps are media time, so a received group is checked like a local + /// write and one that rewinds is refused. #[moq_clock::test] - async fn a_verified_rewind_is_refused() { - let (res, _track) = receive_rewind(Version::Lite07, false).await; + async fn a_lite07_rewind_is_refused() { + let (res, _track) = receive_rewind(Version::Lite07).await; assert!(matches!(res, Err(Error::TimestampMismatch)), "{res:?}"); } @@ -3227,8 +3226,9 @@ mod tests { let broadcast = crate::broadcast::Info::new().produce(); // The broadcast keeps only a weak handle, so the map below owns the only strong // `track::Producer`: dropping it is what ends the track. + // Untimed, as the subscriber drops lite-05 timestamps. let producer = broadcast - .create_track("datagrams", crate::track::Info::timed()) + .create_track("datagrams", crate::track::Info::default()) .unwrap(); let mut received = producer.subscribe(None); subscriber.subscribes.lock().insert( @@ -6237,10 +6237,15 @@ impl kio::Task for TrackServeRun { let res = ready!(info.poll_fetch(&self.serve, waiter)); let request = request.take().expect("request pending"); match res { - Ok((info, held)) => { + Ok((mut info, held)) => { // Lite05 carries per-frame timestamps on the wire at this scale; // `Some` tells the ingest to decode them. let timescale = info.timescale; + // Lite05/06 timestamps may be send times, so the copy is untimed + // and the ingest drops them. + if !self.serve.subscriber.version.has_media_time() { + info.timescale = None; + } let mut serve_loop = ServeLoop::new(&self.serve, request, info, timescale); serve_loop.held = Some(held); self.state = TrackRunState::Serve(serve_loop); @@ -7258,7 +7263,10 @@ impl kio::Task for FetchServeRun { // it. The track::Info only takes effect if the track isn't accepted yet // (a fetch with no live subscription); otherwise the group inherits the // accepted timescale. - let group_info = track::Info::default().with_timescale(self.timescale); + let group_info = track::Info::default().with_timescale( + self.timescale + .filter(|_| self.serve.subscriber.version.has_media_time()), + ); // The joined fetches pick the group up from the cache only when next // polled, so their demand outlives the request until then. let joined = request.result.clone(); diff --git a/rs/moq-net/src/lite/track.rs b/rs/moq-net/src/lite/track.rs index 2f8cdf2d0b1..3d941bb6f9a 100644 --- a/rs/moq-net/src/lite/track.rs +++ b/rs/moq-net/src/lite/track.rs @@ -70,10 +70,6 @@ pub struct TrackInfo { /// Lite07 encodes `None` as zero; Lite05/06 have no untimed encoding, so `None` /// goes out as milliseconds (see [`Version::wire_timescale`]). pub timescale: Option, - /// Whether a timed track's timestamps may rewind, so receivers don't check them (see - /// [`crate::track::Info::unverified`]). Carried on lite-07; every timed track received - /// over lite-05/06 is unverified. - pub unverified: bool, } impl From<&crate::track::Info> for TrackInfo { @@ -82,21 +78,19 @@ impl From<&crate::track::Info> for TrackInfo { refresh: info.refresh, priority: info.priority, max_age: info.max_age, - timescale: info.timescale, - unverified: info.unverified, + // Unchecked timestamps never enter moq-lite, whose receivers check them. + timescale: info.lite_timescale(), } } } impl From for crate::track::Info { fn from(info: TrackInfo) -> Self { - let mut model = Self::default() + Self::default() .with_timescale(info.timescale) .with_max_age(info.max_age) .with_priority(info.priority) - .with_refresh(info.refresh); - model.unverified = info.unverified; - model + .with_refresh(info.refresh) } } @@ -129,24 +123,11 @@ impl Message for TrackInfo { false => None, }; - let unverified = match version.has_unverified() { - true => match r.u8()? { - 0 => false, - // Only a timed track has timestamps to doubt. - 1 if timescale.is_some() => true, - _ => return Err(DecodeError::InvalidValue), - }, - // lite-05/06 always carry a timescale, and an untimed track's send times can't be - // told from media time. - false => timescale.is_some(), - }; - Ok(Self { priority, refresh, max_age, timescale, - unverified, }) } @@ -168,12 +149,6 @@ impl Message for TrackInfo { if version.has_refresh() { w.varint(self.refresh.map_or(0, |ratio| u64::from(ratio) + 1))?; } - if version.has_unverified() { - if self.unverified && self.timescale.is_none() { - return Err(EncodeError::InvalidState); - } - w.u8(self.unverified.into()); - } Ok(()) } } @@ -188,7 +163,6 @@ mod test { priority: 7, max_age: Some(Duration::from_millis(2000)), timescale: Some(Timescale::MICRO), - unverified: false, } } @@ -310,77 +284,6 @@ mod test { assert_eq!(info_roundtrip(Version::Lite07, &info).timescale, None); } - #[test] - fn lite07_carries_unverified() { - for unverified in [false, true] { - let info = TrackInfo { - unverified, - ..info_sample() - }; - let buf = info.encode_bytes(Version::Lite07).unwrap(); - assert_eq!(buf.last(), Some(&u8::from(unverified))); - assert_eq!(info_roundtrip(Version::Lite07, &info).unverified, unverified); - } - } - - #[test] - fn unverified_matches_cross_language_wire_bytes_on_lite07() { - let info = TrackInfo { - refresh: None, - priority: 1, - max_age: None, - timescale: Some(Timescale::MILLI), - unverified: true, - }; - let buf = info.encode_bytes(Version::Lite07).unwrap(); - assert_eq!(buf.as_ref(), [0x06, 0x01, 0x00, 0x83, 0xe8, 0x00, 0x01]); - } - - /// lite-05/06 can't say, and an untimed track's send times look like media time there. - #[test] - fn lite06_timed_tracks_arrive_unverified() { - for version in [Version::Lite05, Version::Lite06] { - for timescale in [None, Some(Timescale::MICRO)] { - let info = TrackInfo { - timescale, - ..info_sample() - }; - assert!(info_roundtrip(version, &info).unverified, "{version}"); - } - } - } - - #[test] - fn lite07_refuses_unverified_on_an_untimed_track() { - let info = TrackInfo { - timescale: None, - unverified: true, - ..info_sample() - }; - assert!(matches!( - info.encode_bytes(Version::Lite07), - Err(EncodeError::InvalidState) - )); - - let mut raw = TrackInfo { - timescale: None, - ..info_sample() - } - .encode_bytes(Version::Lite07) - .unwrap() - .to_vec(); - *raw.last_mut().unwrap() = 1; - assert!(matches!( - TrackInfo::decode_slice(&raw, Version::Lite07), - Err(DecodeError::InvalidValue) - )); - *raw.last_mut().unwrap() = 2; - assert!(matches!( - TrackInfo::decode_slice(&raw, Version::Lite07), - Err(DecodeError::InvalidValue) - )); - } - #[test] fn lite06_declares_an_untimed_track_as_milliseconds() { for version in [Version::Lite05, Version::Lite06] { @@ -425,7 +328,6 @@ mod test { priority: 255, max_age: None, timescale: Some(Timescale::new((1u64 << 62) - 1).unwrap()), - unverified: false, }; let got = info_roundtrip(Version::Lite05, &info); assert_eq!(got.priority, 255); @@ -440,7 +342,6 @@ mod test { priority: 0, max_age: Some(Duration::from_nanos(999_999)), timescale: Some(Timescale::MILLI), - unverified: false, }; let got = info_roundtrip(Version::Lite05, &info); assert_eq!(got.max_age, Some(Duration::ZERO)); @@ -454,7 +355,6 @@ mod test { priority: 7, max_age: Some(Duration::from_millis(u64::MAX)), timescale: Some(Timescale::MILLI), - unverified: false, }; // The partial bytes are the Writer's to drop; see `a_failed_encode_leaves_no_partial_bytes`. assert!(matches!( diff --git a/rs/moq-net/src/lite/tracks.rs b/rs/moq-net/src/lite/tracks.rs index 650c9d245c3..c39fdeb40cb 100644 --- a/rs/moq-net/src/lite/tracks.rs +++ b/rs/moq-net/src/lite/tracks.rs @@ -131,7 +131,6 @@ mod test { priority: 3, max_age: Some(Duration::from_secs(2)), timescale: Some(Timescale::MICRO), - unverified: true, }; let TracksEvent::Added { name, info: got } = roundtrip(&TracksEvent::Added { name: "video".into(), diff --git a/rs/moq-net/src/lite/version.rs b/rs/moq-net/src/lite/version.rs index 7e28ba9cab1..f67e5f09085 100644 --- a/rs/moq-net/src/lite/version.rs +++ b/rs/moq-net/src/lite/version.rs @@ -320,11 +320,11 @@ impl Version { } } - /// Whether TRACK_INFO says if a timed track's timestamps are unverified. Added in - /// lite-07; earlier, a receiver cannot tell an untimed track's send times from media - /// time, so it treats every timed track as unverified. + /// Whether received frame timestamps are the publisher's media time. Lite05/06 stamp an + /// untimed track's payloads with their send time, which a receiver cannot tell from it, + /// so it drops every timestamp received on them and the track arrives untimed. #[allow(clippy::match_like_matches_macro)] - pub(crate) fn has_unverified(self) -> bool { + pub(crate) fn has_media_time(self) -> bool { // Match form so future versions default forward (AGENTS.md convention). match self { Self::Lite01 | Self::Lite02 | Self::Lite03 | Self::Lite04 | Self::Lite05 | Self::Lite06 => false, diff --git a/rs/moq-net/src/model/group.rs b/rs/moq-net/src/model/group.rs index a6737f39043..1cca9635a6a 100644 --- a/rs/moq-net/src/model/group.rs +++ b/rs/moq-net/src/model/group.rs @@ -2228,6 +2228,14 @@ impl Consumer { self.track.timescale } + /// The timescale a moq-lite hop serves the parent track at (see + /// [`track::Info::lite_timescale`]). + #[cfg_attr(not(feature = "async"), allow(dead_code))] + // private: passthrough + pub(crate) fn lite_timescale(&self) -> Option { + self.track.lite_timescale() + } + /// The index of the next frame this consumer will return. /// /// Starts at 0, or at the group's first available frame once [`Self::set_frames`] has diff --git a/rs/moq-net/src/model/track.rs b/rs/moq-net/src/model/track.rs index ad143f4b2ea..eb604f78c71 100644 --- a/rs/moq-net/src/model/track.rs +++ b/rs/moq-net/src/model/track.rs @@ -157,16 +157,14 @@ pub struct Info { #[cfg_attr(not(feature = "async"), allow(dead_code))] // private: passthrough pub(crate) properties: Option>, - /// Whether a hop stamped this timed track's timestamps, or received them over a wire - /// that cannot say they weren't, so they may rewind and no group is checked for it. + /// Whether this timed track's timestamps arrived over moq-transport, whose peers may + /// stamp arrival times, so no group is checked for a rewind. /// - /// A local publisher's writes are always checked. A received track is unverified when it - /// arrived over moq-lite 05/06 (an untimed track there carries send times) or - /// moq-transport (a peer may stamp arrival times), or over lite-07 from a hop that says - /// so. Never set on an untimed track. + /// They stay on moq-transport: a moq-lite hop serves the track untimed, since its + /// receivers check every timestamp. Never set on an untimed track. #[cfg_attr(not(feature = "async"), allow(dead_code))] // pending: /quest/m1/public-model/ingest.md - pub(crate) unverified: bool, + pub(crate) unchecked: bool, } impl Default for Info { @@ -177,7 +175,7 @@ impl Default for Info { priority: DEFAULT_PRIORITY, refresh: None, properties: None, - unverified: false, + unchecked: false, } } } @@ -210,6 +208,14 @@ impl Info { self.refresh = refresh.into(); self } + + /// The timescale a moq-lite hop serves this track at: none for an + /// [unchecked](Self::unchecked) one. + #[cfg_attr(not(feature = "async"), allow(dead_code))] + // private: passthrough + pub(crate) fn lite_timescale(&self) -> Option { + self.timescale.filter(|_| !self.unchecked) + } } #[cfg(test)] @@ -782,9 +788,9 @@ impl TrackState { /// so the watermark caps it instead. /// /// Usually one group lookup each way; only unstarted or partial neighbors are skipped. - /// `None` on an [unverified](Info::unverified) track, whose timestamps may rewind. + /// `None` on an [unchecked](Info::unchecked) track, whose timestamps may rewind. pub(super) fn neighbors(&self, sequence: u64) -> Option { - if self.info.as_ref().is_some_and(|info| info.unverified) { + if self.info.as_ref().is_some_and(|info| info.unchecked) { return None; } let bounds = self.bounds.lock().unwrap_or_else(PoisonError::into_inner); diff --git a/rs/moq-net/src/test_interop.rs b/rs/moq-net/src/test_interop.rs index d53ae3018a0..a72b0dd1480 100644 --- a/rs/moq-net/src/test_interop.rs +++ b/rs/moq-net/src/test_interop.rs @@ -174,7 +174,6 @@ fn lite_varint_interop() { priority: 1, max_age: None, timescale: None, - unverified: false, } .encode_bytes(version) .unwrap(); diff --git a/rs/moq-net/tests/integration/datagram.rs b/rs/moq-net/tests/integration/datagram.rs index ccce0f587a0..2b373c9c6f6 100644 --- a/rs/moq-net/tests/integration/datagram.rs +++ b/rs/moq-net/tests/integration/datagram.rs @@ -48,7 +48,8 @@ async fn connect_datagram_track() -> Fixture { let producer = broadcast.create_track("datagrams", timed()).unwrap(); broadcast.announce(Default::default()).unwrap(); - let mut options = MockConnectOptions::new("moq-lite-05".parse::().unwrap()); + // Lite-07, since a lite-05/06 subscriber drops every timestamp it receives. + let mut options = MockConnectOptions::new("moq-lite-07-wip".parse::().unwrap()); options.server_publish = Some(publisher.consume()); options.client_subscribe = Some(consumer_origin.clone()); let pair = connect_mock(options).await; diff --git a/rs/moq-net/tests/integration/received_rewind.rs b/rs/moq-net/tests/integration/received_rewind.rs index 73eb0f3907f..913d91a3de4 100644 --- a/rs/moq-net/tests/integration/received_rewind.rs +++ b/rs/moq-net/tests/integration/received_rewind.rs @@ -1,11 +1,11 @@ #![cfg(feature = "async")] -//! A relay re-serves unverified timestamps as unverified. +//! A relay drops the timestamps it receives over lite-05/06. //! //! Lite-06 stamps an untimed track's frames with their send time and declares -//! milliseconds, so the relay's copy reads as timed and re-advertises that scale. A group -//! sent after a higher one carries the later stamp, which no hop may refuse: the relay's -//! copy is unverified, and it says so to a lite-07 subscriber. +//! milliseconds, so a group sent after a higher one carries the later stamp. The relay +//! cannot tell that from media time, so its copy is untimed and every hop past it sees an +//! untimed track that nothing can refuse. use crate::support; @@ -28,21 +28,32 @@ fn produce_origin(hop: u64) -> moq_net::origin::Producer { } fn write_group(track: &moq_net::track::Producer, sequence: u64) { + // A timed track's group starts at its sequence in seconds, so none rewinds. + let timestamp = track + .info() + .timescale + .map(|_| moq_net::Timestamp::from_millis(sequence * 1000).unwrap()); let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); - group.write_frame(None, &b"x"[..]).unwrap(); + group.write_frame(timestamp, &b"x"[..]).unwrap(); group.finish().unwrap(); } -/// Groups 1 then 0 of an untimed track, into a relay over lite-06 and out over `downstream`. -/// Returns the scale the subscriber learned and the groups it read. -async fn relay(downstream: &str) -> (Option, Vec) { +/// Groups 1 then 0 of a track at `timescale`, into a relay over `upstream` and out over +/// `downstream`. Returns the scale the subscriber learned and the groups it read. +async fn relay( + upstream: &str, + downstream: &str, + timescale: Option, +) -> (Option, Vec) { let publisher = produce_origin(1); let broadcast = publisher.create_broadcast("bcast").unwrap(); - let track = broadcast.create_track("data", Info::default()).unwrap(); + let track = broadcast + .create_track("data", Info::default().with_timescale(timescale)) + .unwrap(); broadcast.announce(Default::default()).unwrap(); let relay_origin = produce_origin(2); - let mut options = MockConnectOptions::new("moq-lite-06".parse::().unwrap()); + let mut options = MockConnectOptions::new(upstream.parse::().unwrap()); options.server_publish = Some(publisher.consume()); options.client_subscribe = Some(relay_origin.clone()); let upstream = connect_mock(options).await; @@ -114,9 +125,27 @@ async fn relay(downstream: &str) -> (Option, Vec) { async fn a_relay_forwards_descending_send_times_without_closing_the_track() { let mut failures = Vec::new(); for downstream in ["moq-lite-06", "moq-lite-07-wip", "moq-transport-22"] { - let got = relay(downstream).await; - // The relay's copy reads as timed, so every hop past it sees a timescale. - if got.0.is_none() || got.1 != [1, 0] { + let got = relay("moq-lite-06", downstream, None).await; + if got.0.is_some() || got.1 != [1, 0] { + failures.push(format!("{downstream}: {got:?}")); + } + } + assert!(failures.is_empty(), "{failures:#?}"); +} + +/// A moq-transport peer may stamp arrival times, so a relay keeps a timed track's timestamps +/// unchecked and passes them on only over moq-transport: a moq-lite hop, which checks every +/// timestamp it receives, is served the track untimed. +#[moq_clock::test] +async fn moq_transport_timestamps_stay_on_moq_transport() { + let mut failures = Vec::new(); + for (downstream, expected) in [ + ("moq-lite-06", None), + ("moq-lite-07-wip", None), + ("moq-transport-22", Some(moq_net::Timescale::MILLI)), + ] { + let got = relay("moq-transport-22", downstream, Some(moq_net::Timescale::MILLI)).await; + if got.0 != expected || got.1 != [1, 0] { failures.push(format!("{downstream}: {got:?}")); } } @@ -125,7 +154,7 @@ async fn a_relay_forwards_descending_send_times_without_closing_the_track() { /// Fetch groups 1 then 0 of a finished untimed track over `version`, with no live /// subscription. Each answer carries its send time, so group 0 lands after group 1; the -/// copy the fetches fill is still received, so neither is refused. +/// copy the fetches fill drops them, so neither is refused. async fn fetch_only(version: &str) -> Vec> { let publisher = produce_origin(1); let broadcast = publisher.create_broadcast("bcast").unwrap(); diff --git a/rs/moq-net/tests/integration/untimed.rs b/rs/moq-net/tests/integration/untimed.rs index fce1facdfa8..6338e5adb6d 100644 --- a/rs/moq-net/tests/integration/untimed.rs +++ b/rs/moq-net/tests/integration/untimed.rs @@ -4,8 +4,8 @@ //! that can say so, and a timed track keeps its timestamps. //! //! No receiver fills in arrival time. Lite-05 and lite-06 cannot encode absence, so their -//! encoder writes its send time instead, and those hops are the one place an untimed track -//! gains a timeline. +//! encoder writes its send time instead, and their receivers drop every timestamp, which +//! leaves a timed track untimed too. use crate::support; @@ -27,16 +27,14 @@ const TIMEOUT: Duration = Duration::from_secs(10); enum Arrives { /// Both keep what the publisher wrote. Faithful, - /// The untimed track carries the encoder's send times (lite-05 and lite-06). - SendTime, - /// The wire declares no timescale, so even the timed track arrives untimed. + /// The wire carries no trustworthy timescale, so even the timed track arrives untimed. Untimed, } fn expected(version: &str) -> Arrives { match version { - "moq-lite-03" | "moq-lite-04" => Arrives::Untimed, - "moq-lite-05" | "moq-lite-06" => Arrives::SendTime, + // Lite-05 and lite-06 stamp untimed tracks with send times, so their receivers drop them. + "moq-lite-03" | "moq-lite-04" | "moq-lite-05" | "moq-lite-06" => Arrives::Untimed, // Drafts 14 to 16 can't carry TIMESCALE. "moq-transport-14" | "moq-transport-16" => Arrives::Untimed, _ => Arrives::Faithful, @@ -136,10 +134,6 @@ async fn frames_arrive_as_published() { assert_eq!(untimed, None, "{version}: absence survives"); assert_eq!(untimed_info.timescale, None, "{version}: no timeline is declared"); } - Arrives::SendTime => { - assert_eq!(timed, Some(millis(7)), "{version}: the timestamp survives"); - assert!(untimed.is_some(), "{version}: a lite encoder writes its send time"); - } Arrives::Untimed => { assert_eq!(timed, None, "{version}: no units, so no timestamp"); assert_eq!(timed_info.timescale, None, "{version}: no timeline is declared"); @@ -175,9 +169,8 @@ async fn a_relay_forwards_an_untimed_track_untimed() { /// An explicit start on an untimed track is honored the same way on lite and /// moq-transport: only an unfloored subscriber jumps to the latest group. /// -/// The subscription asks for history the way a resume does, with a budget covering it: -/// lite-05 and lite-06 carry send times, so downstream the groups are timed and a zero budget would -/// skip the older ones there, as it would on any timed track. moq-transport before draft +/// The subscription asks for history the way a resume does, with a budget covering it. +/// moq-transport before draft /// 20 can only join a past group through a joining FETCH, which our publisher refuses for /// groups before the live one, so draft 20 stands in for it. #[moq_clock::test] @@ -213,8 +206,8 @@ async fn an_explicit_start_holds_on_an_untimed_track() { } } -/// An untimed track's datagram arrives untimed, except with a send time on lite-05 and -/// lite-06. +/// An untimed track's datagram arrives untimed, though lite-05 and lite-06 carry a send +/// time that the receiver drops. #[moq_clock::test] async fn datagrams_arrive_as_published() { for version in ["moq-lite-05", "moq-lite-07-wip", "moq-transport-16", "moq-transport-20"] { @@ -235,8 +228,7 @@ async fn datagrams_arrive_as_published() { .expect("datagram timeout") .unwrap() .expect("a datagram"); - let stamps = version == "moq-lite-05"; - assert_eq!(datagram.timestamp.is_some(), stamps, "{version}"); + assert_eq!(datagram.timestamp, None, "{version}"); } } @@ -268,7 +260,6 @@ async fn a_fetched_untimed_frame_arrives_untimed() { let mut fetched = waiting.await.unwrap(); let frame = fetched.read_frame().await.unwrap().unwrap(); - let stamps = version == "moq-lite-06"; - assert_eq!(frame.timestamp.is_some(), stamps, "{version}"); + assert_eq!(frame.timestamp, None, "{version}"); } } diff --git a/rs/moq-net/tests/integration/wide_sequence.rs b/rs/moq-net/tests/integration/wide_sequence.rs index d1f626d950f..745329c806d 100644 --- a/rs/moq-net/tests/integration/wide_sequence.rs +++ b/rs/moq-net/tests/integration/wide_sequence.rs @@ -52,9 +52,9 @@ async fn largest_quic_sequence_reaches_the_subscriber() { assert_eq!(got.sequence, sequence, "{alpn} relayed={relayed}"); let frame = got.read_frame().await.unwrap().unwrap(); assert_eq!(frame.payload, b"ladder".as_ref()); - if alpn != "moq-lite-02" { - assert_eq!(frame.timestamp, Some(timestamp)); - } + // Lite-02 carries no timestamps, and a lite-06 subscriber drops them. + let timed = alpn == "moq-transport-19"; + assert_eq!(frame.timestamp, timed.then_some(timestamp), "{alpn} relayed={relayed}"); } } } From c4700ab11655a6157943490cd8906cfc106d6a23 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 23:09:08 -0700 Subject: [PATCH 04/14] quest: claim quest/m1/lite06-max-sequence From 10ad2aaedbd298f6ff8926485df7fd2668701213 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 23:20:21 -0700 Subject: [PATCH 05/14] fix(net)!: bound timestamps to 2^61 - 1 so lite-05/06 can carry every one lite-05/06 zigzag a group's first timestamp from zero into a QUIC varint, which holds at most 2^61 - 1. A larger timestamp was accepted by the model, then the lite-06 encoder failed the group stream before its header was visible, leaving the subscriber waiting forever. Bounding Timestamp itself refuses it at the write, and a received timestamp past the bound is a malformed track on every version, so a relay never holds one it cannot forward. Co-Authored-By: Claude Opus 5.5 --- drafts/draft-lcurley-moq-lite.md | 4 ++ quest/m1/README.md | 1 - quest/m1/lite06-max-sequence.md | 49 ----------------- rs/moq-cli/src/play/timeline.rs | 6 +- rs/moq-e2ee/src/tests.rs | 4 +- rs/moq-mux/src/clock.rs | 2 +- rs/moq-net/MIGRATE.md | 6 ++ rs/moq-net/src/ietf/object.rs | 6 +- rs/moq-net/src/lite/subscriber.rs | 38 ++++++++++--- rs/moq-net/src/model/time.rs | 55 +++++++++++++------ rs/moq-net/tests/integration/wide_sequence.rs | 2 +- rs/moq-relay/tests/integration/connection.rs | 2 +- rs/moq-video/src/capture/channel.rs | 2 +- test/wasm/generator.md | 9 +-- test/wasm/uniffi/cases.ts | 42 ++++++++------ 15 files changed, 121 insertions(+), 107 deletions(-) delete mode 100644 quest/m1/lite06-max-sequence.md diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 122a16e41a7..249d9f324d0 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -182,6 +182,8 @@ A Group Stream normally starts at frame 0, but MAY start later when the publishe Each frame of a timed Track carries a presentation timestamp expressed in the Track's `Timescale` (see [TRACK_INFO](#track-info)), used by the moq-lite layer for [expiration](#expiration) decisions. A Track without a `Timescale` is untimed: its frames and datagrams carry no timestamp. +A timestamp MUST be less than 2^61: lite-05 and lite-06 zigzag a Group's first [Timestamp Delta](#frame) into a QUIC varint, which carries no more, and the bound lets a relay forward any timestamp to any version. +A receiver treats a larger timestamp as a malformed Track. A publisher MUST NOT start a timed Group's frame 0 earlier than that of any lower-sequence Group it publishes. lite-05 and lite-06 cannot encode an untimed Track, so a publisher on those versions declares a `Timescale` of 1000 for it and stamps every frame and datagram with its send time in milliseconds. A receiver cannot tell such send times from presentation timestamps, and they need not rise with sequence, so it does not check received Groups against this rule. @@ -1761,6 +1763,8 @@ The `Message Length` describes the payload size on the wire. - Replaced ANNOUNCE_OK Active Count with an ordered, empty SYNCED event (0x4), marking the initial set without retaining routes that end before delivery. - A publisher must not start a timed Group's frame 0 earlier than a lower-sequence Group's. Receivers do not check it, since a received timestamp may be a send time. + +- Bounded every timestamp below 2^61, since a lite-05/06 Group's first Timestamp Delta carries no more, so a relay never holds one it cannot forward. A larger one is a malformed Track. - Added the reliable ordered RTCDataChannel binding, with one Qmux Record per binary message and no datagrams. - Added `Group Order` to SUBSCRIBE and SUBSCRIBE_UPDATE: descending (0x0, the previous fixed order) or ascending (0x1). A relay requests descending upstream unless every subscription it serves is ascending. diff --git a/quest/m1/README.md b/quest/m1/README.md index 0cec85655e8..7e84459fbc5 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -108,7 +108,6 @@ lands before the cut. - [GStreamer linger](/quest/m1/linger-gst.md) - `moq-gst` sinks finish their broadcast at end of stream instead of closing it - [Gateway linger](/quest/m1/linger-gateways.md) - HLS, RTMP, SRT and RTC imports linger at a clean end, cut short by a newer epoch at the same path - [Exporters follow a rendition to a new track](/quest/m1/export-rendition-rebind.md) - the fMP4, TS, MKV and FLV exporters rebuild a rendition whose resolved `track` or `broadcast` changes under the same ID -- [lite-06 max sequence](/quest/m1/lite06-max-sequence.md) - group sequence 2^62-1 reaches a lite-06 subscriber or is refused loudly, instead of vanishing - [Track live signal](/quest/m1/track-live.md) - a reader can wait for its track's feed to be live, so a player skips old cached groups while a fresh upstream subscription starts - [moq-mux docs](/quest/m1/docs-mux.md) - every public moq-mux item has rustdoc and the crate warns on missing docs - [Binding docs](/quest/m1/docs-bindings.md) - every public moq-ffi and moq-c item has rustdoc, carried into the generated bindings and `moq.h` diff --git a/quest/m1/lite06-max-sequence.md b/quest/m1/lite06-max-sequence.md deleted file mode 100644 index 8545cbb70a9..00000000000 --- a/quest/m1/lite06-max-sequence.md +++ /dev/null @@ -1,49 +0,0 @@ -# [S] Resolve lite-06's max-sequence browser gap - -## Goal - -A group whose sequence is 2^62-1, the largest QUIC varint, either reaches a moq-lite-06 -subscriber through the relay or is refused loudly where it cannot be carried. In the -browser fixture the publisher accepts it and the subscriber waits forever, while -moq-lite-02 and moq-transport-19 deliver it and 2^53+1 crosses all three. That fixture -also uses the sequence as the frame timestamp; isolate the two before changing behavior. - -## Plan - -Found by the browser binding spike (#5838): `test/wasm/uniffi/cases.ts` publishes the -ladder 5, 2^32, 2^53-1, 2^53+1, 2^62-1 from one moq-ffi session to another through each -relay flavour. On lite-06 it records 2^62-1 as a known gap, and the case fails once the -gap closes. - -Reproduce natively first, with a moq-net test over the memory transport and then through -moq-relay, to find whether the publisher, the relay, or the subscriber drops it. Fail -loud if the value is illegal on lite-06; otherwise deliver it. Land a regression test -with the fix, then remove the browser case's known-gap branch. - -Public API: none expected. Wire: none if the value is legal; otherwise the refusal -matches the published draft. - -Native reproduction on `0955b19c1` separates the two values the browser fixture couples: - -- The whole sequence ladder, including 2^62-1, arrives over moq-net memory transport - both directly and through a forwarding origin when timestamps are representable. -- The browser also writes `timestampUs = sequence`. Published lite-06 - [FRAME section 7.20](https://datatracker.ietf.org/doc/html/draft-lcurley-moq-lite-06#section-7.20) - zigzags the first timestamp from zero. Its largest positive first-frame timestamp is - 2^61-1 in the track's timescale; 2^62-1 encodes as 2^63-2 and cannot fit a QUIC varint. -- The actual moq-relay `Connection` over memory transport reproduces the browser hang - with `Timescale::MICRO` and `Timestamp::from_micros(sequence)`. On paused Tokio time, - `connection::largest_quic_sequence_reaches_the_subscriber` fails at the final lite-06 - group with `moq-lite-06 sequence=4611686018427387903 hung`. To reproduce from the - retained test, replace its timestamp's `sequence.min((1 << 61) - 1)` with `sequence`. - The command was - `nix develop --max-jobs 2 --command env CARGO_BUILD_JOBS=2 NEXTEST_TEST_THREADS=4 just rs test -p moq-relay --test integration -E 'test(largest_quic_sequence)'`. - -Keep the sequence tests independent of unsupported timestamps: use the same ladder -with each timestamp capped at 2^61-1 and check exact timestamp delivery where carried. -The browser known-gap branch remains until browser verification and an explicit -decision about surfacing an unencodable timestamp. The wire encoder already refuses -it, but resetting its group stream leaves the live subscription waiting when that -group has not become visible. Decide the refusal boundary and recovery contract -before changing it; do not reinterpret the published lite-05/06 framing or silently -alter the independent SUBSCRIBE_DROP work. diff --git a/rs/moq-cli/src/play/timeline.rs b/rs/moq-cli/src/play/timeline.rs index c699ce9d82f..881b791b2cc 100644 --- a/rs/moq-cli/src/play/timeline.rs +++ b/rs/moq-cli/src/play/timeline.rs @@ -519,9 +519,9 @@ pub(super) fn timestamp(timestamp: Timestamp) -> Duration { /// A duration from the start of the track as a wire timestamp, the inverse of /// [`timestamp`]. fn media(duration: Duration) -> Timestamp { - // A wire timestamp caps at 2^62 - 1, which is ~146,000 years of microseconds. - const MAX: u128 = (1 << 62) - 1; - Timestamp::from_micros(duration.as_micros().min(MAX) as u64).expect("clamped to the wire maximum") + // A timestamp caps at 2^61 - 1, which is ~73,000 years of microseconds. + const MAX: u128 = (1 << 61) - 1; + Timestamp::from_micros(duration.as_micros().min(MAX) as u64).expect("clamped to the timestamp maximum") } /// Where the audio track has reached, measured from its own origin so diff --git a/rs/moq-e2ee/src/tests.rs b/rs/moq-e2ee/src/tests.rs index 7454c8f0f3a..22f2f3e8c47 100644 --- a/rs/moq-e2ee/src/tests.rs +++ b/rs/moq-e2ee/src/tests.rs @@ -986,8 +986,8 @@ fn secrets_stay_out_of_debug() { fn failed_group_write_does_not_burn_nonce() { let mut pair = pair("video"); let mut group = pair.producer.append_group().unwrap(); - // (2^62-1) seconds overflows conversion to the milli-scale track. - let huge = moq_net::Timestamp::from_secs((1 << 62) - 1).unwrap(); + // (2^61-1) seconds overflows conversion to the milli-scale track. + let huge = moq_net::Timestamp::from_secs((1 << 61) - 1).unwrap(); assert!(group.write_frame(huge, b"bad").is_err()); assert_eq!(group.invocations(), 0); let big = vec![0u8; MAX_GROUPED_PLAINTEXT + 1]; diff --git a/rs/moq-mux/src/clock.rs b/rs/moq-mux/src/clock.rs index 1a237b14782..5352f84215f 100644 --- a/rs/moq-mux/src/clock.rs +++ b/rs/moq-mux/src/clock.rs @@ -469,7 +469,7 @@ mod tests { fn wall_clock_validates_bounds() { let clock = Clock::at(epoch(), moq_epoch()).unwrap(); // The largest representable broadcast timestamp still maps. - let max = moq_net::Timestamp::from_micros((1u64 << 62) - 1).unwrap(); + let max = moq_net::Timestamp::from_micros((1u64 << 61) - 1).unwrap(); assert!(clock.wall_clock(max).is_err(), "past the JSON-safe range is refused"); } } diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index 8009db2c319..dcfcad40420 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Timestamps stop at 2^61 - 1 + +`Timestamp::new`, its `from_*` constructors, `convert`, `checked_add`, and `TryFrom` return `TimeOverflow` past `2^61 - 1` instead of `2^62 - 1`, the most lite-05/06 can carry as a group's first frame. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `(1 << 61) - 1`. + ### Anchors can name a time (#6074) `track::Anchor` gains `Time(Timestamp)`, the group containing a presentation time, so exhaustive matches need an arm for it. diff --git a/rs/moq-net/src/ietf/object.rs b/rs/moq-net/src/ietf/object.rs index e95fab140fa..8640a64db6c 100644 --- a/rs/moq-net/src/ietf/object.rs +++ b/rs/moq-net/src/ietf/object.rs @@ -103,13 +103,13 @@ pub(crate) fn max_block(timescale: Option) -> usize { if timescale.is_none() { return MAX_BLOCK; } - // A Timestamp's value is a QUIC varint at most, and its key, a delta of at most 0x10, + // A Timestamp's value is at most MAX_TIMESTAMP, and its key, a delta of at most 0x10, // only shrinks the key of the property after it. let largest = [Version::Draft14, Version::Draft16, Version::Draft17, Version::Draft18] .into_iter() .map(|version| { let form = version.into(); - crate::coding::varint::size(0x10, form).unwrap() + crate::coding::varint::size((1 << 62) - 1, form).unwrap() + crate::coding::varint::size(0x10, form).unwrap() + crate::coding::varint::size(crate::model::MAX_TIMESTAMP, form).unwrap() }) .max() .unwrap(); @@ -609,7 +609,7 @@ mod tests { let decoded = read(&list(limit), Version::Draft22, milli).unwrap(); assert_eq!(decoded.properties.heap(), limit as u64); assert!(check_frame(&decoded.properties, milli).is_ok()); - let latest = Timestamp::new((1 << 62) - 1, Timescale::MILLI).unwrap(); + let latest = Timestamp::new(crate::model::MAX_TIMESTAMP, Timescale::MILLI).unwrap(); let written = write(&decoded.properties, Some(latest), Version::Draft22, milli).unwrap(); assert_eq!(written.len(), MAX_BLOCK); diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 4b882f0f2fa..dbab5c38167 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -2863,9 +2863,10 @@ mod tests { ); } - /// Receive group 5 starting at 500ms over `version`, on a received timed track whose - /// group 4 starts at 1000ms, returning the stream's result and the track. - async fn receive_rewind(version: Version) -> (Result<(), Error>, track::Producer) { + /// Receive group 5 over `version`, one frame per millisecond timestamp delta, on a + /// received timed track whose group 4 starts at 1000ms, returning the stream's result + /// and the track. + async fn receive_group(version: Version, deltas: &[i64]) -> (Result<(), Error>, track::Producer) { use crate::transport::poll::Session as _; let mut script = Vec::new(); @@ -2877,9 +2878,11 @@ mod tests { } .encode(&mut encoder, version) .unwrap(); - // Frame 0 at 500ms: the first delta is absolute. - encoder.varint(crate::coding::varint::zigzag(500)).unwrap(); - encoder.bytes(b"rewound").unwrap(); + // The first delta is absolute. + for delta in deltas { + encoder.varint(crate::coding::varint::zigzag(*delta)).unwrap(); + encoder.bytes(b"frame").unwrap(); + } let mut session = crate::lite::test_transport::ScriptedSession::eof(script); let subscriber = Subscriber::new(SubscriberConfig { runtime: crate::time::sim(), @@ -2926,12 +2929,33 @@ mod tests { #[moq_clock::test] async fn a_received_rewind_is_accepted_at_every_version() { for version in [Version::Lite05, Version::Lite06, Version::Lite07] { - let (res, track) = receive_rewind(version).await; + // Frame 0 at 500ms, before group 4's 1000ms. + let (res, track) = receive_group(version, &[500]).await; res.unwrap_or_else(|err| panic!("{version}: {err:?}")); assert!(track.closed().now_or_never().is_none(), "{version}"); } } + /// A received timestamp past 2^61 - 1 is refused at every version, so a relay never + /// holds one that lite-05/06 could not carry as a group's first frame. lite-06 reaches + /// it with a second delta, lite-07 with a first frame its 64-bit varint can encode. + #[moq_clock::test] + async fn a_received_timestamp_past_2_61_is_refused() { + let max = crate::model::MAX_TIMESTAMP as i64; + for version in [Version::Lite05, Version::Lite06, Version::Lite07] { + let (res, track) = receive_group(version, &[max]).await; + res.unwrap_or_else(|err| panic!("{version}: {err:?}")); + let mut group = track.consume().peek_group(5).unwrap(); + let frame = group.read_frame().now_or_never().unwrap().unwrap().unwrap(); + assert_eq!(frame.timestamp.map(Timestamp::value), Some(max as u64), "{version}"); + + let (res, _) = receive_group(version, &[max, 1]).await; + assert!(matches!(res, Err(Error::BoundsExceeded(_))), "{version}: {res:?}"); + } + let (res, _) = receive_group(Version::Lite07, &[max + 1]).await; + assert!(matches!(res, Err(Error::BoundsExceeded(_))), "{res:?}"); + } + /// An idle copy that resubscribes gets its newest cached group again. That stream only /// duplicates the cache, but lite-07's SUBSCRIBE_END still counts it, so the tail must /// too, or the subscription waits out the grace before it ends. diff --git a/rs/moq-net/src/model/time.rs b/rs/moq-net/src/model/time.rs index 8d433a1571b..54986236afb 100644 --- a/rs/moq-net/src/model/time.rs +++ b/rs/moq-net/src/model/time.rs @@ -2,19 +2,25 @@ use std::num::NonZero; use crate::coding::varint::MAX_QUIC; -/// `value`, or `None` past the QUIC varint limit (`2^62 - 1`), so every timestamp stays -/// encodable on moq-lite. -const fn quic(value: u128) -> Option { - if value <= MAX_QUIC as u128 { +/// The largest [`Timestamp`] value, `2^61 - 1`. +/// +/// lite-05/06 zigzag a group's first timestamp from zero into a QUIC varint, which halves +/// its positive range. Bounding every timestamp here keeps each one encodable on every +/// version, including after a relay forwards it from lite-07 or moq-transport. +pub(crate) const MAX_TIMESTAMP: u64 = (1 << 61) - 1; + +/// `value`, or `None` past `max`. +const fn bounded(value: u128, max: u64) -> Option { + if value <= max as u128 { Some(value as u64) } else { None } } -/// Returned when a [`Timestamp`] operation would exceed the QUIC varint range -/// (`2^62 - 1`), overflow during scale conversion or arithmetic, or attempt -/// arithmetic between timestamps with mismatched scales. +/// Returned when a [`Timestamp`] operation would exceed its range (`2^61 - 1`), a +/// [`Timescale`] would exceed the QUIC varint range (`2^62 - 1`), scale conversion or +/// arithmetic would overflow, or arithmetic mixes timestamps with mismatched scales. #[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] #[error("time overflow")] pub struct TimeOverflow; @@ -60,7 +66,7 @@ impl Timescale { pub const fn new(units_per_second: u64) -> Result { // Reject values that wouldn't fit in a QUIC varint, keeping the constraint // symmetric with Timestamp's raw value. - if quic(units_per_second as u128).is_none() { + if bounded(units_per_second as u128, MAX_QUIC).is_none() { return Err(TimeOverflow); } match NonZero::new(units_per_second) { @@ -126,8 +132,8 @@ impl std::fmt::Display for Timescale { /// A timestamp in a track's timescale (units per second). /// /// All timestamps within a track are relative, so zero for one track is not zero for another. -/// The underlying value is constrained to fit within a QUIC varint (`2^62 - 1`) so it can be -/// encoded and decoded easily; the scale is carried alongside so frames from different +/// The underlying value is at most `2^61 - 1`, the most every moq-lite version can encode as +/// a group's first frame; the scale is carried alongside so frames from different /// sources can be compared and converted without lossy detours through a single fixed scale. /// /// The scale is a [`Timescale`] (always non-zero), so unit conversions (`as_secs`, `as_millis`, @@ -172,9 +178,9 @@ impl Timestamp { pub const ZERO: Self = Self::new_const(0, Timescale::SECOND); /// Construct a timestamp directly from a raw value at the given scale. - /// Returns [`TimeOverflow`] if `value` exceeds `2^62 - 1`. + /// Returns [`TimeOverflow`] if `value` exceeds `2^61 - 1`. pub const fn new(value: u64, scale: Timescale) -> Result { - match quic(value as u128) { + match bounded(value as u128, MAX_TIMESTAMP) { Some(value) => Ok(Self { value, scale }), None => Err(TimeOverflow), } @@ -184,7 +190,7 @@ impl Timestamp { const fn new_const(value: u64, scale: Timescale) -> Self { match Self::new(value, scale) { Ok(time) => time, - Err(_) => panic!("timestamp value exceeds 2^62 - 1"), + Err(_) => panic!("timestamp value exceeds 2^61 - 1"), } } @@ -230,13 +236,13 @@ impl Timestamp { } /// Re-express this timestamp at a new scale. Returns [`TimeOverflow`] if the new - /// value would exceed `2^62 - 1`. + /// value would exceed `2^61 - 1`. pub const fn convert(self, new_scale: Timescale) -> Result { if self.scale.0.get() == new_scale.0.get() { return Ok(self); } match (self.value as u128).checked_mul(new_scale.0.get() as u128) { - Some(scaled) => match quic(scaled / self.scale.0.get() as u128) { + Some(scaled) => match bounded(scaled / self.scale.0.get() as u128, MAX_TIMESTAMP) { Some(value) => Ok(Self { value, scale: new_scale, @@ -272,7 +278,7 @@ impl Timestamp { self.as_scale(Timescale::NANO) } - /// Add two timestamps. Returns [`TimeOverflow`] if the sum exceeds `2^62 - 1` or + /// Add two timestamps. Returns [`TimeOverflow`] if the sum exceeds `2^61 - 1` or /// if the scales differ. pub const fn checked_add(self, rhs: Self) -> Result { if self.scale.0.get() != rhs.scale.0.get() { @@ -312,7 +318,7 @@ impl TryFrom for Timestamp { /// Convert a [`std::time::Duration`] into a nanosecond-scale timestamp. fn try_from(duration: std::time::Duration) -> Result { - match quic(duration.as_nanos()) { + match bounded(duration.as_nanos(), MAX_TIMESTAMP) { Some(value) => Ok(Self { value, scale: Timescale::NANO, @@ -440,6 +446,21 @@ mod tests { assert!(Timescale::new((1u64 << 62) - 1).is_ok()); } + /// A timestamp is refused where any version could not carry it as a group's first frame, + /// which lite-05/06 zigzag from zero into a QUIC varint. + #[test] + fn test_timestamp_fits_a_lite06_first_frame() { + let max = Timestamp::new(MAX_TIMESTAMP, Timescale::MICRO).unwrap(); + assert!(crate::coding::varint::zigzag(max.value() as i64) <= MAX_QUIC); + + for value in [MAX_TIMESTAMP + 1, MAX_QUIC] { + assert!(Timestamp::from_micros(value).is_err(), "{value}"); + } + assert!(max.checked_add(Timestamp::from_micros(1).unwrap()).is_err()); + assert!(max.convert(Timescale::NANO).is_err()); + assert!(Timestamp::try_from(std::time::Duration::from_nanos(MAX_TIMESTAMP + 1)).is_err()); + } + #[test] fn test_timescale_try_from_nonzero_enforces_varint() { use std::num::NonZero; diff --git a/rs/moq-net/tests/integration/wide_sequence.rs b/rs/moq-net/tests/integration/wide_sequence.rs index d1f626d950f..3dea2632e0e 100644 --- a/rs/moq-net/tests/integration/wide_sequence.rs +++ b/rs/moq-net/tests/integration/wide_sequence.rs @@ -39,7 +39,7 @@ async fn largest_quic_sequence_reaches_the_subscriber() { let remote = client.consume().request_broadcast("ladder", None).await.unwrap(); let mut sub = remote.track("data").unwrap().subscribe(None).await.unwrap(); for sequence in [5, 1 << 32, (1 << 53) - 1, (1 << 53) + 1, (1 << 62) - 1] { - // Lite05/06 zigzag the first timestamp from zero, halving its positive range. + // Timestamps stop at 2^61 - 1, the most lite-05/06 can zigzag as a first frame. let timestamp = Timestamp::from_micros(sequence.min((1 << 61) - 1)).unwrap(); let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); group.write_frame(timestamp, b"ladder".as_ref()).unwrap(); diff --git a/rs/moq-relay/tests/integration/connection.rs b/rs/moq-relay/tests/integration/connection.rs index 9dff056a5d4..86f6cdc07e9 100644 --- a/rs/moq-relay/tests/integration/connection.rs +++ b/rs/moq-relay/tests/integration/connection.rs @@ -47,7 +47,7 @@ async fn largest_quic_sequence_reaches_the_subscriber() { let remote = subscriber.consume().request_broadcast("ladder", None).await.unwrap(); let mut sub = remote.track("data").unwrap().subscribe(None).await.unwrap(); for sequence in [5, 1 << 32, (1 << 53) - 1, (1 << 53) + 1, (1 << 62) - 1] { - // Lite05/06 zigzag the first timestamp from zero, halving its positive range. + // Timestamps stop at 2^61 - 1, the most lite-05/06 can zigzag as a first frame. let timestamp = moq_net::Timestamp::from_micros(sequence.min((1 << 61) - 1)).unwrap(); let mut group = track.create_group(moq_net::group::Info { sequence }).unwrap(); group.write_frame(timestamp, b"ladder".as_ref()).unwrap(); diff --git a/rs/moq-video/src/capture/channel.rs b/rs/moq-video/src/capture/channel.rs index a80a9d0ca7f..884f7ea37a0 100644 --- a/rs/moq-video/src/capture/channel.rs +++ b/rs/moq-video/src/capture/channel.rs @@ -368,7 +368,7 @@ mod tests { let at = |millis| chan.epoch + std::time::Duration::from_millis(millis); chan.push_native_at(frame(1), us(0), at(0)); - chan.push_native_at(frame(2), us((1 << 62) - 1), at(1)); + chan.push_native_at(frame(2), us((1 << 61) - 1), at(1)); chan.push_native_at(frame(3), us(0), at(2)); assert!(matches!(chan.recv().await, Err(Error::TimeOverflow(_)))); diff --git a/test/wasm/generator.md b/test/wasm/generator.md index 49b7523d757..2b58a0589d8 100644 --- a/test/wasm/generator.md +++ b/test/wasm/generator.md @@ -249,7 +249,8 @@ targets, over one dual-annotated crate. The measurements above used a workaround in the pin patch. Since [#5681](https://github.com/moq-dev/moq/pull/5681), moq-mux reads moq-clock's browser wall clock instead, and the cases pass without the workaround. -- **On moq-lite-06, group sequence 2^62-1 never arrives.** lite-02 and transport-19 - carry it, and 2^53+1 crosses all three. Nothing errors: the subscriber waits forever. - Both moq-ffi and the slice hit it under both generators. Tracked in - [lite-06 max sequence](/quest/m1/lite06-max-sequence.md). +- **On moq-lite-06, group sequence 2^62-1 never arrived.** The cases also stamped it as + the frame timestamp, which lite-05/06 cannot zigzag as a group's first frame, so the + publisher dropped the group and the subscriber waited forever. Both moq-ffi and the + slice hit it under both generators. Timestamps now stop at 2^61-1, so `writeFrame` + refuses the unencodable one and the sequence itself crosses every version. diff --git a/test/wasm/uniffi/cases.ts b/test/wasm/uniffi/cases.ts index d6d0c5061ce..ff47a2b5a18 100644 --- a/test/wasm/uniffi/cases.ts +++ b/test/wasm/uniffi/cases.ts @@ -51,6 +51,12 @@ export interface Report { /** Above 2^53, so a lossy `number` conversion anywhere changes it; the largest QUIC varint. */ const HIGH = [2n ** 53n + 1n, 2n ** 62n - 1n]; +/** The largest timestamp, which lite-05/06 can still zigzag as a group's first frame. */ +const MAX_TIMESTAMP = 2n ** 61n - 1n; + +/** The ladder's timestamp for `sequence`, capped where a timestamp stops. */ +const stamp = (sequence: bigint) => (sequence < MAX_TIMESTAMP ? sequence : MAX_TIMESTAMP); + /** Microsecond ticks, so `timestampUs` passes through unscaled. */ const TIMED = MoqTrackInfo.create({ timescale: 1_000_000n }); @@ -131,13 +137,24 @@ const local: Record Promise> = { for (const sequence of HIGH) { const { group, consumer } = localGroup(sequence); if (consumer.sequence() !== sequence) throw new Error(`sequence ${consumer.sequence()} != ${sequence}`); - group.writeFrame({ payload: payload(1), timestampUs: sequence }); + group.writeFrame({ payload: payload(1), timestampUs: stamp(sequence) }); const frame = await consumer.readFrame(); - if (frame?.timestampUs !== sequence) throw new Error(`timestamp ${frame?.timestampUs} != ${sequence}`); + if (frame?.timestampUs !== stamp(sequence)) + throw new Error(`timestamp ${frame?.timestampUs} != ${stamp(sequence)}`); } return undefined; }, + "writeFrame refuses a timestamp past 2^61 - 1": async () => { + const { group } = localGroup(); + try { + group.writeFrame({ payload: payload(1), timestampUs: MAX_TIMESTAMP + 1n }); + } catch (err) { + return (err as Error).message; + } + throw new Error("an unencodable timestamp was written"); + }, + "AbortSignal rejects a pending read without stealing the next frame": async () => { const { group, consumer } = localGroup(); const abort = new AbortController(); @@ -251,25 +268,16 @@ const remote: Record Promise> = { const seen: string[] = []; for (const sequence of [5n, 2n ** 32n, 2n ** 53n - 1n, ...HIGH]) { const group = track.createGroup(sequence); - group.writeFrame({ payload: payload(160), timestampUs: sequence }); + group.writeFrame({ payload: payload(160), timestampUs: stamp(sequence) }); group.finish(); - const next = within(sub.recvGroup(), 5000, `recvGroup ${sequence} after [${seen}]`); - // Known: on moq-lite-06 the largest varint never arrives, while lite-02 and IETF carry it. - if (sequence === HIGH[1] && relay.version === "moq-lite-06") { - const outcome = await next.then( - (group) => `arrived as ${group?.sequence()}`, - (err: unknown) => (err as Error).message, - ); - if (outcome === `arrived as ${sequence}`) - throw new Error("known lite-06 gap now passes; update the case"); - return `known gap: 2^62-1 ${outcome}`; - } - const got = (await next) as MoqGroupConsumerLike | undefined; + const got = (await within(sub.recvGroup(), 5000, `recvGroup ${sequence} after [${seen}]`)) as + | MoqGroupConsumerLike + | undefined; if (got?.sequence() !== sequence) throw new Error(`sequence ${got?.sequence()} != ${sequence}`); const frame = await got.readFrame(); // moq-lite-02 carries no timestamp; every other version must keep it exact. - if (relay.version !== "moq-lite-02" && frame?.timestampUs !== sequence) - throw new Error(`timestamp ${frame?.timestampUs} != ${sequence}`); + if (relay.version !== "moq-lite-02" && frame?.timestampUs !== stamp(sequence)) + throw new Error(`timestamp ${frame?.timestampUs} != ${stamp(sequence)}`); seen.push(String(sequence)); } return undefined; From 4d31d737c326d13f93379948190e8e2bcc9c70c2 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 23:42:07 -0700 Subject: [PATCH 06/14] fix(net): keep the timestamp bound private to the model Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/ietf/object.rs | 6 +++--- rs/moq-net/src/lite/subscriber.rs | 2 +- rs/moq-net/src/model/time.rs | 8 ++------ 3 files changed, 6 insertions(+), 10 deletions(-) diff --git a/rs/moq-net/src/ietf/object.rs b/rs/moq-net/src/ietf/object.rs index 8640a64db6c..6294422fd3b 100644 --- a/rs/moq-net/src/ietf/object.rs +++ b/rs/moq-net/src/ietf/object.rs @@ -103,13 +103,13 @@ pub(crate) fn max_block(timescale: Option) -> usize { if timescale.is_none() { return MAX_BLOCK; } - // A Timestamp's value is at most MAX_TIMESTAMP, and its key, a delta of at most 0x10, + // A Timestamp's value is a QUIC varint at most, and its key, a delta of at most 0x10, // only shrinks the key of the property after it. let largest = [Version::Draft14, Version::Draft16, Version::Draft17, Version::Draft18] .into_iter() .map(|version| { let form = version.into(); - crate::coding::varint::size(0x10, form).unwrap() + crate::coding::varint::size(crate::model::MAX_TIMESTAMP, form).unwrap() + crate::coding::varint::size(0x10, form).unwrap() + crate::coding::varint::size((1 << 62) - 1, form).unwrap() }) .max() .unwrap(); @@ -609,7 +609,7 @@ mod tests { let decoded = read(&list(limit), Version::Draft22, milli).unwrap(); assert_eq!(decoded.properties.heap(), limit as u64); assert!(check_frame(&decoded.properties, milli).is_ok()); - let latest = Timestamp::new(crate::model::MAX_TIMESTAMP, Timescale::MILLI).unwrap(); + let latest = Timestamp::new((1 << 61) - 1, Timescale::MILLI).unwrap(); let written = write(&decoded.properties, Some(latest), Version::Draft22, milli).unwrap(); assert_eq!(written.len(), MAX_BLOCK); diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index dbab5c38167..fa7151eb298 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -2941,7 +2941,7 @@ mod tests { /// it with a second delta, lite-07 with a first frame its 64-bit varint can encode. #[moq_clock::test] async fn a_received_timestamp_past_2_61_is_refused() { - let max = crate::model::MAX_TIMESTAMP as i64; + let max: i64 = (1 << 61) - 1; for version in [Version::Lite05, Version::Lite06, Version::Lite07] { let (res, track) = receive_group(version, &[max]).await; res.unwrap_or_else(|err| panic!("{version}: {err:?}")); diff --git a/rs/moq-net/src/model/time.rs b/rs/moq-net/src/model/time.rs index 54986236afb..82aed585841 100644 --- a/rs/moq-net/src/model/time.rs +++ b/rs/moq-net/src/model/time.rs @@ -7,15 +7,11 @@ use crate::coding::varint::MAX_QUIC; /// lite-05/06 zigzag a group's first timestamp from zero into a QUIC varint, which halves /// its positive range. Bounding every timestamp here keeps each one encodable on every /// version, including after a relay forwards it from lite-07 or moq-transport. -pub(crate) const MAX_TIMESTAMP: u64 = (1 << 61) - 1; +const MAX_TIMESTAMP: u64 = (1 << 61) - 1; /// `value`, or `None` past `max`. const fn bounded(value: u128, max: u64) -> Option { - if value <= max as u128 { - Some(value as u64) - } else { - None - } + if value <= max as u128 { Some(value as u64) } else { None } } /// Returned when a [`Timestamp`] operation would exceed its range (`2^61 - 1`), a From 6aa695775b2cd9a033ff4290d4473a96a41d5686 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 23:47:22 -0700 Subject: [PATCH 07/14] test: expect lite-05/06 subscribers to drop timestamps Tests that read timestamps back over lite-05/06 now expect them dropped, or move to lite-07 where the timestamps are the point. The test relay also accepts lite-07 for a client that opts in. TrackInfo::serve replaces the From conversion now that compression depends on the version. Co-Authored-By: Claude Opus 5.5 --- rs/moq-cli/src/publish.rs | 7 +++-- rs/moq-relay/src/test_support.rs | 5 +++ rs/moq-relay/tests/integration/connection.rs | 6 ++-- rs/moq-tokio/tests/integration/broadcast.rs | 33 +++++++------------- 4 files changed, 24 insertions(+), 27 deletions(-) diff --git a/rs/moq-cli/src/publish.rs b/rs/moq-cli/src/publish.rs index 8c37632f784..e1ebb931bda 100644 --- a/rs/moq-cli/src/publish.rs +++ b/rs/moq-cli/src/publish.rs @@ -1177,7 +1177,7 @@ pub(crate) mod tests { let _env = crate::test_env::EnvGuard::clear(&["MOQ_CONNECT", "MOQ_HOP"]); let _ = moq_tokio::crypto::install_default(); - for version in ["moq-lite-06", "moq-transport-14"] { + for version in ["moq-lite-06", "moq-lite-07-wip", "moq-transport-14"] { let fixture = moq_relay::test_relay().await.unwrap(); let ready = fixture.relay.ready(); tokio::spawn(fixture.relay.run()); @@ -1331,8 +1331,9 @@ pub(crate) mod tests { // Joining once every group is out, with a budget half the jump, reads every group // after it past the marker, and none of the groups the jump left more than // the budget behind. An IETF subscription starts at the newest group, so only lite - // has the groups behind it to read. - if version.starts_with("moq-lite") { + // has the groups behind it to read, and a lite-06 subscriber drops the timestamps + // that carry the jump. + if version == "moq-lite-07-wip" { let late = read(start(moq_net::track::Subscription::default().with_max_delay(Duration::from_secs(10))).await) .await diff --git a/rs/moq-relay/src/test_support.rs b/rs/moq-relay/src/test_support.rs index 7303f2355e4..419f5a6f9de 100644 --- a/rs/moq-relay/src/test_support.rs +++ b/rs/moq-relay/src/test_support.rs @@ -19,10 +19,15 @@ pub struct TestRelay { } /// Bind a public test relay on ephemeral loopback ports without TOML or port probes. +/// +/// It also accepts the work-in-progress moq-lite-07, for a client that opts in. pub async fn test_relay() -> anyhow::Result { let mut config = Config::default(); config.drain_timeout = std::time::Duration::ZERO; config.listen.bind = Some("127.0.0.1:0".parse()?); + config.listen.version = std::iter::once("moq-lite-07-wip".parse().map_err(anyhow::Error::msg)?) + .chain(moq_net::Versions::all().iter().copied()) + .collect(); config.listen.tls.generate = vec!["localhost".into()]; config.web.http.listen = Some("127.0.0.1:0".parse()?); config.auth.public = vec![moq_auth::Pattern::all()]; diff --git a/rs/moq-relay/tests/integration/connection.rs b/rs/moq-relay/tests/integration/connection.rs index 9dff056a5d4..45a0133c185 100644 --- a/rs/moq-relay/tests/integration/connection.rs +++ b/rs/moq-relay/tests/integration/connection.rs @@ -60,9 +60,9 @@ async fn largest_quic_sequence_reaches_the_subscriber() { assert_eq!(got.sequence, sequence, "{alpn}"); let frame = got.read_frame().await.unwrap().unwrap(); assert_eq!(frame.payload, b"ladder".as_ref()); - if alpn != "moq-lite-02" { - assert_eq!(frame.timestamp, Some(timestamp)); - } + // Lite-02 carries no timestamps, and a lite-06 subscriber drops them. + let timed = alpn == "moq-transport-19"; + assert_eq!(frame.timestamp, timed.then_some(timestamp), "{alpn}"); } } } diff --git a/rs/moq-tokio/tests/integration/broadcast.rs b/rs/moq-tokio/tests/integration/broadcast.rs index a9021e1cdd8..0411a770512 100644 --- a/rs/moq-tokio/tests/integration/broadcast.rs +++ b/rs/moq-tokio/tests/integration/broadcast.rs @@ -135,7 +135,9 @@ async fn broadcast_test(scheme: &str, client_version: Option<&str>, server_versi } /// Lite05 publisher↔subscriber round-trip exercising the per-frame timestamp -/// delta encoding, including negative deltas (B-frame ordering). +/// delta encoding, including negative deltas (B-frame ordering). The subscriber decodes +/// each delta to stay in step with the payloads, then drops it, since a lite-05 timestamp +/// may be a send time. async fn lite05_timestamp_roundtrip(scheme: &str) { use moq_tokio::moq_net::{Timescale, Timestamp}; @@ -235,12 +237,9 @@ async fn lite05_timestamp_roundtrip(scheme: &str) { .expect("next_frame failed") .expect("group closed prematurely"); - let ts = frame_sub.timestamp.expect("timed frame"); - assert_eq!(ts.scale(), Timescale::MICRO); - assert_eq!(ts.value(), expected_us); - - // Drain the payload so the stream advances to the next frame. - let _ = frame_sub.read_all().await; + assert_eq!(frame_sub.timestamp, None, "a lite-05 subscriber drops timestamps"); + let payload = frame_sub.read_all().await.expect("failed to read frame"); + assert_eq!(payload, bytes::Bytes::from(format!("frame@{expected_us}"))); } drop(connection); @@ -258,7 +257,7 @@ async fn broadcast_moq_lite_05_timestamps_webtransport() { /// Lite05 FETCH round-trip: retrieve a past group by sequence without holding a /// subscription, exercising the bare-FRAME fetch response and per-frame timestamp -/// decoding on the fetch stream. The track is also `compress`-hinted, so the +/// decoding on the fetch stream, whose timestamps the subscriber then drops. The track is also `compress`-hinted, so the /// fetched frames are Deflate-compressed (matching TRACK_INFO) and inflated by the /// subscriber, exercising fetch/TRACK_INFO codec consistency. async fn lite05_fetch_roundtrip(scheme: &str) { @@ -353,9 +352,7 @@ async fn lite05_fetch_roundtrip(scheme: &str) { .expect("next_frame failed") .expect("group closed prematurely"); - let ts = frame_sub.timestamp.expect("timed frame"); - assert_eq!(ts.scale(), Timescale::MICRO); - assert_eq!(ts.value(), expected_us); + assert_eq!(frame_sub.timestamp, None, "a lite-05 subscriber drops timestamps"); let payload = frame_sub.read_all().await.expect("failed to read frame"); assert_eq!(payload, bytes::Bytes::from(format!("frame@{expected_us}"))); @@ -643,13 +640,11 @@ async fn broadcast_moq_lite_05_fetch_during_subscribe_webtransport() { } /// Lite05 can't declare an untimed track, so a track that sets no timescale goes out -/// in milliseconds, its frames stamped with their send time. The subscriber receives -/// `Some(ts)` at that scale. +/// in milliseconds, its frames stamped with their send time. The subscriber drops them, +/// so the track arrives untimed. #[tracing_test::traced_test] #[tokio::test] async fn broadcast_moq_lite_05_untimed_track() { - use moq_tokio::moq_net::Timescale; - let pub_origin = moq_tokio::origin::spawn(); let broadcast = pub_origin.create_broadcast("test").expect("create broadcast"); broadcast.announce(Default::default()).expect("create broadcast"); @@ -724,12 +719,8 @@ async fn broadcast_moq_lite_05_untimed_track() { .expect("next_frame failed") .expect("group closed"); - let ts = frame_sub.timestamp.expect("timed frame"); - assert_eq!( - ts.scale(), - Timescale::MILLI, - "an untimed track's send times are milliseconds" - ); + assert_eq!(frame_sub.timestamp, None, "the send time is dropped"); + assert_eq!(track_sub.info().timescale, None, "the track arrives untimed"); drop(connection); server_handle From 4f51debba92c5aaadbf1f5841d5ed965aeb7a0a2 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 23:49:56 -0700 Subject: [PATCH 08/14] docs(net): link the timestamp bound's migration to its PR Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/MIGRATE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rs/moq-net/MIGRATE.md b/rs/moq-net/MIGRATE.md index dcfcad40420..331eae07766 100644 --- a/rs/moq-net/MIGRATE.md +++ b/rs/moq-net/MIGRATE.md @@ -4,7 +4,7 @@ How to update your code for each breaking change, newest first. ## Unreleased -### Timestamps stop at 2^61 - 1 +### Timestamps stop at 2^61 - 1 (#6168) `Timestamp::new`, its `from_*` constructors, `convert`, `checked_add`, and `TryFrom` return `TimeOverflow` past `2^61 - 1` instead of `2^62 - 1`, the most lite-05/06 can carry as a group's first frame. A received timestamp past it fails the group as a malformed track. From 3f869eb05dfd6b4178506a531686e7cdc3f2d9e0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 09:37:46 -0700 Subject: [PATCH 09/14] quest: point arrival-stamps at moq-dev/moq#6150 for the completed provenance quest Co-Authored-By: Claude Opus 5.5 --- quest/m1/ts-reliability/arrival-stamps.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m1/ts-reliability/arrival-stamps.md b/quest/m1/ts-reliability/arrival-stamps.md index aca55f9235b..e16afc71181 100644 --- a/quest/m1/ts-reliability/arrival-stamps.md +++ b/quest/m1/ts-reliability/arrival-stamps.md @@ -10,7 +10,7 @@ path no longer lets held frames out past their deadline: the ignored arm `#[ignore]`. Non-goals: wall-clock or cross-hop provenance -([received timestamps are media time](/quest/m1/lite-time-provenance.md)), +([received timestamps are media time](https://github.com/moq-dev/moq/pull/6150)), and any wire change. ## Plan From 409c75c7521ccce4430732316368ae3c19c5d902 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 13:15:48 -0700 Subject: [PATCH 10/14] test: expect lite-05/06 subscribers to drop received timestamps The C++ probe's default lite-06 session and the wasm baseline and UniFFI cases on the lite-06 relay now expect untimed frames, as lite-02 already did. Co-Authored-By: Claude Opus 5.5 --- cpp/moq/test/probe.cpp | 3 ++- test/wasm/baseline/consumer.ts | 24 +++++++++--------------- test/wasm/generator.md | 2 +- test/wasm/uniffi/cases.ts | 6 +++--- 4 files changed, 15 insertions(+), 20 deletions(-) diff --git a/cpp/moq/test/probe.cpp b/cpp/moq/test/probe.cpp index 678b7118f3e..da67a836041 100644 --- a/cpp/moq/test/probe.cpp +++ b/cpp/moq/test/probe.cpp @@ -182,7 +182,8 @@ int main() { CHECK(!reading.valid()); CHECK(frame.has_value()); CHECK(frame->payload == bytes("hello")); - CHECK(frame->timestamp_us == 1000); + // The default moq-lite-06 session drops received timestamps, so the track arrives untimed. + CHECK(!frame->timestamp_us.has_value()); // A continuation receives the next frame on the executor. { diff --git a/test/wasm/baseline/consumer.ts b/test/wasm/baseline/consumer.ts index 0ff9b378e3d..1b30a668051 100644 --- a/test/wasm/baseline/consumer.ts +++ b/test/wasm/baseline/consumer.ts @@ -36,6 +36,8 @@ async function run(config: Sample) { let grew = false; let received = 0; const frames = config.publishers * config.subscribers * config.frames; + // Lite-02 carries no timestamps, and a lite-05/06 subscriber drops them. + const timed = !/^moq-lite-0[1-6]$/.test(config.version); const check = (bytes: Uint8Array, frame: number) => { first ??= performance.now() - start; verify(bytes, config.size, frame); @@ -90,12 +92,7 @@ async function run(config: Sample) { try { for (let frame = 0; frame < config.frames; frame++) { const got = await group.readFrame(); - if ( - !got || - (config.version === "moq-lite-02" - ? got.timestamp !== undefined - : got.timestamp?.asMillis() !== frame) - ) + if (!got || (timed ? got.timestamp?.asMillis() !== frame : got.timestamp !== undefined)) throw new Error(`incorrect JS frame timestamp ${frame}`); check(got.payload, frame); } @@ -178,9 +175,7 @@ async function run(config: Sample) { const got = await group.readFrame(); if ( !got || - (config.version === "moq-lite-02" - ? got.timestampUs !== undefined - : got.timestampUs !== BigInt(frame * 1000)) + (timed ? got.timestampUs !== BigInt(frame * 1000) : got.timestampUs !== undefined) ) throw new Error(`incorrect UniFFI frame timestamp ${frame}`); check(new Uint8Array(got.payload), frame); @@ -210,12 +205,11 @@ async function run(config: Sample) { }, metadata: { sequence: String(config.sequence), - timestamp: - config.version === "moq-lite-02" - ? "verified absent: this wire version carries none" - : config.backend === "wasm" - ? "unavailable: payload-only binding" - : "verified", + timestamp: !timed + ? "verified absent: this version delivers none" + : config.backend === "wasm" + ? "unavailable: payload-only binding" + : "verified", }, }; } finally { diff --git a/test/wasm/generator.md b/test/wasm/generator.md index 49b7523d757..09bf2338c47 100644 --- a/test/wasm/generator.md +++ b/test/wasm/generator.md @@ -179,7 +179,7 @@ three bindings. moq-wasm's column is the existing suite in [`src/main.ts`](src/m | reads a published broadcast, every byte, 128 KiB frames | yes | yes | yes | | refuses a missing track | yes | yes: rejects with `NotFound` (transport-19) or `Protocol` (lite) | yes: rejects with the slice's `Net { reason: "not found" }` | | publishes | no: consume only | yes | yes | -| frame timestamps | no: payload only | yes, exact, as `bigint` microseconds | yes | +| frame timestamps | no: payload only | yes, exact, as `bigint` microseconds on a version that delivers them (not lite-02 or lite-06) | yes | | u64 sequence above 2^53, end to end | not testable: no publisher can send it | yes, 2^53+1 on all three; 2^62-1 on lite-02 and transport-19 (see bugs) | same as moq-ffi | | cancellation through `AbortSignal` | no: `free()` rejects pending calls | rejects with `AbortError`; the next call still gets the next frame | UniFFI: `AbortError`; BoltFFI: `BoltFFICancelledError`; neither steals the next frame | | aborting releases the call's state | n/a | yes: a track's demand drops only once the aborted future is freed | not testable: the slice has no demand | diff --git a/test/wasm/uniffi/cases.ts b/test/wasm/uniffi/cases.ts index d6d0c5061ce..f50a476c159 100644 --- a/test/wasm/uniffi/cases.ts +++ b/test/wasm/uniffi/cases.ts @@ -267,9 +267,9 @@ const remote: Record Promise> = { const got = (await next) as MoqGroupConsumerLike | undefined; if (got?.sequence() !== sequence) throw new Error(`sequence ${got?.sequence()} != ${sequence}`); const frame = await got.readFrame(); - // moq-lite-02 carries no timestamp; every other version must keep it exact. - if (relay.version !== "moq-lite-02" && frame?.timestampUs !== sequence) - throw new Error(`timestamp ${frame?.timestampUs} != ${sequence}`); + // Lite-02 carries no timestamp and a lite-05/06 subscriber drops it; the rest keep it exact. + const expected = /^moq-lite-0[1-6]$/.test(relay.version) ? undefined : sequence; + if (frame?.timestampUs !== expected) throw new Error(`timestamp ${frame?.timestampUs} != ${expected}`); seen.push(String(sequence)); } return undefined; From 2c5a6eddf36a5b2401ebb6e6c935d8ffad3d79d3 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 13:28:40 -0700 Subject: [PATCH 11/14] quest: point resume-outage links at moq-dev/moq#6183 #6183 completed and deleted quest/m1/resume-outage.md while two Related lists still linked it, which fails quest check on main. Co-Authored-By: Claude Opus 5.5 --- quest/m1/js-start-parity.md | 2 +- quest/m2/lite-legacy-gap.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/quest/m1/js-start-parity.md b/quest/m1/js-start-parity.md index 72659b6a1f4..81719b80957 100644 --- a/quest/m1/js-start-parity.md +++ b/quest/m1/js-start-parity.md @@ -18,4 +18,4 @@ Public API: none. Wire: none, only the value JS sends. ## Related - [lite-05/06 gap judgement](/quest/m2/lite-legacy-gap.md) - the same misjudgement on versions whose START cannot say it -- [Resume through an outage](/quest/m1/resume-outage.md) - the resume work that found it +- [Resume through an outage](https://github.com/moq-dev/moq/pull/6183) - the resume work that found it diff --git a/quest/m2/lite-legacy-gap.md b/quest/m2/lite-legacy-gap.md index e06bfac683b..8d31fa76701 100644 --- a/quest/m2/lite-legacy-gap.md +++ b/quest/m2/lite-legacy-gap.md @@ -22,4 +22,4 @@ Public API: none. Wire: none. ## Related - [JS answers SUBSCRIBE_START like Rust](/quest/m1/js-start-parity.md) - the lite-07 publisher half -- [Resume through an outage](/quest/m1/resume-outage.md) - the lite-07 fix +- [Resume through an outage](https://github.com/moq-dev/moq/pull/6183) - the lite-07 fix From 4a58ec5211d40debe5de2babc7f89208c63404d1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 16:17:35 -0700 Subject: [PATCH 12/14] fix(lite): stamp an unchecked track with its send time on lite-05/06 An unchecked track is served untimed to moq-lite, which on lite-05/06 means each frame and datagram carries its send time. The publisher still wrote the frame's own timestamp, a possible moq-transport arrival time. Co-Authored-By: Claude Opus 5.5 --- doc/concept/moq-lite.md | 5 +- js/net/src/lite/publisher.test.ts | 35 ++++++++ js/net/src/lite/publisher.ts | 71 +++++++++-------- rs/moq-net/src/lite/publisher.rs | 127 +++++++++++++++++++----------- 4 files changed, 159 insertions(+), 79 deletions(-) diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index f8fe5594004..cbe0349ea5a 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -284,8 +284,9 @@ frames go out stamped with their send time; the work-in-progress 07 carries it untimed. A publisher refuses a group that starts before a lower one, and so does a -receiver. Since moq-lite 05 and 06 send times look like media time, a receiver -drops every timestamp it gets over them and the track arrives untimed. +moq-lite 07 receiver. Since moq-lite 05 and 06 send times look like media time, +a receiver drops every timestamp it gets over them and the track arrives +untimed. Timestamps received over moq-transport, whose peers may stamp arrival times, are not checked and are only passed on over moq-transport: a relay serves that track to moq-lite untimed. diff --git a/js/net/src/lite/publisher.test.ts b/js/net/src/lite/publisher.test.ts index b08b5d546bc..049c3ad8ab7 100644 --- a/js/net/src/lite/publisher.test.ts +++ b/js/net/src/lite/publisher.test.ts @@ -534,6 +534,41 @@ test("lite draft-05: the fetch response ranks the publisher's own writes", async } }); +// An unchecked track goes out untimed, so lite-06 carries its send time, never the frame's own +// timestamp, which may be a moq-transport arrival time. +test("lite draft-06: an unchecked track's fetched frames carry their send time", async () => { + const pair = createMockTransportPair(ALPN_06); + const origin = new OriginProducer(); + const publisher = new Publisher(pair.server, Version.DRAFT_06, randomHop(), origin.consume()); + + const broadcast = publish(origin, Path.from("test")); + const track = broadcast.createTrack("video", { timescale: Timescale.MICRO, unchecked: true }); + + const group = new GroupProducer(0); + group.writeFrame(textFrame("hello", Timestamp.fromMillis(36_000_000))); + group.close(); + track.writeGroup(group); + + const client = await Stream.open(pair.client, { version: Version.DRAFT_06 }); + const server = await Stream.accept(pair.server, Version.DRAFT_06); + if (!server) throw new Error("publisher never saw the fetch stream"); + + const msg = new Fetch({ broadcast: Path.from("test"), track: "video", priority: 0, group: 0 }); + try { + const before = Math.floor(Timestamp.now().value); + await publisher.runFetch(msg, server); + const after = Math.ceil(Timestamp.now().value); + + // The first frame's zigzag delta is against 0, in the milliseconds TRACK_INFO declares. + const sent = Number((await client.reader.u62()) >> 1n); + expect(sent).toBeGreaterThanOrEqual(before); + expect(sent).toBeLessThanOrEqual(after); + } finally { + publisher.close(); + client.close(); + } +}); + // How long a served-subscription test waits for the next group stream before calling the // publisher idle. Nothing waits this out: it only bounds the read when a group never comes. const IDLE_MS = 500; diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index bf2fcd1fb2c..552e64cce0b 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -65,12 +65,26 @@ function zigzag(delta: bigint): bigint { return delta >= 0n ? delta << 1n : (-delta << 1n) - 1n; } -/** - * A frame or datagram timestamp as its raw value at the wire `timescale`. Lite-05/06 cannot - * encode an untimed track, so its payloads carry their send time instead. - */ -function wireTime(timestamp: Timestamp | undefined, timescale: Timescale): number { - return Math.round((timestamp ?? Timestamp.now()).as(timescale)); +/** How a lite hop stamps a track's frames and datagrams. */ +interface Stamp { + /** The wire timescale TRACK_INFO declared (see `wireTimescale`). */ + timescale: Timescale; + /** + * Whether each frame's own timestamp goes out. Lite-05/06 cannot encode an untimed track, so + * one served untimed carries its send time instead, unchecked ones included. + */ + timed: boolean; +} + +/** How `version` stamps a track served at `timescale` (see `liteTimescale`), or undefined when it carries none. */ +function stamping(version: Version, timescale: Timescale | undefined): Stamp | undefined { + const wire = wireTimescale(version, timescale); + return wire === undefined ? undefined : { timescale: wire, timed: timescale !== undefined }; +} + +/** A frame or datagram's `timestamp` as its raw wire value, or its send time when the track goes out untimed. */ +function wireTime(timestamp: Timestamp | undefined, stamp: Stamp): number { + return Math.round(((stamp.timed ? timestamp : undefined) ?? Timestamp.now()).as(stamp.timescale)); } /** @@ -100,8 +114,8 @@ interface RunGroup { /** The group to serve. */ group: group.Consumer; - /** The track's wire timescale, applied to every frame timestamp, or undefined to send none. */ - timescale: Timescale | undefined; + /** How every frame is stamped, or undefined to send no timestamps. */ + stamp: Stamp | undefined; /** The subscription's ranking, which this stream joins for as long as it runs. */ priority: Priority; @@ -156,8 +170,8 @@ function frameRange(bounds: FrameBounds, sequence: number): { start: number; end type ServeGroup = { /** The Subscribe ID the GROUP message references. */ sub: bigint; - /** The track's wire timescale, which every frame timestamp is converted to, or undefined to send none. */ - timescale: Timescale | undefined; + /** How every frame is stamped, or undefined to send no timestamps. */ + stamp: Stamp | undefined; /** First frame to send; anything below it was excluded by the subscription. */ start: number; /** Last frame to send (inclusive), or undefined for the rest of the group. */ @@ -855,7 +869,7 @@ export class Publisher { try { // Older drafts carry no timestamps. - let timescale: Timescale | undefined; + let stamp: Stamp | undefined; if (hasTrackStream(this.version)) { // Lite-05+ accepts implicitly: no SUBSCRIBE_OK (the immutable @@ -868,7 +882,7 @@ export class Publisher { // (accept never called, track closed) as an error here, which resets the // stream. const info = await untilAborted(track.info(), left.signal); - timescale = wireTimescale(this.version, liteTimescale(info)); + stamp = stamping(this.version, liteTimescale(info)); } else { // Older drafts acknowledge with SUBSCRIBE_OK and stream frames verbatim. const ok = new SubscribeOk({ @@ -885,7 +899,7 @@ export class Publisher { // Serve datagrams concurrently with groups whenever the transport carries them // (the writer exists iff so). No group fallback: otherwise they simply aren't sent. if (this.#datagramWriter) { - datagrams = this.#runDatagrams(msg.id, track, timescale, served); + datagrams = this.#runDatagrams(msg.id, track, stamp, served); } controls = new SubscriptionControls({ @@ -914,7 +928,7 @@ export class Publisher { const finished = await this.#runTrack(track, stream.writer, controls, { sub: msg.id, broadcast: msg.broadcast, - timescale, + stamp, served, bounds: { startGroup: msg.startGroup, @@ -1043,7 +1057,7 @@ export class Publisher { return; } await this.#runFetchGroup(group, stream.writer, { - timescale: wireTimescale(this.version, info.timescale), + stamp: stamping(this.version, info.timescale), start: msg.startFrame, end: msg.endFrame, }); @@ -1082,12 +1096,12 @@ export class Publisher { serving: { sub: bigint; broadcast: Path.Valid; - timescale: Timescale | undefined; + stamp: Stamp | undefined; served: Run[] | undefined; bounds: FrameBounds; }, ): Promise { - const { sub, broadcast, timescale, served, bounds } = serving; + const { sub, broadcast, stamp, served, bounds } = serving; // Lite-05+ resolves the range on the subscribe stream: SUBSCRIBE_START once the // first group is known, SUBSCRIBE_END when the track finishes. const emitRange = hasTrackStream(this.version); @@ -1327,7 +1341,7 @@ export class Publisher { const options: RunGroup = { sub, group, - timescale, + stamp, priority, unsubscribed, start: range.start, @@ -1485,12 +1499,7 @@ export class Publisher { * * @internal */ - async #runDatagrams( - sub: bigint, - track: track.Subscriber, - timescale: Timescale | undefined, - served: Run[] | undefined, - ) { + async #runDatagrams(sub: bigint, track: track.Subscriber, stamp: Stamp | undefined, served: Run[] | undefined) { const writer = this.#datagramWriter; if (!writer) return; // Only reached with a writer (see the #datagramWriter gate). const maxSize = DatagramStream.maxDatagramSize(this.#quic); @@ -1501,7 +1510,7 @@ export class Publisher { if (!datagram) return; // Track finished; #runTrack tears the subscription down. // Convert the timestamp to the track's advertised timescale, matching #serveGroup. - const ts = timescale === undefined ? undefined : wireTime(datagram.timestamp, timescale); + const ts = stamp === undefined ? undefined : wireTime(datagram.timestamp, stamp); const body = new DatagramMessage(sub, datagram.sequence, ts, datagram.payload).encode(this.version); // No group fallback: drop anything that doesn't fit a single datagram. @@ -1526,7 +1535,7 @@ export class Publisher { async #runFetchGroup( group: group.Consumer, stream: Writer, - { timescale, start: startFrame, end: endFrame }: ServeFetch, + { stamp, start: startFrame, end: endFrame }: ServeFetch, ) { // The response carries no header, so the receiver numbers the first frame it gets // as `startFrame`. Skipping the head here is the only thing keeping those numbers @@ -1542,8 +1551,8 @@ export class Publisher { const frame = await race([group.readFrame(), stream.closed]); if (!frame) break; - if (timescale !== undefined) { - const ts = BigInt(wireTime(frame.timestamp, timescale)); + if (stamp !== undefined) { + const ts = BigInt(wireTime(frame.timestamp, stamp)); await stream.u62(zigzag(ts - prevTs)); prevTs = ts; } @@ -1594,7 +1603,7 @@ export class Publisher { * @internal */ async #serveGroup(stream: Writer, options: RunGroup) { - const { sub, group, timescale, priority, start: startFrame, end: endFrame } = options; + const { sub, group, stamp, priority, start: startFrame, end: endFrame } = options; // This model holds whole groups, so frame `startFrame` is always reachable unless // the group ends first. Declaring it up front keeps the stream self-describing. const msg = new GroupMessage({ subscribe: sub, sequence: group.sequence, frameStart: startFrame }); @@ -1634,9 +1643,9 @@ export class Publisher { if (read.sequence < startFrame) continue; if (endFrame !== undefined && read.sequence > endFrame) break; - if (timescale !== undefined) { + if (stamp !== undefined) { // Convert each frame to the track's wire timescale. - const ts = BigInt(wireTime(read.frame.timestamp, timescale)); + const ts = BigInt(wireTime(read.frame.timestamp, stamp)); await hooks.guardGroup(group, () => stream.u62(zigzag(ts - prevTs))); prevTs = ts; } diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index c744e5db242..12bd73ba344 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -2127,7 +2127,7 @@ impl Request for SubscribeServe { // `None` omits the per-frame timestamp prefix: an untimed track on // Lite07+, or any track before Lite05 (whose frames still carry // timestamps in the model, just not on this wire). - let timescale = shared.version.wire_timescale(track.info().lite_timescale()); + let stamp = Stamp::new(shared.version, track.info().lite_timescale()); // Lite05+ accepts implicitly: no SUBSCRIBE_OK, the immutable properties // live in TRACK_INFO, and the resolved range arrives as @@ -2164,7 +2164,7 @@ impl Request for SubscribeServe { steer: steer_tx.consume(), steer_seen: steer, version: shared.version, - timescale, + stamp, runtime: shared.runtime.clone(), outcomes: Default::default(), ietf, @@ -2224,7 +2224,7 @@ enum FetchServe { /// (the subscriber decodes against the same baseline). Serve { group: group::Consumer, - timescale: Option, + stamp: Option, prev_ts: u64, frame: Option, chunk: Option, @@ -2301,17 +2301,18 @@ impl Request for FetchServe { group.end_at(msg.end_frame.map_or(Bound::Unbounded, Bound::Included)); // Matches the timescale TRACK_INFO declared for the track. - let timescale = shared.version.wire_timescale(group.lite_timescale()); + let stamp = Stamp::new(shared.version, group.lite_timescale()); // And its compression, which a lite-07 peer receives as stored. if shared.version.has_compression() { group = group.compressed(); } let capped = msg.end_frame.is_some(); + let timescale = stamp.map(|stamp| stamp.timescale); let records = lite::ietf::Records::start(writer, ietf, &group, capped, timescale, shared.version)?; *self = Self::Serve { group, - timescale, + stamp, prev_ts: 0, frame: None, chunk: None, @@ -2322,7 +2323,7 @@ impl Request for FetchServe { } Self::Serve { group, - timescale, + stamp, prev_ts, frame, chunk, @@ -2354,7 +2355,7 @@ impl Request for FetchServe { batched.frame.timestamp, batched.inflated, batched.frame.payload.len() as u64, - *timescale, + *stamp, prev_ts, &shared.runtime, )?; @@ -2389,7 +2390,7 @@ impl Request for FetchServe { next.timestamp, next.inflated, next.size, - *timescale, + *stamp, prev_ts, &shared.runtime, )?; @@ -3347,12 +3348,12 @@ fn buffer_frame_info( timestamp: Option, inflated: Option, size: u64, - timescale: Option, + stamp: Option, prev_ts: &mut u64, runtime: &crate::time::Deadlines, ) -> Result<(), Error> { - if let Some(timescale) = timescale { - buffer_zigzag_delta(writer, wire_timestamp(timestamp, timescale, runtime)?, prev_ts)?; + if let Some(stamp) = stamp { + buffer_zigzag_delta(writer, stamp.wire(timestamp, runtime)?, prev_ts)?; } // Only a compressed group read in its stored form has one, and only on lite-07. if let Some(inflated) = inflated { @@ -3362,21 +3363,36 @@ fn buffer_frame_info( Ok(()) } -/// A frame or datagram timestamp as its raw value at the wire `timescale`. -/// -/// Lite05/06 cannot encode an untimed track, so its payloads carry their send time on -/// `runtime` instead, at the millisecond scale its TRACK_INFO declares. An unchecked track -/// declares milliseconds there too, so its timestamps are converted to it. -fn wire_timestamp( - timestamp: Option, +/// How a lite hop stamps a track's frames and datagrams. +#[derive(Clone, Copy, Debug)] +struct Stamp { + /// The wire timescale TRACK_INFO declared (see [`Version::wire_timescale`]). timescale: crate::Timescale, - runtime: &crate::time::Deadlines, -) -> Result { - timestamp - .unwrap_or_else(|| crate::Timestamp::from(runtime.now())) - .convert(timescale) - .map(|time| time.value()) - .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded)) + /// Whether each frame's own timestamp goes out. Lite05/06 cannot encode an untimed + /// track, so one served untimed carries its send time instead, unchecked ones included. + timed: bool, +} + +impl Stamp { + /// How `version` stamps a track served at `timescale` (see + /// [`track::Info::lite_timescale`]), or `None` when it carries no timestamps. + fn new(version: Version, timescale: Option) -> Option { + Some(Self { + timescale: version.wire_timescale(timescale)?, + timed: timescale.is_some(), + }) + } + + /// A frame or datagram's `timestamp` as its raw wire value, or its send time on + /// `runtime` when the track goes out untimed. + fn wire(self, timestamp: Option, runtime: &crate::time::Deadlines) -> Result { + timestamp + .filter(|_| self.timed) + .unwrap_or_else(|| crate::Timestamp::from(runtime.now())) + .convert(self.timescale) + .map(|time| time.value()) + .map_err(|_| Error::BoundsExceeded(crate::coding::BoundsExceeded)) + } } /// Buffer `curr` as a zigzag-mapped varint delta against `*prev`, then advance @@ -3599,9 +3615,8 @@ struct Subscription { /// Last steer observed by this clone, so a change only fires once. steer_seen: Steer, version: Version, - /// The track's wire timescale, or `None` when its frames and datagrams carry no - /// timestamp (see [`Version::wire_timescale`]). - timescale: Option, + /// How frames and datagrams are stamped, or `None` when they carry no timestamp. + stamp: Option, /// The clock an untimed track's Lite05/06 send times are read from. runtime: crate::time::Deadlines, /// Outcomes shared by the subscription and every group it serves. @@ -3637,9 +3652,9 @@ impl Subscription { return false; } let Ok(timestamp) = self - .timescale + .stamp .filter(|_| !status) - .map(|timescale| wire_timestamp(datagram.timestamp, timescale, &self.runtime)) + .map(|stamp| stamp.wire(datagram.timestamp, &self.runtime)) .transpose() else { return false; @@ -3652,7 +3667,7 @@ impl Subscription { if wire .properties .sealed_time - .is_some_and(|sealed| self.timescale != Some(sealed.scale())) => + .is_some_and(|sealed| self.stamp.map(|stamp| stamp.timescale) != Some(sealed.scale())) => { return false; } @@ -3669,7 +3684,7 @@ impl Subscription { payload: payload.clone(), ietf, }; - // has_datagrams is checked before this runs, and the timescale is set wherever the + // has_datagrams is checked before this runs, and the stamp is set wherever the // version needs a timestamp, so encoding never hits a version guard. let Ok(body) = body.encode_bytes(self.version) else { return false; @@ -4366,7 +4381,7 @@ impl GroupServe { self.ctx.ietf, &self.group, self.capped, - self.ctx.timescale, + self.ctx.stamp.map(|stamp| stamp.timescale), self.ctx.version, ) }); @@ -4505,7 +4520,7 @@ impl GroupServe { batched.frame.timestamp, batched.inflated, batched.frame.payload.len() as u64, - self.ctx.timescale, + self.ctx.stamp, &mut self.prev_ts, &self.ctx.runtime, ) @@ -4551,7 +4566,7 @@ impl GroupServe { next.timestamp, next.inflated, next.size, - self.ctx.timescale, + self.ctx.stamp, &mut self.prev_ts, &self.ctx.runtime, ) @@ -4775,6 +4790,26 @@ mod serve_group_test { } } + /// A track served untimed to lite-05/06 carries its send time, never its frames' own + /// timestamps: those of an unchecked track may be moq-transport arrival times. + #[moq_clock::test] + async fn an_unchecked_track_goes_out_with_send_times() { + let runtime = crate::time::sim(); + let stamped = Timestamp::from_millis(36_000_000).unwrap(); + let sent = Timestamp::from(runtime.now()).value(); + let checked = track::Info::timed(); + let unchecked = track::Info { + unchecked: true, + ..track::Info::timed() + }; + + let stamp = Stamp::new(Version::Lite06, checked.lite_timescale()).unwrap(); + assert_eq!(stamp.wire(Some(stamped), &runtime).unwrap(), stamped.value()); + let stamp = Stamp::new(Version::Lite06, unchecked.lite_timescale()).unwrap(); + assert_eq!(stamp.wire(Some(stamped), &runtime).unwrap(), sent); + assert!(Stamp::new(Version::Lite07, unchecked.lite_timescale()).is_none()); + } + #[moq_clock::test] async fn resets_with_the_abort_code() { let log = Log::default(); @@ -4790,7 +4825,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -4836,7 +4871,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -4886,7 +4921,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -4936,7 +4971,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -5004,7 +5039,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -5177,7 +5212,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -5252,7 +5287,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -5301,7 +5336,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -5349,7 +5384,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite06, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -5409,7 +5444,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite06, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: Default::default(), ietf: false, @@ -6541,7 +6576,7 @@ mod serve_group_test { steer: steer.consume(), steer_seen: Steer::default(), version: Version::Lite07, - timescale: Some(crate::Timescale::MILLI), + stamp: Stamp::new(Version::Lite07, Some(crate::Timescale::MILLI)), runtime: crate::time::sim(), outcomes: outcomes.clone(), ietf: false, From dfed0f2b49618288057e702b5e4be6056e8d8bf7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 02:43:54 -0700 Subject: [PATCH 13/14] docs: note the 2^61 - 1 timestamp bound in each binding's MIGRATE The bound reaches every binding through Timestamp::from_micros, so each wrapper's guide names its TimeOverflow error and the clamp. Co-Authored-By: Claude Opus 5.5 --- cpp/moq/MIGRATE.md | 6 ++++++ dart/moq/MIGRATE.md | 6 ++++++ go/wrapper/MIGRATE.md | 6 ++++++ kt/moq/MIGRATE.md | 6 ++++++ py/moq-rs/MIGRATE.md | 6 ++++++ rs/moq-c/MIGRATE.md | 6 ++++++ rs/moq-ffi/MIGRATE.md | 6 ++++++ swift/MIGRATE.md | 6 ++++++ 8 files changed, 48 insertions(+) diff --git a/cpp/moq/MIGRATE.md b/cpp/moq/MIGRATE.md index ec0b7243cd4..3c5970f596d 100644 --- a/cpp/moq/MIGRATE.md +++ b/cpp/moq/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Timestamps stop at 2^61 - 1 microseconds (#6168) + +Writing a frame, or any other call that takes a timestamp, fails with `TimeOverflow` past `2^61 - 1` microseconds instead of `2^62 - 1`, the most moq-lite-05/06 can carry. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `2^61 - 1`. + ### Finished tracks stay until the broadcast ends (#6157) A finished track now stays listed, with its catalog rendition and demand, until its broadcast closes or the new `BroadcastProducer::finish` lets go. diff --git a/dart/moq/MIGRATE.md b/dart/moq/MIGRATE.md index 7a1f1be6ca3..f80a05b220e 100644 --- a/dart/moq/MIGRATE.md +++ b/dart/moq/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Timestamps stop at 2^61 - 1 microseconds (#6168) + +Writing a frame, or any other call that takes a timestamp, fails with `TimeOverflow` past `2^61 - 1` microseconds instead of `2^62 - 1`, the most moq-lite-05/06 can carry. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `2^61 - 1`. + ### Finished tracks stay until the broadcast ends (#6157) A finished track now stays listed, with its catalog rendition and demand, until its broadcast closes or the new `BroadcastProducer.finish()` lets go. diff --git a/go/wrapper/MIGRATE.md b/go/wrapper/MIGRATE.md index b5e60cf4cc7..53badb5b500 100644 --- a/go/wrapper/MIGRATE.md +++ b/go/wrapper/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Timestamps stop at 2^61 - 1 microseconds (#6168) + +Writing a frame, or any other call that takes a timestamp, fails with `ErrTimeOverflow` past `2^61 - 1` microseconds instead of `2^62 - 1`, the most moq-lite-05/06 can carry. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `2^61 - 1`. + ### Finished tracks stay until the broadcast ends (#6157) A finished track now stays listed, with its catalog rendition and demand, until its broadcast closes or the new `BroadcastProducer.Finish()` lets go. diff --git a/kt/moq/MIGRATE.md b/kt/moq/MIGRATE.md index 2ff652f9212..2e5be0e88fd 100644 --- a/kt/moq/MIGRATE.md +++ b/kt/moq/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Timestamps stop at 2^61 - 1 microseconds (#6168) + +Writing a frame, or any other call that takes a timestamp, fails with `TimeOverflow` past `2^61 - 1` microseconds instead of `2^62 - 1`, the most moq-lite-05/06 can carry. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `2^61 - 1`. + ### Finished tracks stay until the broadcast ends (#6157) A finished track now stays listed, with its catalog rendition and demand, until its broadcast ends or the new `BroadcastProducer.finish()` lets go. diff --git a/py/moq-rs/MIGRATE.md b/py/moq-rs/MIGRATE.md index 4175ecd3ec6..c63550cad70 100644 --- a/py/moq-rs/MIGRATE.md +++ b/py/moq-rs/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Timestamps stop at 2^61 - 1 microseconds (#6168) + +Writing a frame, or any other call that takes a timestamp, fails with `TimeOverflow` past `2^61 - 1` microseconds instead of `2^62 - 1`, the most moq-lite-05/06 can carry. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `2^61 - 1`. + ### Finished tracks stay until the broadcast ends (#6157) A finished track now stays listed, with its catalog rendition and demand, until its broadcast closes or the new `BroadcastProducer.finish()` lets go. diff --git a/rs/moq-c/MIGRATE.md b/rs/moq-c/MIGRATE.md index 45a2f272283..af8522c9bb4 100644 --- a/rs/moq-c/MIGRATE.md +++ b/rs/moq-c/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Timestamps stop at 2^61 - 1 microseconds (#6168) + +Writing a frame, or any other call that takes a timestamp, fails with `MOQ_ERROR_TIMESTAMP_OVERFLOW` past `2^61 - 1` microseconds instead of `2^62 - 1`, the most moq-lite-05/06 can carry. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `2^61 - 1`. + ### Finished tracks stay until the broadcast ends (#6157) `moq_publish_track_finish`, `moq_publish_media_finish`, `moq_publish_container_finish`, and the encoder finishes now release the handle while the broadcast keeps the finished track listed, with its catalog rendition and demand, until `moq_publish_close` or the new `moq_publish_finish_close`. diff --git a/rs/moq-ffi/MIGRATE.md b/rs/moq-ffi/MIGRATE.md index 59e13b01410..af10d4c92ab 100644 --- a/rs/moq-ffi/MIGRATE.md +++ b/rs/moq-ffi/MIGRATE.md @@ -5,6 +5,12 @@ These are the UniFFI names; each language wrapper has its own `MIGRATE.md` in it ## Unreleased +### Timestamps stop at 2^61 - 1 microseconds (#6168) + +Every `timestamp_us` input, such as `MoqTrackProducer::write_frame` or `MoqAnchor::Time`, fails with `MoqError::TimeOverflow` past `2^61 - 1` instead of `2^62 - 1`, the most moq-lite-05/06 can carry. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `2^61 - 1`. + ### Finished tracks stay until the broadcast ends (#6157) A finished track, raw or media, now stays listed, with its catalog rendition and demand, until its broadcast closes or the new `MoqBroadcastProducer::finish` lets go, instead of unlisting when its producer finishes or is released. diff --git a/swift/MIGRATE.md b/swift/MIGRATE.md index 044aa7fa273..91c9538bad0 100644 --- a/swift/MIGRATE.md +++ b/swift/MIGRATE.md @@ -4,6 +4,12 @@ How to update your code for each breaking change, newest first. ## Unreleased +### Timestamps stop at 2^61 - 1 microseconds (#6168) + +Writing a frame, or any other call that takes a timestamp, fails with `TimeOverflow` past `2^61 - 1` microseconds instead of `2^62 - 1`, the most moq-lite-05/06 can carry. +A received timestamp past it fails the group as a malformed track. +Clamp timestamps you build from unbounded input to `2^61 - 1`. + ### Finished tracks stay until the broadcast ends (#6157) A finished track now stays listed, with its catalog rendition and demand, until its broadcast closes or the new `BroadcastProducer.finish()` lets go. From aa850a4c12f70e2e53cf7d00b94b55bad4372305 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 14:18:08 -0700 Subject: [PATCH 14/14] quest: drop lite07-finalize's wait on the timestamp bound This PR settles the lite-06 max sequence quest, so lite-07 no longer waits on it. Co-Authored-By: Claude Opus 5.5 --- quest/m1/lite07-finalize.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index 3ec28c55922..5ad27635a31 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -86,10 +86,9 @@ identifier. This release adopts the new routing contract end to end. Public API: the lite-07 version constant and ALPN lose `-wip`. Wire: lite-07 is published; older versions are unchanged. -Decided 2026-10-22 (maintainer): lite-07 also waits for the lite-06 -max-sequence quest, since it may change what the final wire carries. Timestamp provenance landed in -[#6150](https://github.com/moq-dev/moq/pull/6150). +[#6150](https://github.com/moq-dev/moq/pull/6150), and the 2^61 - 1 +timestamp bound in [#6168](https://github.com/moq-dev/moq/pull/6168). ## Required @@ -99,8 +98,6 @@ Timestamp provenance landed in - [Drop set](/quest/m1/drop-set.md) - lite-07's SUBSCRIBE_DROP carries `Max Age`, so a widening re-offers groups reported dropped - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - SUBSCRIBE carries ranges and an order and lite FETCH is gone, in Rust and JS - [Announcement takeover](/quest/m1/announce-takeover.md) - ANNOUNCE_TAKEOVER preserves a proven instance under a new independent route -- [lite-06 max sequence](/quest/m1/lite06-max-sequence.md) - the refusal boundary for an uncarriable sequence is settled before lite-07 freezes - ## Related - [The moq.pro mesh runs lite-07](/quest/m3/lite07-mesh.md) - the deployment condition that follows this