From 1a5620bff394d8538eff3b73f397b6cb1f17caf4 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 22:02:01 -0700 Subject: [PATCH 01/14] quest: claim quest/m1/archive/tracks-js From d6d8001724415838db043c15e935eba9d829b29a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 22:48:37 -0700 Subject: [PATCH 02/14] feat(js/net): list a broadcast's tracks over lite-07 TRACKS broadcast.Consumer.tracks() reports each track with its info, one synced, then adds and removals until the broadcast ends, mirroring Rust's broadcast::Consumer::tracks. A session opens one TRACKS stream per consumed broadcast while it has readers and answers a peer's TRACKS for a broadcast it publishes with the tracks its producer created. Older versions reset the stream type, a covering prefix route answers NOT_FOUND, and listed tracks count toward the announce cap. Adds Rust/JS interop for the Tracks Stream in both directions and retires the tracks-js quest. Co-Authored-By: Claude Opus 5.5 --- doc/concept/moq-lite.md | 4 +- doc/lib/js/net.md | 1 + js/net/bench/broadcasts.ts | 30 ++ js/net/src/broadcast.ts | 46 +++- js/net/src/listing.test.ts | 326 ++++++++++++++++++++++ js/net/src/listing.ts | 439 ++++++++++++++++++++++++++++++ js/net/src/lite/connection.ts | 6 + js/net/src/lite/index.ts | 1 + js/net/src/lite/malformed.test.ts | 4 +- js/net/src/lite/publisher.ts | 96 ++++++- js/net/src/lite/stream.ts | 1 + js/net/src/lite/subscriber.ts | 110 +++++++- js/net/src/lite/tracks.test.ts | 415 ++++++++++++++++++++++++++++ js/net/src/lite/tracks.ts | 100 +++++++ js/net/src/lite/version.ts | 19 ++ js/net/src/origin.ts | 11 + js/net/src/resume.ts | 108 ++++++++ js/net/src/util/limits.test.ts | 23 ++ js/net/src/util/limits.ts | 44 ++- js/net/src/wire.ts | 11 + quest/m1/archive/README.md | 5 +- quest/m1/archive/tracks-js.md | 38 --- quest/m1/lite07-finalize.md | 1 - rs/moq-net/src/test_interop.rs | 60 ++++ test/interop/README.md | 6 + test/interop/tracks.ts | 63 +++++ test/justfile | 5 +- 27 files changed, 1892 insertions(+), 81 deletions(-) create mode 100644 js/net/src/listing.test.ts create mode 100644 js/net/src/listing.ts create mode 100644 js/net/src/lite/tracks.test.ts create mode 100644 js/net/src/lite/tracks.ts delete mode 100644 quest/m1/archive/tracks-js.md create mode 100644 test/interop/tracks.ts diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index d7a9f893ca2..4cb8089adfe 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -56,8 +56,8 @@ always a prefix. A service that serves only some of the paths beneath it refuses the rest as they are requested. Announcements are hints; the request is what decides. -Inside a broadcast, a reader can list its tracks (moq-lite-07, Rust only so -far): each track with its fixed properties, a synced mark, then tracks as they +Inside a broadcast, a reader can list its tracks (moq-lite-07): each track +with its fixed properties, a synced mark, then tracks as they come and go. Listing is discovery, not delivery: subscribe to each track you want, and its groups still cached upstream reach you late. A broadcast lists the tracks its publisher creates, never one served only on request, and a path diff --git a/doc/lib/js/net.md b/doc/lib/js/net.md index b0c833901ec..9acc30c3aa0 100644 --- a/doc/lib/js/net.md +++ b/doc/lib/js/net.md @@ -41,6 +41,7 @@ broadcast.announce(); - **Connections share by URL and versions.** `new Connection({ url })` pools and reconnects with backoff, which the elements use. An ordered `versions` list, spelled like `moq-lite-06` as in Rust, applies to both WebTransport and WebSocket; equivalent lists share, while a different order or an omitted list gets its own pool entry. Supplying your own transport options or origin selects a private loop. Receive estimates are on demand: observe `connection.probe` reactively or subscribe while you need them; `peek()` alone opens no PROBE stream. - **GOAWAY moves the session.** The connection dials the replacement at once while the old session keeps serving its groups, up to `goaway.handover` (default 10s, or the relay's deadline when that is sooner). New requests keep opening on the old session until the replacement's route outranks it, which is a restart without an epoch: an unpinned announced request follows the replacement, while a blind or epoch-pinned request stays on its accepted instance. With an epoch, which travels only when `moq-lite-07-wip` is offered, the replacement serves the same broadcast, so open subscriptions move across without missing a group. An empty GOAWAY redials the same URL. A redirect stays on the same host unless `goaway.redirect` is `"follow"`. - **One send estimate per connection.** `Bandwidth.Allocator` divides it by track priority, max-min fair within a tier. An idle track claims nothing. Publishers reserve against it so their targets sum to the estimate. +- **List a broadcast's tracks.** `for await (const event of broadcast.tracks())` reports each track the broadcast lists (`added`, with its `Track.Info`), one `synced`, then `added` and `removed` as tracks come and go, until the broadcast ends; leaving the loop stops listing. Over a session it needs `moq-lite-07-wip`, syncs at once with nothing, and the peer's tracks follow as `added`. A track served only on request is never listed. See [discovery](/concept/moq-lite#discovery). - **One track per name.** Concurrent subscriptions share one request and producer. `createTrack` on a name with a request still queued answers it. The application owns sequence numbers: a track that replaces an ended one, created or requested, throws on an append that would reuse a sequence the name already wrote, so it writes its first group with an explicit sequence. Only a new broadcast starts over. - **Timing is per track, with no default.** A track that declares a `timescale` carries a `timestamp` on every frame; one without is [untimed](/concept/moq-lite#subscriptions) and its frames carry none. Omitting it means untimed, as in Rust. Nothing stamps a frame for you: the publisher passes the timestamp, and the `Timed` type (`{ value, at }`) carries one into the `@moq/json` and `@moq/flate` producers. - **Subscriber staleness is `maxDelay`.** It counts from a group's successor, by media time and by wall clock, whichever reaches it first. Passing the old `maxAge` key throws a `TypeError` naming `maxDelay`. Publisher retention is the separate `Track.Info.maxAge`. diff --git a/js/net/bench/broadcasts.ts b/js/net/bench/broadcasts.ts index 39d1b154c0c..0dd6c0fe315 100644 --- a/js/net/bench/broadcasts.ts +++ b/js/net/bench/broadcasts.ts @@ -102,6 +102,36 @@ for (const trackCount of [8, 32, 128]) { } } +// One created and removed track reaches every listing, however many tracks the broadcast holds. +const listed = 256; +console.log("tracks,listings,create_remove_and_read_us"); +for (const trackCount of [8, 128, 1024]) { + for (const listingCount of [1, 8, 32]) { + const broadcast = new BroadcastProducer(); + for (let index = 0; index < trackCount; index++) broadcast.createTrack(`held-${index}`); + const listings = Array.from({ length: listingCount }, () => broadcast.consume().tracks()); + // Past each listing's initial set and SYNCED. + for (const listing of listings) { + for (let index = 0; index <= trackCount; index++) await listing.next(); + } + const start = performance.now(); + for (let index = 0; index < listed; index++) { + broadcast.createTrack(`new-${index}`); + for (const listing of listings) { + const event = await listing.next(); + if (event?.kind !== "added" || event.name !== `new-${index}`) throw new Error("listing missed a track"); + } + broadcast.removeTrack(`new-${index}`); + for (const listing of listings) { + if ((await listing.next())?.kind !== "removed") throw new Error("listing missed a removal"); + checksum++; + } + } + console.log(`${trackCount},${listingCount},${(((performance.now() - start) * 1000) / listed).toFixed(1)}`); + broadcast.close(); + } +} + // Resolve one touched path while both the table and that path's readers grow. console.log("request,routes,readers,update_and_read_us"); const epoch = Epoch.parse("01900000-0000-7000-8000-000000000001"); diff --git a/js/net/src/broadcast.ts b/js/net/src/broadcast.ts index 3dab4121a7b..c102b1a8ee0 100644 --- a/js/net/src/broadcast.ts +++ b/js/net/src/broadcast.ts @@ -9,11 +9,14 @@ import { NotFound } from "./error.ts"; import type { Consumer as GroupConsumer } from "./group.ts"; import { Route } from "./hop.ts"; import { hooks, TrackSequences } from "./internal.ts"; +import { Lister, Listing, listTracks, type Tracks } from "./listing.ts"; import * as Path from "./path.ts"; import * as track from "./track.ts"; import { untilAborted } from "./util/abort.ts"; import { registerWire, trackOf } from "./wire.ts"; +export { Tracks, type TracksEvent } from "./listing.ts"; + /** The origin callback a created broadcast uses to advertise its exact path. @internal */ export interface Announcer { /** Advertise or re-price this broadcast's path. */ @@ -47,6 +50,8 @@ class BroadcastState { requested = new Signal([]); pending = new Set(); closed = new Once(); + // The tracks the broadcast lists, and the readers listing them. + listing = new Listing(this.closed); // The tracks the application inserted. A finished one stays and keeps serving its cache; only an // abort or removeTrack drops it, the way only a dropped producer leaves a Rust broadcast. tracks = new Map(); @@ -140,6 +145,7 @@ async function requested(state: BroadcastState): Promise { @@ -296,18 +302,19 @@ function takeQueued(state: BroadcastState, name: string): track.Request | undefi return queued; } -// Serve `producer` as the broadcast's track for its name. -function adopt(state: BroadcastState, producer: track.Producer): void { +// Serve `producer` as the broadcast's track for its name, listed with `info`. +function adopt(state: BroadcastState, producer: track.Producer, info: track.Info): void { const lookups = state.lookups.get(producer) ?? new Lookups(); state.lookups.set(producer, lookups); watchDemand(state, producer, lookups); state.tracks.set(producer.name, producer); + state.listing.listOwn(producer.name, producer, info); // A finished track keeps serving its cache, so only an abort evicts it. void producer.closed.then((closed) => { - if (closed instanceof Error && state.tracks.get(producer.name) === producer) { - state.tracks.delete(producer.name); - } + if (!(closed instanceof Error)) return; + if (state.tracks.get(producer.name) === producer) state.tracks.delete(producer.name); + state.listing.unlistOwn(producer.name, producer); }); } @@ -359,6 +366,8 @@ export class Producer { largest: (name) => largest(this.#state, name), fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), requested: () => requested(this.#state), + tracks: (owner) => listTracks(this.#state.listing, owner), + lister: () => new Lister(this.#state.listing), }); } @@ -411,7 +420,7 @@ export class Producer { const lookups = this.#state.lookups.get(producer); if (lookups) lookups.opened = false; this.#state.requests.delete(name); - adopt(this.#state, producer); + adopt(this.#state, producer, track.infoDefaults(info)); return producer; } @@ -424,16 +433,20 @@ export class Producer { const producer = new track.Producer(name).accept(info); hooks.bindSequence(producer, this.#state.sequences.get(name)); - adopt(this.#state, producer); + adopt(this.#state, producer, track.infoDefaults(info)); return producer; } - /** Remove a statically inserted track, including a finished cached one, and stop counting it toward {@link demand} at once. */ + /** + * Remove a statically inserted track, including a finished cached one: stop counting it + * toward {@link demand} and listing it at once. + */ removeTrack(name: string): void { const track = this.#state.tracks.get(name); if (!track) return; this.#state.tracks.delete(name); this.#state.demands.get(track)?.(); + this.#state.listing.unlistOwn(name, track); } /** A lazy read handle for a track on this broadcast. */ @@ -536,6 +549,8 @@ export class Consumer { largest: (name) => largest(this.#state, name), fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), requested: () => requested(this.#state), + tracks: (owner) => listTracks(this.#state.listing, owner), + lister: () => new Lister(this.#state.listing), }); } @@ -607,6 +622,21 @@ export class Consumer { return trackOf(name, this); } + /** + * List the broadcast's tracks: each one with its info, then `synced`, then changes as they + * happen, until the broadcast ends. + * + * Lists the tracks its producer created (until they abort or are removed) and, for a + * broadcast consumed from a session, the tracks the peer lists. Through a session the + * listing syncs at once, with what this side holds, and the peer's tracks follow as live + * adds; every listing of one broadcast shares one request to the peer. A track served only + * on request is never listed, and listing never requests one. Throws once the broadcast + * has closed. + */ + tracks(): Tracks { + return listTracks(this.#state.listing); + } + /** * Release this handle. The broadcast is closed once this was the last live handle; * while other {@link clone}s remain open it stays live. diff --git a/js/net/src/listing.test.ts b/js/net/src/listing.test.ts new file mode 100644 index 00000000000..d3eb31325c9 --- /dev/null +++ b/js/net/src/listing.test.ts @@ -0,0 +1,326 @@ +import { expect, test } from "bun:test"; +import { Producer as BroadcastProducer } from "./broadcast.ts"; +import { NotFound, StreamCode, StreamError } from "./error.ts"; +import type { Tracks } from "./listing.ts"; +import { wireOf } from "./wire.ts"; + +/** Lets queued microtasks and the listing's signal notifications run. */ +function settle(): Promise { + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +/** + * Reads a listing as `+name`, `-name`, `synced`, `end`, or `error: ..`, keeping one pending read + * across calls so a read that loses to {@link drain}'s settle never drops an event. + */ +class Events { + readonly tracks: Tracks; + #pending?: Promise; + + constructor(tracks: Tracks) { + this.tracks = tracks; + } + + /** Every event ready now, through the end of the listing. */ + async drain(): Promise { + const events: string[] = []; + for (;;) { + this.#pending ??= this.tracks.next().then( + (event) => { + if (!event) return "end"; + if (event.kind === "synced") return "synced"; + return `${event.kind === "added" ? "+" : "-"}${event.name}`; + }, + (err: unknown) => `error: ${err instanceof StreamError ? err.code : String(err)}`, + ); + const event = await Promise.race([this.#pending, settle().then(() => undefined)]); + if (event === undefined) return events; + this.#pending = undefined; + events.push(event); + if (event === "end" || event.startsWith("error")) return events; + } + } +} + +function list(broadcast: BroadcastProducer): Events { + return new Events(broadcast.consume().tracks()); +} + +test("tracks present arrive before synced", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + expect(await list(broadcast).drain()).toEqual(["+a", "+b", "synced"]); +}); + +test("an empty listing syncs at once", async () => { + const broadcast = new BroadcastProducer(); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["synced"]); + broadcast.createTrack("a"); + expect(await tracks.drain()).toEqual(["+a"]); +}); + +test("a listed track reports its info", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a", { priority: 9, refresh: 4 }); + const event = await broadcast.consume().tracks().next(); + expect(event?.kind === "added" && event.info).toMatchObject({ priority: 9, refresh: 4 }); +}); + +test("a track leaving the table is removed", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + const b = broadcast.createTrack("b"); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+a", "+b", "synced"]); + broadcast.removeTrack("a"); + b.close(new Error("aborted")); + expect(await tracks.drain()).toEqual(["-a", "-b"]); +}); + +test("a finished track is listed while the table holds it", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a").close(); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+a", "synced"]); + broadcast.removeTrack("a"); + expect(await tracks.drain()).toEqual(["-a"]); + broadcast.close(); + expect(await tracks.drain()).toEqual(["end"]); +}); + +test("a recreated track is removed before it is added again", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a").close(); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+a", "synced"]); + // The finished track still holds the name; its replacement takes it over. + broadcast.createTrack("a"); + expect(await tracks.drain()).toEqual(["-a", "+a"]); +}); + +test("a track served on request is never listed", async () => { + const broadcast = new BroadcastProducer(); + const consumer = broadcast.consume(); + const tracks = new Events(consumer.tracks()); + const pulled = wireOf(broadcast).requested(); + expect(await tracks.drain()).toEqual(["synced"]); + + consumer.track("x").subscribe(); + const request = await pulled; + request?.accept({}); + expect(await tracks.drain()).toEqual([]); + + // A queued request answered by createTrack is the producer's track, so it is listed. + consumer.track("z").subscribe(); + broadcast.createTrack("z"); + expect(await tracks.drain()).toEqual(["+z"]); + broadcast.close(); +}); + +test("an initial track removed before it is read is omitted", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + const tracks = list(broadcast); + broadcast.removeTrack("a"); + expect(await tracks.drain()).toEqual(["+b", "synced"]); +}); + +test("churn never postpones synced", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + const tracks = broadcast.consume().tracks(); + // New names keep arriving faster than the reader drains them. + broadcast.createTrack("c"); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "a" }); + broadcast.createTrack("d"); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "b" }); + broadcast.createTrack("e"); + expect(await new Events(tracks).drain()).toEqual(["synced", "+c", "+d", "+e"]); +}); + +test("a removal of a seen track may precede synced", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + broadcast.createTrack("c"); + const tracks = broadcast.consume().tracks(); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "a" }); + broadcast.removeTrack("a"); + broadcast.removeTrack("c"); + expect(await new Events(tracks).drain()).toEqual(["+b", "-a", "synced"]); +}); + +test("a stalled reader holds one change per name", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("seen"); + broadcast.createTrack("stay"); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+seen", "+stay", "synced"]); + broadcast.removeTrack("seen"); + for (let round = 0; round < 1000; round++) { + broadcast.createTrack(`churn${round}`); + broadcast.createTrack("seen"); + broadcast.removeTrack(`churn${round}`); + broadcast.removeTrack("seen"); + } + expect(await tracks.drain()).toEqual(["-seen"]); +}); + +test("the broadcast ending ends the listing", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + const consumer = broadcast.consume(); + const tracks = new Events(consumer.tracks()); + broadcast.close(); + expect(await tracks.drain()).toEqual(["end"]); + expect(() => consumer.tracks()).toThrow("broadcast is closed"); +}); + +test("a closed listing reads nothing more, and leaving a loop early closes it", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + const tracks = broadcast.consume().tracks(); + for await (const event of tracks) { + expect(event).toMatchObject({ kind: "added", name: "a" }); + break; + } + broadcast.createTrack("b"); + expect(await tracks.next()).toBeUndefined(); +}); + +test("readers wait for the lister to sync", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("own"); + const lister = wireOf(broadcast).lister(); + expect(lister.used.peek()).toBe(false); + + const tracks = list(broadcast); + expect(lister.used.peek()).toBe(true); + expect(await tracks.drain()).toEqual(["+own"]); + lister.add("up", { priority: 0 }); + expect(() => lister.add("up", { priority: 0 })).toThrow("duplicate"); + lister.synced(); + expect(await tracks.drain()).toEqual(["+up", "synced"]); + + lister.remove("up"); + expect(() => lister.remove("up")).toThrow("not listed"); + expect(await tracks.drain()).toEqual(["-up"]); + + tracks.tracks.close(); + expect(lister.used.peek()).toBe(false); +}); + +test("a name listed by both is reported once", async () => { + const broadcast = new BroadcastProducer(); + broadcast.createTrack("a"); + const lister = wireOf(broadcast).lister(); + lister.add("a", { priority: 0 }); + lister.synced(); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+a", "synced"]); + broadcast.removeTrack("a"); + expect(await tracks.drain()).toEqual([]); + lister.remove("a"); + expect(await tracks.drain()).toEqual(["-a"]); +}); + +test("an aborted answer fails its readers and a later reader asks again", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + const tracks = list(broadcast); + lister.add("up", { priority: 0 }); + lister.abort(new NotFound("broadcast")); + expect(await tracks.drain()).toEqual([`error: ${StreamCode.NotFound}`]); + expect(lister.used.peek()).toBe(false); + + const again = list(broadcast); + expect(lister.used.peek()).toBe(true); + expect(await again.drain()).toEqual([]); + lister.synced(); + expect(await again.drain()).toEqual(["synced"]); + // The failed reader leaving releases nothing. + tracks.tracks.close(); + expect(lister.used.peek()).toBe(true); +}); + +test("a closed lister withdraws its answer and syncs", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + lister.add("up", { priority: 0 }); + const tracks = list(broadcast); + expect(await tracks.drain()).toEqual(["+up"]); + lister.close(); + expect(await tracks.drain()).toEqual(["-up", "synced"]); +}); + +test("an owner lists each broadcast once", async () => { + const broadcast = new BroadcastProducer(); + const other = new BroadcastProducer(); + broadcast.createTrack("a"); + const owner = {}; + const older = new Events(wireOf(broadcast).tracks(owner)); + const elsewhere = new Events(wireOf(other).tracks(owner)); + const stranger = new Events(wireOf(broadcast).tracks({})); + const newer = new Events(wireOf(broadcast).tracks(owner)); + expect(await older.drain()).toEqual([`error: ${StreamCode.Cancel}`]); + expect(await newer.drain()).toEqual(["+a", "synced"]); + expect(await elsewhere.drain()).toEqual(["synced"]); + expect(await stranger.drain()).toEqual(["+a", "synced"]); + + // The replaced reader leaving releases nothing of the newer one's. + older.tracks.close(); + broadcast.createTrack("b"); + expect(await newer.drain()).toEqual(["+b"]); + newer.tracks.close(); + stranger.tracks.close(); + expect(wireOf(broadcast).lister().used.peek()).toBe(false); +}); + +test("a broadcast has one lister", () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + expect(() => wireOf(broadcast).lister()).toThrow("already has a lister"); + lister.close(); + wireOf(broadcast).lister(); +}); + +test("a finished answer ends its readers and a later reader asks again", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + const tracks = list(broadcast); + lister.add("up", { priority: 0 }); + lister.synced(); + expect(await tracks.drain()).toEqual(["+up", "synced"]); + lister.finish(); + expect(await tracks.drain()).toEqual(["end"]); + expect(lister.used.peek()).toBe(false); + + const again = list(broadcast); + expect(await again.drain()).toEqual([]); +}); + +test("a cleared answer unsyncs a reader still waiting", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + broadcast.createTrack("a"); + broadcast.createTrack("b"); + lister.synced(); + const tracks = broadcast.consume().tracks(); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "a" }); + lister.clear(); + const events = new Events(tracks); + expect(await events.drain()).toEqual(["+b"]); + lister.synced(); + expect(await events.drain()).toEqual(["synced"]); +}); + +test("a lister sees the broadcast end", async () => { + const broadcast = new BroadcastProducer(); + const lister = wireOf(broadcast).lister(); + broadcast.close(); + expect(lister.ended.peek()).toBeNull(); +}); diff --git a/js/net/src/listing.ts b/js/net/src/listing.ts new file mode 100644 index 00000000000..71e631e5a9d --- /dev/null +++ b/js/net/src/listing.ts @@ -0,0 +1,439 @@ +/** + * A broadcast's track listing: the tracks it lists, reported to each reader in order. + * + * A track is listed while it is in the broadcast's table: created by its producer until it + * aborts or is removed, or answered by a {@link Lister}, as a session lists its peer's tracks. + * A track served only on request is never listed, and listing never requests one. + * + * Each {@link Tracks} reader keeps its own queue, keyed by name in insertion order, so a + * stalled reader holds at most one pending change per name: an add and a removal it has not + * reached cancel, and a removal and re-add of a name it saw collapse in place. Its state is + * bounded by the names it saw plus the live set, however long it stalls. + * + * @module + */ +import { type GetPromise, type Getter, Signal } from "@moq/signals"; +import { StreamCode, StreamError } from "./error.ts"; +import type * as track from "./track.ts"; + +/** + * What a {@link Tracks} listing reports, in order. + * + * `added`: a track the broadcast lists, with its fixed properties. `synced`: every track + * listed when the listing began, and still listed, was reported; sent once, and later events + * are live. `removed`: a track an earlier `added` reported is no longer listed, though a + * handler may still serve its name on request. + * + * @public + */ +export type TracksEvent = + | { kind: "added"; name: string; info: track.Info } + | { kind: "synced" } + | { kind: "removed"; name: string }; + +/** One listed track and who lists it. */ +interface Entry { + info: track.Info; + /** The producer whose track lists it, so a stale unlist never removes a later track of the name. */ + own?: object; + /** Whether the lister's answer lists it. */ + answered: boolean; +} + +/** A change a reader has yet to read. `readd` is a removal of a name it saw, then the name listed again. */ +type Pending = { kind: "add"; info: track.Info } | { kind: "remove" } | { kind: "readd"; info: track.Info }; + +/** One reader's pending changes, keyed by name in the order they happened. */ +class Queue { + pending = new Map(); + /** Events left to send before SYNCED, once the listing has synced. */ + bound?: number; + /** SYNCED was sent. */ + synced = false; + /** `null` once the listing ended, or the error that ended it. */ + end?: Error | null; + /** The reader left the listing, or a newer reader of its owner replaced it. */ + detached = false; + changed = new Signal(0); + + wake() { + this.changed.update((n) => n + 1); + } + + add(name: string, info: track.Info) { + // A name is listed once at a time, so only its removal can be pending. + if (this.pending.get(name)?.kind === "remove") this.pending.set(name, { kind: "readd", info }); + else this.pending.set(name, { kind: "add", info }); + this.wake(); + } + + remove(name: string) { + // Never read: the two cancel. Otherwise the removal replaces a pending re-add in place. + if (this.pending.get(name)?.kind === "add") this.pending.delete(name); + else this.pending.set(name, { kind: "remove" }); + this.wake(); + } + + /** The events the pending changes take to read. */ + events(): number { + let events = 0; + for (const pending of this.pending.values()) events += pending.kind === "readd" ? 2 : 1; + return events; + } + + /** The listing synced: SYNCED goes out once what is queued now was read, or earlier if the queue drains first. */ + sync() { + if (this.synced || this.bound !== undefined) return; + this.bound = this.events(); + this.wake(); + } + + /** The listing's answer restarted before this reader was told it synced. */ + unsync() { + if (!this.synced) this.bound = undefined; + } + + finish(end: Error | null) { + this.end ??= end; + this.detached = true; + this.wake(); + } + + ready(): boolean { + return this.end !== undefined || this.pending.size > 0 || (!this.synced && this.bound !== undefined); + } + + next(): TracksEvent | undefined { + if (this.end !== undefined) { + if (this.end) throw this.end; + return undefined; + } + if (!this.synced && this.bound !== undefined && (this.bound === 0 || this.pending.size === 0)) { + this.synced = true; + return { kind: "synced" }; + } + const first = this.pending.entries().next(); + if (first.done) throw new Error("a ready queue has a pending change"); + const [name, pending] = first.value; + if (!this.synced && this.bound !== undefined) this.bound -= 1; + switch (pending.kind) { + case "add": + this.pending.delete(name); + return { kind: "added", name, info: pending.info }; + case "remove": + this.pending.delete(name); + return { kind: "removed", name }; + case "readd": + // Setting an existing key keeps its place, so the add is read next. + this.pending.set(name, { kind: "add", info: pending.info }); + return { kind: "removed", name }; + } + } +} + +/** + * The listing half of a broadcast's state. + * + * @internal + */ +export class Listing { + /** Every listed track, in the order it was listed. */ + #live = new Map(); + /** The queue of every reader of the current answer. */ + #readers = new Set(); + /** Each owner's reader. */ + #owners = new Map(); + /** Whether a {@link Lister} exists. While one does, readers wait for its answer to sync. */ + #lister = false; + /** Whether the lister's answer has synced. */ + #answered = false; + #used = new Signal(false); + #closed: GetPromise; + + constructor(closed: GetPromise) { + this.#closed = closed; + } + + /** Whether anyone lists the broadcast. */ + get used(): Getter { + return this.#used; + } + + /** Settles once the broadcast ended. */ + get closed(): GetPromise { + return this.#closed; + } + + #synced(): boolean { + return !this.#lister || this.#answered; + } + + #counted() { + this.#used.set(this.#readers.size > 0); + } + + /** Re-evaluate whether the listing has synced, telling readers when it changed. */ + #resync(was: boolean) { + const now = this.#synced(); + if (!was && now) for (const queue of this.#readers) queue.sync(); + if (was && !now) for (const queue of this.#readers) queue.unsync(); + } + + #insert(name: string, entry: Entry) { + this.#live.set(name, entry); + for (const queue of this.#readers) queue.add(name, entry.info); + } + + #remove(name: string) { + if (!this.#live.delete(name)) return; + for (const queue of this.#readers) queue.remove(name); + } + + /** + * A reader with every listed track queued, ending `owner`'s older one with CANCELLED so an + * owner's listings cost one queue per broadcast however many requests reach it. + */ + read(owner?: object): Queue { + if (this.#closed.peek() !== undefined) { + throw new StreamError(StreamCode.Unroutable, { message: "broadcast is closed" }); + } + const queue = new Queue(); + for (const [name, entry] of this.#live) queue.pending.set(name, { kind: "add", info: entry.info }); + if (this.#synced()) queue.bound = queue.pending.size; + this.#readers.add(queue); + if (owner) { + const older = this.#owners.get(owner); + this.#owners.set(owner, queue); + if (older) { + this.#readers.delete(older); + older.finish(new StreamError(StreamCode.Cancel, { message: "replaced by a newer listing" })); + } + } + this.#counted(); + return queue; + } + + /** A reader left. A replaced or ended one already did. */ + release(queue: Queue, owner?: object) { + if (queue.detached) return; + queue.finish(null); + this.#readers.delete(queue); + if (owner && this.#owners.get(owner) === queue) this.#owners.delete(owner); + this.#counted(); + } + + /** List a track `own` created. */ + listOwn(name: string, own: object, info: track.Info) { + const entry = this.#live.get(name); + if (!entry) { + this.#insert(name, { info, own, answered: false }); + } else if (entry.own && !entry.answered) { + // The earlier track of the name has not unlisted yet, but it left the table for this + // one: readers see it go and the new one arrive. + this.#remove(name); + this.#insert(name, { info, own, answered: false }); + } else { + // Listed already by the answer, and it never stopped being listed: only the owner moves. + entry.own = own; + } + } + + /** Unlist a track `own` created, unless a later one took its name. */ + unlistOwn(name: string, own: object) { + const entry = this.#live.get(name); + if (entry?.own !== own) return; + entry.own = undefined; + if (!entry.answered) this.#remove(name); + } + + /** Lister support; see {@link Lister}. */ + attach() { + if (this.#lister) throw new Error("the broadcast already has a lister"); + const was = this.#synced(); + this.#lister = true; + this.#resync(was); + } + + detachLister() { + const was = this.#synced(); + this.#lister = false; + // Synced from here on, with only the producer's tracks. + this.#unanswer(); + this.#resync(was); + } + + answerAdd(name: string, info: track.Info) { + const entry = this.#live.get(name); + if (entry?.answered) throw new Error(`duplicate listed track: ${name}`); + if (entry) entry.answered = true; + else this.#insert(name, { info, answered: true }); + } + + answerRemove(name: string) { + const entry = this.#live.get(name); + if (!entry?.answered) throw new Error(`track not listed: ${name}`); + entry.answered = false; + if (!entry.own) this.#remove(name); + } + + answerSynced() { + const was = this.#synced(); + this.#answered = true; + this.#resync(was); + } + + /** Drop the answer: every track it listed, and its sync. */ + answerClear() { + const was = this.#synced(); + this.#unanswer(); + this.#resync(was); + } + + #unanswer() { + for (const [name, entry] of [...this.#live]) { + if (entry.answered) this.answerRemove(name); + } + this.#answered = false; + } + + /** End every reader with `end` and detach them. */ + detach(end: Error | null) { + for (const queue of this.#readers) queue.finish(end); + this.#readers.clear(); + this.#owners.clear(); + this.#counted(); + } +} + +let makeTracks: (listing: Listing, owner?: object) => Tracks; + +/** + * List a broadcast's tracks, ending `owner`'s older listing of it with CANCELLED. + * + * @internal + */ +export function listTracks(listing: Listing, owner?: object): Tracks { + return makeTracks(listing, owner); +} + +/** + * The tracks a broadcast lists, from its consumer's `tracks()`. + * + * Reports every track listed when it was created, then `synced`, then each change as it + * happens, until the broadcast ends. Listing a broadcast never requests a track: subscribe to + * each listed one through the consumer's `track(name)`. + * + * @public + */ +export class Tracks { + #listing: Listing; + #queue: Queue; + #owner?: object; + + private constructor(listing: Listing, owner?: object) { + this.#listing = listing; + this.#queue = listing.read(owner); + this.#owner = owner; + } + + static { + makeTracks = (listing, owner) => new Tracks(listing, owner); + } + + /** + * The next event, or undefined once the broadcast ended or this listing closed. Throws the + * error that ended the listing, such as the peer refusing it. + */ + async next(): Promise { + for (;;) { + if (this.#queue.ready()) return this.#queue.next(); + await Signal.race(this.#queue.changed); + } + } + + /** The events as they arrive; leaving the loop early closes the listing. */ + async *[Symbol.asyncIterator](): AsyncGenerator { + try { + for (;;) { + const event = await this.next(); + if (!event) return; + yield event; + } + } finally { + this.close(); + } + } + + /** Stop listing. Idempotent. */ + close(): void { + this.#listing.release(this.#queue, this.#owner); + } +} + +// pending: /quest/m1/public-model/answer.md +/** + * Answers a broadcast's listings for a handler that knows the tracks it would serve, as a + * session lists its peer's. A broadcast has at most one. + * + * While it exists, readers wait for it to {@link synced} before reporting `synced`. Answer + * while {@link used}: list what the publisher behind it holds and keep it current, then + * {@link clear} once unused. {@link close} withdraws what it listed. + */ +export class Lister { + #listing: Listing; + #closed = false; + + constructor(listing: Listing) { + listing.attach(); + this.#listing = listing; + } + + /** Whether anyone lists the broadcast. */ + get used(): Getter { + return this.#listing.used; + } + + /** Settles once the broadcast ended. */ + get ended(): GetPromise { + return this.#listing.closed; + } + + /** List a track. Throws if this answer already lists it. */ + add(name: string, info: track.Info): void { + this.#listing.answerAdd(name, info); + } + + /** Stop listing a track. Throws if this answer does not list it. */ + remove(name: string): void { + this.#listing.answerRemove(name); + } + + /** The answer holds everything this hop knows now; later changes are live. */ + synced(): void { + this.#listing.answerSynced(); + } + + /** Withdraw the answer: every track it listed, and its sync. */ + clear(): void { + this.#listing.answerClear(); + } + + /** End every current reader as if the broadcast ended, and withdraw the answer. A later reader asks again. */ + finish(): void { + this.#listing.detach(null); + this.#listing.answerClear(); + } + + /** Fail every current reader with `err`, and withdraw the answer. A later reader asks again. */ + abort(err: Error): void { + this.#listing.detach(err); + this.#listing.answerClear(); + } + + /** Stop answering: the broadcast lists only its producer's tracks from here on. Idempotent. */ + close(): void { + if (this.#closed) return; + this.#closed = true; + this.#listing.detachLister(); + } +} diff --git a/js/net/src/lite/connection.ts b/js/net/src/lite/connection.ts index 86dc40c6bc3..0aba0ef7ee1 100644 --- a/js/net/src/lite/connection.ts +++ b/js/net/src/lite/connection.ts @@ -28,6 +28,7 @@ import { DataType, StreamId } from "./stream.ts"; import { Subscribe } from "./subscribe.ts"; import { Subscriber } from "./subscriber.ts"; import { Track as TrackMessage } from "./track.ts"; +import { Tracks as TracksMessage } from "./tracks.ts"; import { hasAuth, hasDatagrams, @@ -35,6 +36,7 @@ import { hasProbeRtt, hasSetupStream, hasTrackStream, + hasTracks, Version, versionName, } from "./version.ts"; @@ -362,6 +364,10 @@ export class Connection implements Established { await this.#publisher.runTrackInfo(msg, stream); })(), ); + } else if (typ === StreamId.Tracks) { + if (!hasTracks(this.#version)) throw new Error("TRACKS is not supported on this version"); + const msg = await TracksMessage.decode(stream.reader, this.#version); + await this.#publisher.runTracks(msg, stream); } else if (typ === StreamId.Probe) { if (this.#version === Version.DRAFT_01 || this.#version === Version.DRAFT_02) throw new Error("PROBE is not supported on this version"); diff --git a/js/net/src/lite/index.ts b/js/net/src/lite/index.ts index 278e7b731e9..8e588403e91 100644 --- a/js/net/src/lite/index.ts +++ b/js/net/src/lite/index.ts @@ -10,4 +10,5 @@ export * from "./setup.ts"; export * from "./stream.ts"; export * from "./subscribe.ts"; export * from "./track.ts"; +export * from "./tracks.ts"; export * from "./version.ts"; diff --git a/js/net/src/lite/malformed.test.ts b/js/net/src/lite/malformed.test.ts index abc5967e37c..f81e1e460ef 100644 --- a/js/net/src/lite/malformed.test.ts +++ b/js/net/src/lite/malformed.test.ts @@ -131,8 +131,8 @@ for (const [alpn, version] of versions) { } for (const [alpn, version] of [[ALPN_04, Version.DRAFT_04], ...versions] as const) { - // An unassigned type, and the Auth Stream on versions that predate it. - for (const typ of version === Version.DRAFT_07 ? [8] : [8, StreamId.Auth]) { + // An unassigned type, and the Tracks and Auth Streams on versions that predate them. + for (const typ of version === Version.DRAFT_07 ? [9] : [9, StreamId.Tracks, StreamId.Auth]) { test(`${alpn}: unknown stream type ${typ} resets only its stream`, async () => { const f = fixture(alpn, version); try { diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index c9e64cc4336..8ede3954838 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -38,6 +38,7 @@ import { SubscribeUpdate, } from "./subscribe.ts"; import { TrackInfo as TrackInfoMessage, type Track as TrackMessage } from "./track.ts"; +import { encodeTracksEvent, type Tracks as TracksMessage } from "./tracks.ts"; import { hasAnnounceId, hasAnnounceOk, @@ -80,6 +81,19 @@ function acknowledged(writer: Writer): Promise { return writer.closed.catch(() => {}); } +// The TRACK_INFO a TRACK stream, or a listing's TRACKS_ADD, answers for a track. +function trackInfoMessage(info: track.Info): TrackInfoMessage { + return new TrackInfoMessage({ + priority: info.priority, + refresh: info.refresh, + // Publisher Max Age: the publisher's retention bound, advertised so relays re-serve + // with the same window. + maxAge: info.maxAge, + // `#serveGroup` emits each frame converted to it. + timescale: info.timescale ?? undefined, + }); +} + /** A track's cached TRACK_INFO, and the request behind it while requesters hold it. */ interface TrackInfoEntry { info: Promise; @@ -1407,6 +1421,75 @@ export class Publisher { } } + /** + * Answers a TRACKS stream (0x8): the broadcast's listed tracks, SYNCED, then each change, + * until the broadcast ends (FIN) or a newer listing of it on this session replaces this one. + * + * Each listing queues the broadcast's whole listed set, so the session lists each broadcast + * once: a newer listing replaces the older rather than being refused, since a subscriber + * that stops listing and lists again can open the new stream before we see the old one + * close. The requester's FIN or reset cancels it. Not owed: it lasts as long as the broadcast. + * + * @internal + */ + async runTracks(msg: TracksMessage, stream: Stream) { + // Charged before anything else and held until the listing ends, as in Rust. A peer past + // the cap loses the session. + const interest = this.#interests.acquire(msg.broadcast, "", "tracks"); + const cancel = new AbortController(); + const watch = this.#gate.watch(msg.broadcast, () => { + console.debug(`tracks revoked: broadcast=${msg.broadcast}`); + cancel.abort(unauthorized(msg.broadcast)); + }); + // TRACKS is the requester's only message: its FIN or reset ends the listing. + void stream.reader.done().then( + (fin) => { + if (!fin) stream.abort(new ProtocolViolation("data after TRACKS")); + cancel.abort(new StreamError(StreamCode.Cancel, { message: "listing cancelled" })); + }, + (err: unknown) => cancel.abort(error(err)), + ); + let tracks: broadcast.Tracks | undefined; + try { + if (watch.revoked()) throw unauthorized(msg.broadcast); + const origin = this.#publish && wireOf(this.#publish); + // A route claiming a prefix names no broadcast, so a path it serves on request has + // nothing to list. + if (origin?.covered(msg.broadcast)) throw new NotFound(`broadcast ${msg.broadcast}`); + const front = + origin && + (origin.local(msg.broadcast, msg.epoch) ?? + (await origin.demand(msg.broadcast, msg.epoch, cancel.signal))); + if (!front) throw new NotFound(`broadcast ${msg.broadcast}`); + + tracks = wireOf(front).tracks(this); + console.debug(`tracks: broadcast=${msg.broadcast}`); + for (;;) { + // Pull an event only once the last is written, so a stalled requester's changes + // collapse in the listing's queue rather than pile up here. + const event = await untilAborted(tracks.next(), cancel.signal); + if (!event) break; + await untilAborted( + encodeTracksEvent( + stream.writer, + event.kind === "added" ? { ...event, info: trackInfoMessage(event.info) } : event, + this.version, + ), + cancel.signal, + ); + } + stream.writer.close(); + } catch (err) { + const e = cancel.signal.aborted ? error(cancel.signal.reason) : error(err); + console.debug(`tracks end: broadcast=${msg.broadcast} error=${reason(e)}`); + stream.writer.reset(e); + } finally { + tracks?.close(); + watch.dispose(); + interest.release(); + } + } + // Resolve (and cache) a track's immutable TRACK_INFO by asking the application. // `resolveTrackInfo` triggers a TrackRequest the app answers with accept(TrackInfo); // only the immutable properties are needed (not the groups). Cached because they're @@ -1432,18 +1515,7 @@ export class Publisher { if (reuse && entry?.answered) return entry.info; if (!entry || entry.release.signal.aborted) { const release = new AbortController(); - const info = (async () => { - const info = await wireOf(front).resolveTrackInfo(track, release.signal); - return new TrackInfoMessage({ - priority: info.priority, - refresh: info.refresh, - // Publisher Max Age: the publisher's retention bound, advertised so - // relays re-serve with the same window. - maxAge: info.maxAge, - // `#serveGroup` emits each frame converted to it. - timescale: info.timescale ?? undefined, - }); - })(); + const info = (async () => trackInfoMessage(await wireOf(front).resolveTrackInfo(track, release.signal)))(); const created: TrackInfoEntry = { info, holders: 0, answered: false, release }; info.then( diff --git a/js/net/src/lite/stream.ts b/js/net/src/lite/stream.ts index 5cec9caa8e2..740cfc35381 100644 --- a/js/net/src/lite/stream.ts +++ b/js/net/src/lite/stream.ts @@ -7,6 +7,7 @@ export const StreamId = { Goaway: 5, Track: 6, Auth: 7, + Tracks: 8, ClientCompat: 0x20, ServerCompat: 0x21, } as const; diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 4a1cacc8009..a410b6163da 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -9,11 +9,22 @@ import type { Probe as ProbeStats } from "../connection/stats.ts"; import { BroadcastCache } from "../consume.ts"; import * as DatagramStream from "../datagram_stream.ts"; import type * as Epoch from "../epoch.ts"; -import { error, ProtocolViolation, reason, StreamCode, StreamError, sessionCause, unauthorized } from "../error.ts"; +import { + error, + ProtocolViolation, + reason, + SessionCode, + SessionError, + StreamCode, + StreamError, + sessionCause, + unauthorized, +} from "../error.ts"; import * as netGroup from "../group.ts"; import { Cost, type Hop, MAX_HOPS, type Route, routesEqual, UNKNOWN_HOP } from "../hop.ts"; import { groupBounds, hiddenBelow, hooks, scopeCaptures, scopeHead, scopeOverlaps } from "../internal.ts"; import { grace } from "../ledger.ts"; +import type { Lister } from "../listing.ts"; import * as Path from "../path.ts"; import { sendOrder } from "../priority.ts"; import { type OpenOptions, type Reader, Stream, Writer } from "../stream.ts"; @@ -50,6 +61,7 @@ import { SubscribeUpdate, } from "./subscribe.ts"; import { TrackInfo, Track as TrackMessage } from "./track.ts"; +import { decodeTracksEventMaybe, Tracks as TracksMessage } from "./tracks.ts"; import { hasAnnounceId, hasAnnounceOk, @@ -60,6 +72,7 @@ import { hasRefresh, hasSynced, hasTrackStream, + hasTracks, resolvesStart, updateSupported, Version, @@ -627,6 +640,8 @@ export class Subscriber { }, }; const consumer = new ConsumeBroadcast(this, path, consumed); + // Registered before anyone can list the broadcast, so no reader syncs without the peer. + void this.#runListing(path, consumed, wireOf(consumer).lister()); void (async () => { for (;;) { @@ -639,6 +654,99 @@ export class Subscriber { return this.#consumes.insert(path, instance, consumer); } + // Answer listings of a consumed broadcast from the peer's TRACKS while anyone lists it, until + // the broadcast closes. Every listing of the broadcast shares one TRACKS stream. + async #runListing(broadcast: Path.Valid, consumed: Consumed, lister: Lister) { + try { + for (;;) { + while (!lister.used.peek()) { + if (lister.ended.peek() !== undefined || this.#closed.signal.aborted) return; + await untilAborted(race([Signal.race(lister.used, lister.ended)]), this.#closed.signal).catch( + () => {}, + ); + } + if (!hasTracks(this.version)) { + // An empty answer would claim the peer lists nothing. + lister.abort(new Error("track listing requires moq-lite-07 or newer")); + continue; + } + // Synced with this hop, which holds nothing yet: a relay never waits on its upstream, + // so the peer's tracks follow as live adds. + lister.synced(); + await this.#listTracks(broadcast, consumed, lister); + } + } finally { + lister.close(); + } + } + + // Forward the peer's TRACKS listing of `broadcast` into `lister` until the peer ends it, it + // fails, or nobody lists the broadcast any more. + async #listTracks(broadcast: Path.Valid, consumed: Consumed, lister: Lister): Promise { + // Each listed track holds an announces slot: discovery state the peer makes us hold. + const listed = new Set(); + const unused = new AbortController(); + const stop = AbortSignal.any([unused.signal, this.#closed.signal]); + const disposeUsed = lister.used.subscribe((used) => { + if (!used) unused.abort(new StreamError(StreamCode.Cancel, { message: "listing unused" })); + }); + let stream: Stream | undefined; + try { + const opening = Stream.open(this.#quic, { version: this.version, ...requestOptions }); + void opening.then( + (late) => { + if (stop.aborted) late.abort(error(stop.reason)); + }, + () => {}, + ); + stream = await untilAborted(opening, stop); + await stream.writer.u53(StreamId.Tracks); + await new TracksMessage(broadcast, consumed.epoch).encode(stream.writer, this.version); + let synced = false; + for (;;) { + const event = await untilAborted(decodeTracksEventMaybe(stream.reader, this.version), stop); + if (!event) { + // The peer's broadcast ended, whether or not its retraction has arrived: end the + // listing and let go of what it listed. + lister.finish(); + return; + } + if (event.kind === "added") { + if (listed.has(event.name)) throw new ProtocolViolation(`track listed twice: ${event.name}`); + if (!this.#announces.tryAcquire()) { + throw new SessionError(SessionCode.TooManyRequests, { reason: "too many listed tracks" }); + } + listed.add(event.name); + lister.add(event.name, this.#toModelInfo(event.info)); + } else if (event.kind === "removed") { + if (!listed.delete(event.name)) + throw new ProtocolViolation(`unlisted track removed: ${event.name}`); + this.#announces.release(); + lister.remove(event.name); + } else { + // This hop synced its readers when the listing began; only the order matters here. + if (synced) throw new ProtocolViolation("duplicate SYNCED"); + synced = true; + } + } + } catch (err) { + if (unused.signal.aborted) { + // Nobody lists the broadcast any more: cancel the peer's listing. + stream?.close(); + lister.clear(); + return; + } + closeViolation(this.#quic, err); + const e = await sessionCause(this.#quic, err); + console.debug(`tracks failed: broadcast=${broadcast} error=${reason(e)}`); + stream?.abort(e); + lister.abort(e); + } finally { + disposeUsed(); + this.#announces.release(listed.size); + } + } + async #runSubscribe(broadcast: Path.Valid, consumed: Consumed, request: track.Request) { const id = this.#subscribeNext++; const subscription = request.subscription; diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts new file mode 100644 index 00000000000..b48c856e9b5 --- /dev/null +++ b/js/net/src/lite/tracks.test.ts @@ -0,0 +1,415 @@ +import { afterEach, expect, mock, spyOn, test } from "bun:test"; +import { Milli } from "@moq/clock"; +import { Producer as BroadcastProducer, type Tracks } from "../broadcast.ts"; +import { accept, connect } from "../connection/index.ts"; +import * as Epoch from "../epoch.ts"; +import { fromTransport, SessionCode, StreamCode, StreamError } from "../error.ts"; +import { randomHop } from "../hop.ts"; +import { createMockTransportPair } from "../mock.ts"; +import { Producer as OriginProducer } from "../origin.ts"; +import * as Path from "../path.ts"; +import { Reader, Stream, Writer } from "../stream.ts"; +import { Timescale } from "../time.ts"; +import { wireOf } from "../wire.ts"; +import { Publisher } from "./publisher.ts"; +import { StreamId } from "./stream.ts"; +import { Subscriber } from "./subscriber.ts"; +import { TrackInfo } from "./track.ts"; +import { decodeTracksEventMaybe, encodeTracksEvent, type TracksEvent, Tracks as TracksMessage } from "./tracks.ts"; +import { ALPN_06, ALPN_07_WIP, Version } from "./version.ts"; + +const url = new URL("https://localhost:4443/test"); +const version = Version.DRAFT_07; + +afterEach(() => mock.restore()); + +/** Lets queued microtasks, signal notifications, and mock stream deliveries run. */ +function settle(): Promise { + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +async function bytes(f: (w: Writer) => Promise): Promise { + const chunks: number[] = []; + const writer = new Writer( + new WritableStream({ write: (chunk) => void chunks.push(...chunk) }), + version, + ); + await f(writer); + writer.close(); + await writer.closed; + return Uint8Array.from(chunks); +} + +function format(event: Awaited>): string { + if (!event) return "end"; + if (event.kind === "synced") return "synced"; + return `${event.kind === "added" ? "+" : "-"}${event.name}`; +} + +/** The next event of a listing, formatted like the model tests, or the error that ended it. */ +async function next(tracks: Tracks): Promise { + try { + return format(await tracks.next()); + } catch (err) { + return `error: ${err instanceof StreamError ? err.code : String(err)}`; + } +} + +test("SYNCED is an empty type four, and an unknown event is refused", async () => { + expect(await bytes((w) => encodeTracksEvent(w, { kind: "synced" }, version))).toEqual(new Uint8Array([4, 0])); + await expect( + decodeTracksEventMaybe(new Reader(undefined, new Uint8Array([2, 0]), version), version), + ).rejects.toThrow("unknown tracks event"); + await expect( + decodeTracksEventMaybe(new Reader(undefined, new Uint8Array([4, 1, 0]), version), version), + ).rejects.toThrow("too few bytes"); +}); + +test("events and the request round-trip", async () => { + const events: TracksEvent[] = [ + { + kind: "added", + name: "video", + info: new TrackInfo({ priority: 3, refresh: 4, maxAge: 2000, timescale: 1_000_000 }), + }, + { kind: "removed", name: "video" }, + { kind: "synced" }, + ]; + const encoded = await bytes(async (w) => { + for (const event of events) await encodeTracksEvent(w, event, version); + }); + const reader = new Reader(undefined, encoded, version); + const added = await decodeTracksEventMaybe(reader, version); + expect(added).toMatchObject({ kind: "added", name: "video", info: { priority: 3, refresh: 4, maxAge: 2000 } }); + expect(added?.kind === "added" && added.info.timescale).toBe(Timescale(1_000_000)); + expect(await decodeTracksEventMaybe(reader, version)).toEqual({ kind: "removed", name: "video" }); + expect(await decodeTracksEventMaybe(reader, version)).toEqual({ kind: "synced" }); + expect(await decodeTracksEventMaybe(reader, version)).toBeUndefined(); + + const epoch = Epoch.parse("01900000-0000-7000-8000-000000000001"); + const request = await bytes((w) => new TracksMessage(Path.from("room/alice"), epoch).encode(w, version)); + const got = await TracksMessage.decode(new Reader(undefined, request, version), version); + expect(got.broadcast).toBe(Path.from("room/alice")); + expect(got.epoch).toBe(epoch); +}); + +test("older versions have no tracks stream", async () => { + await expect(bytes((w) => new TracksMessage(Path.from("room")).encode(w, Version.DRAFT_06))).rejects.toThrow( + "not supported", + ); + await expect(bytes((w) => encodeTracksEvent(w, { kind: "synced" }, Version.DRAFT_06))).rejects.toThrow( + "not supported", + ); +}); + +/** A lite-07 subscriber whose publisher the test plays by hand. */ +function handPlayed(limits?: { announces: number; subscriptions: number; fetches: number }) { + const pair = createMockTransportPair(ALPN_07_WIP); + const closing = spyOn(pair.client, "close"); + const subscriber = new Subscriber( + pair.client, + version, + randomHop(), + undefined, + undefined, + undefined, + undefined, + limits, + ); + // The TRACKS request the subscriber opens next. + const listed = async () => { + const stream = await Stream.accept(pair.server, version); + if (!stream) throw new Error("the subscriber never listed"); + expect(await stream.reader.u53()).toBe(StreamId.Tracks); + const msg = await TracksMessage.decode(stream.reader, version); + return { + msg, + stream, + send: (event: TracksEvent) => encodeTracksEvent(stream.writer, event, version), + }; + }; + return { pair, closing, subscriber, listed }; +} + +const info = new TrackInfo({ priority: 2, timescale: 1000 }); + +test("a session syncs at once and forwards the peer's tracks as live adds", async () => { + const { subscriber, listed } = handPlayed(); + const remote = subscriber.consume(Path.from("room")); + const tracks = remote.tracks(); + expect(await next(tracks)).toBe("synced"); + + const peer = await listed(); + expect(peer.msg.broadcast).toBe(Path.from("room")); + await peer.send({ kind: "added", name: "a", info }); + await peer.send({ kind: "synced" }); + const added = await tracks.next(); + expect(added).toMatchObject({ kind: "added", name: "a", info: { priority: 2, timescale: Timescale.MILLI } }); + + await peer.send({ kind: "removed", name: "a" }); + expect(await next(tracks)).toBe("-a"); + + // The peer's broadcast ended: so does the listing, whether or not its retraction arrived. + peer.stream.writer.close(); + expect(await next(tracks)).toBe("end"); + expect(remote.closed.peek()).toBeUndefined(); +}); + +test("every listing of a broadcast shares one TRACKS stream, cancelled once unused", async () => { + const { subscriber, listed, pair } = handPlayed(); + const remote = subscriber.consume(Path.from("room")); + const first = remote.tracks(); + const peer = await listed(); + await peer.send({ kind: "added", name: "a", info }); + expect(await next(first)).toBe("synced"); + expect(await next(first)).toBe("+a"); + + // A second reader, through a shared handle too, joins the same stream with what it holds. + const second = subscriber.consume(Path.from("room")).tracks(); + expect(await next(second)).toBe("+a"); + expect(await next(second)).toBe("synced"); + let opened = false; + const another = Stream.accept(pair.server, version).then((stream) => { + opened = true; + return stream; + }); + await settle(); + expect(opened).toBe(false); + + let cancelled = false; + void peer.stream.reader.done().then( + () => (cancelled = true), + () => (cancelled = true), + ); + first.close(); + await settle(); + expect(cancelled).toBe(false); + // The last reader leaving ends the requester's side. + second.close(); + await settle(); + expect(cancelled).toBe(true); + + // A later reader asks again. + const third = remote.tracks(); + await another; + expect(opened).toBe(true); + third.close(); +}); + +test("a peer's malformed listing closes the session", async () => { + for (const events of [ + [{ kind: "added", name: "a", info } as const, { kind: "added", name: "a", info } as const], + [{ kind: "removed", name: "a" } as const], + [{ kind: "synced" } as const, { kind: "synced" } as const], + ]) { + const { subscriber, listed, closing } = handPlayed(); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + for (const event of events) await peer.send(event); + let event: string; + do event = await next(tracks); + while (event === "synced" || event.startsWith("+")); + expect(event).toMatch(/^error: /); + expect(closing).toHaveBeenCalledWith(expect.objectContaining({ closeCode: SessionCode.ProtocolViolation })); + } +}); + +test("a listing past the announces cap is reset, and the session stays", async () => { + const cap = 3; + const { subscriber, listed, closing } = handPlayed({ announces: cap, subscriptions: 10, fetches: 10 }); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + for (let i = 0; i <= cap; i++) await peer.send({ kind: "added", name: `t${i}`, info }); + let added = 0; + let end: string; + for (;;) { + end = await next(tracks); + if (end.startsWith("+")) added++; + else if (end !== "synced") break; + } + expect(added).toBe(cap); + expect(end).toMatch(/^error: /); + await settle(); + expect(closing).not.toHaveBeenCalled(); + // The reset released every slot: another listing lists the whole cap again. + const again = subscriber.consume(Path.from("room")).tracks(); + const retry = await listed(); + for (let i = 0; i < cap; i++) await retry.send({ kind: "added", name: `t${i}`, info }); + expect(await next(again)).toBe("synced"); + for (let i = 0; i < cap; i++) expect(await next(again)).toBe(`+t${i}`); + again.close(); +}); + +test("a version without the Tracks Stream refuses the listing", async () => { + const pair = createMockTransportPair(); + const subscriber = new Subscriber(pair.client, Version.DRAFT_06, randomHop()); + const tracks = subscriber.consume(Path.from("room")).tracks(); + expect(await next(tracks)).toBe("error: Error: track listing requires moq-lite-07 or newer"); +}); + +/** A lite-07 publisher serving `origin`, answering TRACKS streams the test writes by hand. */ +function publisher(origin: OriginProducer) { + const pair = createMockTransportPair(ALPN_07_WIP); + const publisher = new Publisher(pair.server, version, randomHop(), origin.consume()); + const list = async (broadcast: string) => { + const requester = await Stream.open(pair.client, { version }); + void (async () => { + const stream = await Stream.accept(pair.server, version); + if (!stream) throw new Error("no stream"); + expect(await stream.reader.u53()).toBe(StreamId.Tracks); + await publisher.runTracks(await TracksMessage.decode(stream.reader, version), stream); + })(); + await requester.writer.u53(StreamId.Tracks); + await new TracksMessage(Path.from(broadcast)).encode(requester.writer, version); + return { + requester, + next: async () => { + try { + const event = await decodeTracksEventMaybe(requester.reader, version); + return event?.kind === "added" + ? `+${event.name}` + : event?.kind === "removed" + ? `-${event.name}` + : (event?.kind ?? "end"); + } catch (err) { + const reset = fromTransport(err); + return `reset: ${reset instanceof StreamError ? reset.code : reset.message}`; + } + }, + }; + }; + return { list }; +} + +test("a publisher lists its broadcast's own tracks, never one served on request", async () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a", { priority: 5 }); + // Served on request: answered when asked, never listed. + void wireOf(broadcast) + .requested() + .then((request) => request?.accept({})); + const { list } = publisher(origin); + const listing = await list("room"); + expect(await listing.next()).toBe("+a"); + expect(await listing.next()).toBe("synced"); + + broadcast.consume().track("x").subscribe(); + broadcast.createTrack("b"); + expect(await listing.next()).toBe("+b"); + + broadcast.close(); + expect(await listing.next()).toBe("end"); +}); + +test("a newer listing of a broadcast on one session replaces the older", async () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a"); + const { list } = publisher(origin); + const older = await list("room"); + expect(await older.next()).toBe("+a"); + expect(await older.next()).toBe("synced"); + const newer = await list("room"); + expect(await newer.next()).toBe("+a"); + expect(await newer.next()).toBe("synced"); + expect(await older.next()).toBe(`reset: ${StreamCode.Cancel}`); + broadcast.createTrack("b"); + expect(await newer.next()).toBe("+b"); + broadcast.close(); +}); + +test("a path served through a covering prefix route is NOT_FOUND", async () => { + const origin = new OriginProducer(); + const dynamic = origin.dynamic(Path.from("room")); + void (async () => { + for await (const request of dynamic.requested()) { + const served = new BroadcastProducer(); + served.createTrack("video"); + request.accept(served); + } + })(); + const { list } = publisher(origin); + expect(await (await list("room/alice")).next()).toBe(`reset: ${StreamCode.NotFound}`); + expect(await (await list("absent")).next()).toBe(`reset: ${StreamCode.NotFound}`); + dynamic.close(); +}); + +test("a listing over a session carries each track's info, end to end", async () => { + const pair = createMockTransportPair(ALPN_07_WIP); + const origin = new OriginProducer(); + const [client, server] = await Promise.all([ + connect({ url, transport: pair.client }), + accept({ transport: pair.server, url, publish: origin.consume() }), + ]); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a", { timescale: Timescale.MILLI, maxAge: Milli(3000), priority: 9, refresh: 4 }); + + const tracks = wireOf(client).consume(Path.from("room")).tracks(); + expect(await next(tracks)).toBe("synced"); + expect(await tracks.next()).toEqual({ + kind: "added", + name: "a", + info: { timescale: Timescale.MILLI, maxAge: Milli(3000), priority: 9, refresh: 4 }, + }); + broadcast.createTrack("b"); + expect(await next(tracks)).toBe("+b"); + broadcast.removeTrack("a"); + expect(await next(tracks)).toBe("-a"); + broadcast.close(); + expect(await next(tracks)).toBe("end"); + client.abort(); + server.abort(); +}); + +test("an epoch-pinned front lists its serving route's tracks", async () => { + const pair = createMockTransportPair(ALPN_07_WIP); + const serverOrigin = new OriginProducer(); + const clientOrigin = new OriginProducer(); + const [client, server] = await Promise.all([ + connect({ url, transport: pair.client, consume: clientOrigin }), + accept({ transport: pair.server, url, publish: serverOrigin.consume() }), + ]); + const broadcast = serverOrigin.createBroadcast(Path.from("room")); + broadcast.createTrack("a"); + broadcast.announce({ epoch: Epoch.parse("01900000-0000-7000-8000-000000000001") }); + + const reader = clientOrigin.consume(); + const announced = reader.announced(); + expect(await announced.next()).toMatchObject({ prefix: Path.from("room"), kind: "start" }); + const request = reader.request(Path.from("room")); + await settle(); + const front = request.active.peek(); + if (!front) throw new Error("expected the origin to route the broadcast"); + const tracks = front.tracks(); + // The front syncs with its source's listing, which syncs at once with what the session holds. + expect(await next(tracks)).toBe("synced"); + expect(await next(tracks)).toBe("+a"); + broadcast.createTrack("b"); + expect(await next(tracks)).toBe("+b"); + tracks.close(); + request.close(); + announced.close(); + broadcast.close(); + client.abort(); + server.abort(); +}); + +test("lite-06 cannot list over a session", async () => { + const pair = createMockTransportPair(ALPN_06); + const origin = new OriginProducer(); + const [client, server] = await Promise.all([ + connect({ url, transport: pair.client }), + accept({ transport: pair.server, url, publish: origin.consume() }), + ]); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + const tracks = wireOf(client).consume(Path.from("room")).tracks(); + expect(await next(tracks)).toMatch(/requires moq-lite-07/); + broadcast.close(); + client.abort(); + server.abort(); +}); diff --git a/js/net/src/lite/tracks.ts b/js/net/src/lite/tracks.ts new file mode 100644 index 00000000000..6bb2e121f27 --- /dev/null +++ b/js/net/src/lite/tracks.ts @@ -0,0 +1,100 @@ +import type * as Epoch from "../epoch.ts"; +import { ProtocolViolation } from "../error.ts"; +import * as Path from "../path.ts"; +import type { Reader, Writer } from "../stream.ts"; +import { decodeError } from "../util/decode_error.ts"; +import { decodeEpoch, encodeEpoch } from "./epoch.ts"; +import * as Message from "./message.ts"; +import { TrackInfo } from "./track.ts"; +import { hasTracks, type Version } from "./version.ts"; + +const TRACKS_ADD = 0x0; +const TRACKS_REMOVE = 0x1; +// The same event the announce stream marks its initial set with. +const SYNCED = 0x4; + +function guardTracks(version: Version) { + if (!hasTracks(version)) throw new Error("tracks stream not supported for this version"); +} + +/** + * TRACKS request: the subscriber's only message on a Tracks Stream (0x8), listing one + * broadcast's tracks. Lite-07+. + */ +export class Tracks { + broadcast: Path.Valid; + /** The publisher instance the subscriber expects. */ + epoch?: Epoch.Valid; + + constructor(broadcast: Path.Valid, epoch?: Epoch.Valid) { + this.broadcast = broadcast; + this.epoch = epoch; + } + + async encode(w: Writer, version: Version): Promise { + guardTracks(version); + return Message.encode(w, async (w) => { + await w.string(Path.encode(this.broadcast)); + await encodeEpoch(w, version, this.epoch); + }); + } + + static async decode(r: Reader, version: Version): Promise { + guardTracks(version); + return Message.decode(r, async (r) => { + const broadcast = Path.decode(await r.string()); + const epoch = await decodeEpoch(r, version); + return new Tracks(broadcast, epoch); + }); + } +} + +/** + * One event the publisher sends on a Tracks Stream: TRACKS_ADD with the TRACK_INFO a Track + * Stream would answer, TRACKS_REMOVE, or SYNCED once the listing's initial set was sent. + */ +export type TracksEvent = + | { kind: "added"; name: string; info: TrackInfo } + | { kind: "removed"; name: string } + | { kind: "synced" }; + +/** Write one {@link TracksEvent}: its type, then its size-prefixed body. */ +export async function encodeTracksEvent(w: Writer, event: TracksEvent, version: Version): Promise { + guardTracks(version); + switch (event.kind) { + case "added": + return Message.encode( + w, + async (w) => { + await w.string(event.name); + await event.info.encode(w, version); + }, + { id: TRACKS_ADD }, + ); + case "removed": + return Message.encode(w, (w) => w.string(event.name), { id: TRACKS_REMOVE }); + case "synced": + return Message.encode(w, async () => {}, { id: SYNCED }); + } +} + +/** Read the next {@link TracksEvent}, or undefined at the publisher's FIN. */ +export async function decodeTracksEventMaybe(r: Reader, version: Version): Promise { + guardTracks(version); + if (await r.done()) return undefined; + const typ = await r.u53(); + switch (typ) { + case TRACKS_ADD: + return Message.decode(r, async (r) => { + const name = await r.string(); + return { kind: "added", name, info: await TrackInfo.decode(r, version) }; + }); + case TRACKS_REMOVE: + return Message.decode(r, async (r) => ({ kind: "removed", name: await r.string() })); + case SYNCED: + return Message.decode(r, async () => ({ kind: "synced" })); + default: + // A new stream with no earlier builds to tolerate. + throw decodeError(new ProtocolViolation(`unknown tracks event: ${typ}`)); + } +} diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index 7ad0208cba6..d5bf9270844 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -118,6 +118,25 @@ export function hasAuth(version: Version): boolean { } } +/** + * Whether a subscriber may open a Tracks Stream (0x8) to list a broadcast's tracks. + * Added in lite-07. + */ +export function hasTracks(version: Version): boolean { + // Explicitly list older versions so future versions default to having the stream. + switch (version) { + case Version.DRAFT_01: + case Version.DRAFT_02: + case Version.DRAFT_03: + case Version.DRAFT_04: + case Version.DRAFT_05: + case Version.DRAFT_06: + return false; + default: + return true; + } +} + /** Whether announce streams begin with ANNOUNCE_OK and omit the sender's origin from each hop chain. */ export function hasAnnounceOk(version: Version): boolean { // Explicitly list older versions so future versions keep the lite-05+ announce behavior. diff --git a/js/net/src/origin.ts b/js/net/src/origin.ts index 36046dacdb1..66c328b3b52 100644 --- a/js/net/src/origin.ts +++ b/js/net/src/origin.ts @@ -1657,6 +1657,7 @@ export class Consumer { : new Derived([state.originated], (routes) => scope.projectRoutes(routes)), local: (path, epoch) => this.#local(scope.path(path), epoch), demand: (path, epoch, signal) => this.#demand(scope.path(path), epoch, signal), + covered: (path) => this.#covered(scope.path(path)), }); } @@ -2126,6 +2127,16 @@ export class Consumer { return local; } + /** + * Whether the originated route that would serve `path` claims it only through a shorter + * prefix. Such a route claims capability, not inventory, so it names no broadcast to list. + */ + #covered(path: Path.Valid): boolean { + if (this.#local(path)) return false; + const entry = this.#state.bestEntry(path, received); + return entry !== undefined && !this.#state.routes.peek()?.get(path)?.includes(entry); + } + /** * Resolve `path` for serving: an announced local broadcast, or wait for an originated * dynamic to accept it. Undefined when nothing here can serve the path. diff --git a/js/net/src/resume.ts b/js/net/src/resume.ts index afc76f21b2b..f43086e2952 100644 --- a/js/net/src/resume.ts +++ b/js/net/src/resume.ts @@ -39,6 +39,7 @@ import { type ResumeShell, type ResumeSource, } from "./internal.ts"; +import type { Lister, Tracks, TracksEvent } from "./listing.ts"; import * as track from "./track.ts"; import { type Broadcast as BroadcastWire, registerWire, wireOf } from "./wire.ts"; @@ -967,6 +968,111 @@ export class Front { constructor() { this.consumer = makeFrontConsumer(this); void this.consumer.closed.then(() => this.conclude()); + void this.#list(wireOf(this.consumer).lister()); + } + + /** + * Answer listings of the front's broadcast with its serving source's listing, while anyone + * lists it. A source of the same instance lists the same tracks, so a takeover shows readers + * only what differs, once the new source's listing synced. + */ + async #list(lister: Lister): Promise { + const closed = this.consumer.closed; + // What the front lists, the source being read, and its pending next event. + let listing: + | { + listed: Set; + /** Names an earlier source listed that the current one has yet to list. */ + stale: Set; + source?: Source; + reader?: Tracks; + next?: Promise<{ event?: TracksEvent; err?: Error }>; + } + | undefined; + const wait = () => race([Signal.race(lister.used, this.#changed), closed]); + const stop = () => { + listing?.reader?.close(); + listing = undefined; + }; + try { + for (;;) { + if (closed.peek() !== undefined) return; + if (!lister.used.peek()) { + if (listing) { + stop(); + lister.clear(); + } + await wait(); + continue; + } + if (this.#concluded) { + // No route will serve it again, so nothing more will be listed. + stop(); + lister.finish(); + continue; + } + listing ??= { listed: new Set(), stale: new Set() }; + if (listing.source !== this.#source) { + listing.reader?.close(); + listing.source = this.#source; + listing.stale = new Set(listing.listed); + listing.next = undefined; + try { + listing.reader = this.#source?.wire.tracks(); + } catch { + // A closing source lists nothing more; the front moves off it soon. + listing.reader = undefined; + } + } + const reader = listing.reader; + if (!reader) { + await wait(); + continue; + } + listing.next ??= reader.next().then( + (event) => ({ event }), + (err: unknown) => ({ err: err instanceof Error ? err : new Error(String(err)) }), + ); + const result = await race([listing.next, wait().then(() => undefined)]); + if (!result) continue; + listing.next = undefined; + const { event, err } = result; + if (err) { + // Its readers fail, and a later reader asks again. + stop(); + lister.abort(err); + } else if (!event) { + // A closing source ended its listing: the front moves to another source, or ends + // with it. A live one's handler ended it, as a session does once the peer's + // broadcast ended, so the front's listing ends too. + if (listing.source?.from.closed.peek() !== undefined) { + listing.reader = undefined; + } else { + stop(); + lister.finish(); + } + } else if (event.kind === "added") { + if (!listing.stale.delete(event.name) && !listing.listed.has(event.name)) { + listing.listed.add(event.name); + lister.add(event.name, event.info); + } + } else if (event.kind === "removed") { + listing.stale.delete(event.name); + if (listing.listed.delete(event.name)) lister.remove(event.name); + } else { + // What the new source still lacks at its SYNCED is gone. + for (const name of listing.stale) { + listing.listed.delete(name); + lister.remove(name); + } + listing.stale.clear(); + lister.synced(); + } + } + } finally { + stop(); + lister.close(); + } } /** Serve every track from `from` from now on, once each copy proves compatible. */ @@ -1266,6 +1372,8 @@ class FrontConsumer extends broadcast.Consumer { const view = wireOf(this); registerWire(this, { requested: view.requested, + tracks: view.tracks, + lister: view.lister, subscribe: (name, options) => front.subscribe(name, options), resolveTrackInfo: (name, hold) => front.resolveTrackInfo(name, hold), largest: (name) => front.largest(name), diff --git a/js/net/src/util/limits.test.ts b/js/net/src/util/limits.test.ts index bde91d6b7ab..9537a5de1ec 100644 --- a/js/net/src/util/limits.test.ts +++ b/js/net/src/util/limits.test.ts @@ -65,3 +65,26 @@ test("duplicate interests count individually, and a failed acquisition leaves no for (const held of [...tracks, ...subscribes, other]) held.release(); interests.acquire("path", "track", "subscribe").release(); }); + +test("each listing is charged on top of the interest under the same name", () => { + const interests = new Interests(new Slots(3, { reason: "full" })); + const track = interests.acquire("room", "", "track"); + const subscribe = interests.acquire("room", "", "subscribe"); + const listing = interests.acquire("room", "", "tracks"); + // A replacement listing must fit under the cap too, until the older is gone. + const replacement = interests.acquire("room", "", "tracks"); + expect(() => interests.acquire("room", "", "tracks")).toThrow(); + listing.release(); + replacement.release(); + track.release(); + subscribe.release(); + for (const held of [0, 1, 2].map(() => interests.acquire("room", "", "tracks"))) held.release(); +}); + +test("tryAcquire refuses at the cap without closing the session", () => { + const slots = new Slots(1, { reason: "full" }); + expect(slots.tryAcquire()).toBe(true); + expect(slots.tryAcquire()).toBe(false); + slots.release(); + expect(slots.tryAcquire()).toBe(true); +}); diff --git a/js/net/src/util/limits.ts b/js/net/src/util/limits.ts index 6259bc94b67..7c6cb6ec52b 100644 --- a/js/net/src/util/limits.ts +++ b/js/net/src/util/limits.ts @@ -28,12 +28,17 @@ export class Slots { /** Take one slot, or close the session with TOO_MANY_REQUESTS and throw at the cap. */ acquire() { - if (this.#live >= this.#max) { - const err = new Session(SessionCode.TooManyRequests, { reason: this.#reason }); - this.#quic?.close({ closeCode: err.code, reason: this.#reason }); - throw err; - } + if (this.tryAcquire()) return; + const err = new Session(SessionCode.TooManyRequests, { reason: this.#reason }); + this.#quic?.close({ closeCode: err.code, reason: this.#reason }); + throw err; + } + + /** Take one slot, or leave the session alone and return false at the cap. */ + tryAcquire(): boolean { + if (this.#live >= this.#max) return false; this.#live += 1; + return true; } /** Give back `count` slots. */ @@ -68,15 +73,21 @@ export class SlotMap extends Map { } } -type Kind = "track" | "subscribe"; -type Counts = { track: number; subscribe: number }; +type Kind = "track" | "subscribe" | "tracks"; +type Counts = Record; + +// A TRACK stream and its SUBSCRIBE are one interest; each listing is charged on top. +function cost(counts: Counts): number { + return Math.max(counts.track, counts.subscribe) + counts.tracks; +} /** * A session's subscription cap, charged per track as moq-net does. * * A TRACK stream held open and the SUBSCRIBE that follows it are one interest in the track, * so a track costs the larger of its two counts: a peer at the cap can still subscribe to - * the tracks it holds. + * the tracks it holds. A TRACKS listing is charged under the empty track name on top of + * that, each until it is gone, replaced or not. * * @internal */ @@ -91,9 +102,17 @@ export class Interests { /** Charge one request for `broadcast`'s `track` until the returned handle is released. */ acquire(broadcast: string, track: string, kind: Kind): Interest { let tracks = this.#tracks.get(broadcast); - const counts = tracks?.get(track) ?? { track: 0, subscribe: 0 }; - if (counts[kind] >= counts[kind === "track" ? "subscribe" : "track"]) this.#slots.acquire(); + const counts = tracks?.get(track) ?? { track: 0, subscribe: 0, tracks: 0 }; + const before = cost(counts); counts[kind] += 1; + if (cost(counts) > before) { + try { + this.#slots.acquire(); + } catch (err) { + counts[kind] -= 1; + throw err; + } + } if (!tracks) { tracks = new Map(); this.#tracks.set(broadcast, tracks); @@ -106,9 +125,10 @@ export class Interests { const tracks = this.#tracks.get(broadcast); const counts = tracks?.get(track); if (!tracks || !counts) throw new Error("interest already released"); - if (counts[kind] > counts[kind === "track" ? "subscribe" : "track"]) this.#slots.release(); + const before = cost(counts); counts[kind] -= 1; - if (counts.track === 0 && counts.subscribe === 0) { + if (cost(counts) < before) this.#slots.release(); + if (cost(counts) === 0) { tracks.delete(track); if (tracks.size === 0) this.#tracks.delete(broadcast); } diff --git a/js/net/src/wire.ts b/js/net/src/wire.ts index 1ae5b386c53..c378f71ad47 100644 --- a/js/net/src/wire.ts +++ b/js/net/src/wire.ts @@ -13,6 +13,7 @@ import type { Drain } from "./connection/goaway.ts"; import type * as Epoch from "./epoch.ts"; import type { Consumer as GroupConsumer } from "./group.ts"; import type { Route } from "./hop.ts"; +import type { Lister, Tracks } from "./listing.ts"; import type * as origin from "./origin.ts"; import type * as Path from "./path.ts"; import type * as track from "./track.ts"; @@ -31,6 +32,13 @@ export interface Broadcast { fetchGroup(name: string, sequence: number, options?: track.FetchGroupOptions): Promise; // pending: /quest/m1/public-model/answer.md requested(): Promise; + // private: capability gap + // Sessions dedup a peer's listings per broadcast; a custom session has no public owner yet. + /** List the broadcast's tracks for `owner`, ending its older listing of this broadcast with CANCELLED. */ + tracks(owner?: object): Tracks; + // pending: /quest/m1/public-model/answer.md + /** Answer the broadcast's listings, as a session does with its peer's; throws while another answers. */ + lister(): Lister; // pending: /quest/m1/public-model/answer.md /** * A session delivered this broadcast. An originated route refuses to serve it: @@ -98,6 +106,9 @@ export interface OriginConsumer { // pending: /quest/m1/public-model/answer.md /** Resolve `path` for serving; throws `Unroutable` when the route serves another `epoch`. */ demand(path: Path.Valid, epoch?: Epoch.Valid, signal?: AbortSignal): Promise; + // pending: /quest/m1/public-model/answer.md + /** Whether the route serving `path` claims it only through a shorter prefix, so it names no broadcast to list. */ + covered(path: Path.Valid): boolean; } // pending: /quest/m1/public-model/answer.md diff --git a/quest/m1/archive/README.md b/quest/m1/archive/README.md index 1d940249f59..cdb5695c572 100644 --- a/quest/m1/archive/README.md +++ b/quest/m1/archive/README.md @@ -28,8 +28,8 @@ archive (per-track `.timeline.z` records named by the catalog's root - **Discovery is pull**, decided 2026-10-19 in place of SUBSCRIBE_ALL's push: ANNOUNCE finds the broadcasts under a prefix, a moq-lite TRACKS request lists each broadcast's tracks as they appear, and the archive SUBSCRIBEs per - track, gating the lite-07 cut (`broadcast::Consumer::tracks` in Rust; - [TRACKS in js/net](/quest/m1/archive/tracks-js.md)). + track, gating the lite-07 cut (`broadcast::Consumer::tracks` in Rust, + `broadcast.Consumer.tracks()` in JS). moq-transport answers SUBSCRIBE_TRACKS through the same hook and learns an IETF peer's tracks with a FORWARD=0 SUBSCRIBE_TRACKS ([SUBSCRIBE_TRACKS](/quest/m1/archive/subscribe-tracks.md)). Rejected: @@ -84,7 +84,6 @@ from a host. Each child fixes the stale docs it touches. ## Required -- [TRACKS in js/net](/quest/m1/archive/tracks-js.md) - `@moq/net` lists a broadcast's tracks over lite-07's Tracks Stream and answers listings of its own broadcasts, mirroring Rust - [Archive producer](/quest/m1/archive/producer.md) - `archive::Producer` stores every matching broadcast as duration-cut per-track objects keyed by group and pts, recording gaps instead of stalling, with an optional DVR mode - [Archive consumer](/quest/m1/archive/consumer.md) - `archive::Consumer` serves a stored prefix as a broadcast from its key listing, and `moq export/import archive` use the new pair - [Catalog HLS field](/quest/m1/archive/catalog-hls.md) - the root `archive` section is gone; an opt-in per-rendition `hls` field marks each aligned HLS segment's first group and pts and declares its nominal duration diff --git a/quest/m1/archive/tracks-js.md b/quest/m1/archive/tracks-js.md deleted file mode 100644 index aaf1a14b5e6..00000000000 --- a/quest/m1/archive/tracks-js.md +++ /dev/null @@ -1,38 +0,0 @@ -# [M] TRACKS in js/net - -## Goal - -`@moq/net` speaks the moq-lite-07 Tracks Stream (0x8) that Rust already -speaks, with the same behavior and mirrored names: - -- **Listing.** A broadcast consumer lists its tracks (Rust - `broadcast::Consumer::tracks`): each listed track with its info, one - `synced` event, then adds and removals until the broadcast ends. Over a - session it opens one TRACKS per broadcast however many readers list it, - syncs its readers at once, and forwards the peer's tracks as live adds. -- **Answering.** A session answers a peer's TRACKS for a broadcast it - publishes with the tracks its producer created (never one served only on - request), using the per-reader queue and SYNCED rule the draft specifies. -- **Refusals.** Older versions reset the stream type, as now. A covering - prefix route answers NOT_FOUND. A listing past the session's announce cap - is reset. - -## Plan - -The wire and the Rust model landed first; read the Tracks section of -`drafts/draft-lcurley-moq-lite.md` and mirror `rs/moq-net/src/model/listing.rs` -(the queue keyed by name in insertion order, where an unread add and removal -cancel and a removal then re-add of a seen name collapses in place) and -`rs/moq-net/src/lite/tracks.rs` (the encoding: TRACKS_ADD 0x0 with an -embedded TRACK_INFO, TRACKS_REMOVE 0x1, SYNCED 0x4). - -`js/net/src/lite/malformed.test.ts` still treats type 8 as unknown on -lite-07; it becomes a real stream here. - -Test the same cases as Rust: snapshot then SYNCED, SYNCED after N events -under churn, a stalled reader's bounded queue, removal before SYNCED, a -dynamic-only track never listed, the broadcast ending the listing, and -`just test interop --all` against the Rust side in both directions. - -Public API: a track-listing method on the JS broadcast consumer, named like -Rust's. Wire: none new. diff --git a/quest/m1/lite07-finalize.md b/quest/m1/lite07-finalize.md index 4c35225c53d..3fcf75b7c67 100644 --- a/quest/m1/lite07-finalize.md +++ b/quest/m1/lite07-finalize.md @@ -89,7 +89,6 @@ is published; older versions are unchanged. - [Announcement takeover](/quest/m1/announce-takeover.md) - ANNOUNCE_TAKEOVER preserves a proven instance under a new independent route - [Compressed tracks in Rust](/quest/m1/compress/core.md) - TRACK_INFO carries the compress flag and the frame header each compressed frame's inflated size - [Compressed tracks in js/net](/quest/m1/compress/js.md) - JS reads and writes compressed lite-07 tracks instead of refusing them -- [TRACKS in js/net](/quest/m1/archive/tracks-js.md) - JS speaks lite-07's Tracks Stream, which Rust already does - [Ingress-version provenance](/quest/m1/ietf-extension/lite.md) - the `ietf` extension carries a track's ingress version once instead of a code per block ## Related diff --git a/rs/moq-net/src/test_interop.rs b/rs/moq-net/src/test_interop.rs index a72b0dd1480..1c32eb2b6c8 100644 --- a/rs/moq-net/src/test_interop.rs +++ b/rs/moq-net/src/test_interop.rs @@ -369,6 +369,66 @@ fn lite_auth_unsupported_interop() { assert!(reader.is_empty()); } +/// moq-lite-07 TRACKS across the Rust/JS boundary: the JS publisher answers a Rust TRACKS request +/// with a listing Rust decodes, and JS decodes Rust's listing events and re-encodes them byte for +/// byte. +#[test] +#[ignore = "requires Bun; run by just test interop"] +fn lite_tracks_interop() { + use crate::coding::Decode; + + let version = lite::Version::Lite07; + let request = lite::Tracks { + broadcast: crate::Path::new("room"), + epoch: None, + } + .encode_bytes(version) + .unwrap(); + let input = serde_json::json!({ "mode": "answer", "bytes": request.to_vec() }); + let answer: Vec = serde_json::from_slice(&bun("tracks.ts", input)).expect("JS returned its listing"); + let mut r = Decoder::new(&answer, version.into()); + let mut events = Vec::new(); + while !r.is_empty() { + events.push(match lite::TracksEvent::decode(&mut r, version).unwrap() { + lite::TracksEvent::Added { name, info } => { + if name == "a" { + assert_eq!(info.timescale, Some(crate::Timescale::MILLI)); + assert_eq!(info.max_age, Some(std::time::Duration::from_secs(3))); + assert_eq!(info.priority, 9); + assert_eq!(info.refresh, Some(4)); + } + format!("+{name}") + } + lite::TracksEvent::Removed { name } => format!("-{name}"), + lite::TracksEvent::Synced => "synced".to_string(), + }); + } + // The broadcast's end is the JS publisher's FIN, after the last event. + assert_eq!(events, ["+a", "+b", "synced", "-a"]); + + let info = lite::TrackInfo { + refresh: Some(4), + priority: 3, + max_age: Some(std::time::Duration::from_secs(2)), + timescale: Some(crate::Timescale::MICRO), + }; + let mut bytes = Vec::new(); + let mut w = Encoder::new(&mut bytes, version.into()); + for event in [ + lite::TracksEvent::Added { + name: "video".into(), + info, + }, + lite::TracksEvent::Removed { name: "video".into() }, + lite::TracksEvent::Synced, + ] { + event.encode(&mut w, version).unwrap(); + } + let input = serde_json::json!({ "mode": "echo", "bytes": bytes }); + let echo: Vec = serde_json::from_slice(&bun("tracks.ts", input)).expect("JS echoed the events"); + assert_eq!(echo, bytes); +} + /// TRACK_STATUS on every draft: the JS publisher decodes what Rust encodes and answers with a /// TRACK_STATUS_OK the Rust subscriber reads, carrying what a SUBSCRIBE_OK would as far as the /// draft has room, or refuses a missing broadcast DOES_NOT_EXIST. diff --git a/test/interop/README.md b/test/interop/README.md index bd8c4aa7a8d..1e972fca249 100644 --- a/test/interop/README.md +++ b/test/interop/README.md @@ -384,6 +384,12 @@ match byte for byte and decode back to the same value. NOT\_SUPPORTED across Rust and JS in both directions, checking that the later refusal retains its meaning and both bindings emit the same bytes. +`lite_tracks_interop` checks moq-lite-07's Tracks Stream both ways: `tracks.ts` +answers a Rust TRACKS request with js/net's publisher, whose listing (each +TRACKS\_ADD with its TRACK\_INFO, SYNCED, a TRACKS\_REMOVE, then the FIN when +the broadcast ends) Rust decodes, and it re-encodes Rust's listing events byte for +byte. + `lite_varint_interop` runs next to it and does the same through moq-lite's version dispatch: `lite-varint.ts` decodes Rust's lite-06 (QUIC) and lite-07 (leading-ones) varints, a SETUP carrying a 62-bit Hop ID, a datagram, and a diff --git a/test/interop/tracks.ts b/test/interop/tracks.ts new file mode 100644 index 00000000000..bc32f3d4170 --- /dev/null +++ b/test/interop/tracks.ts @@ -0,0 +1,63 @@ +// moq-lite-07 TRACKS both ways. `answer`: the JS publisher answers a Rust-encoded TRACKS request +// with its listing, through a removal and the broadcast's end. `echo`: JS decodes Rust-encoded +// listing events and encodes them back. +import { Milli } from "@moq/clock"; +import { randomHop } from "../../js/net/src/hop.ts"; +import { Publisher } from "../../js/net/src/lite/publisher.ts"; +import { decodeTracksEventMaybe, encodeTracksEvent, Tracks } from "../../js/net/src/lite/tracks.ts"; +import { Version } from "../../js/net/src/lite/version.ts"; +import { createMockTransportPair } from "../../js/net/src/mock.ts"; +import { Producer as OriginProducer } from "../../js/net/src/origin.ts"; +import * as Path from "../../js/net/src/path.ts"; +import { Reader, Stream, Writer } from "../../js/net/src/stream.ts"; +import { Timescale } from "../../js/net/src/time.ts"; + +// Stdout is the byte channel back to Rust. +console.debug = console.error; +console.info = console.error; +const input: { mode: "answer" | "echo"; bytes: number[] } = JSON.parse(process.argv[2]); +const version = Version.DRAFT_07; +const bytes = Uint8Array.from(input.bytes); +const written: number[] = []; +const settle = () => new Promise((resolve) => setTimeout(resolve, 0)); + +if (input.mode === "answer") { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a", { timescale: Timescale.MILLI, maxAge: Milli(3000), priority: 9, refresh: 4 }); + broadcast.createTrack("b"); + + const publisher = new Publisher(createMockTransportPair().server, version, randomHop(), origin.consume()); + const stream = new Stream({ + // The requester holds its side open while it lists: a FIN would cancel the listing. + readable: new ReadableStream({ start: (controller) => controller.enqueue(bytes) }), + writable: new WritableStream({ write: (chunk) => void written.push(...chunk) }), + version, + }); + const msg = await Tracks.decode(stream.reader, version); + if (msg.broadcast !== Path.from("room")) throw new Error(`unexpected broadcast: ${msg.broadcast}`); + const listing = publisher.runTracks(msg, stream); + // Past SYNCED before the removal, so the removal is not folded into the initial set. + await settle(); + broadcast.removeTrack("a"); + await settle(); + broadcast.close(); + await listing; + origin.close(); +} else { + const reader = new Reader(undefined, bytes, version); + const writer = new Writer( + new WritableStream({ write: (chunk) => void written.push(...chunk) }), + version, + ); + for (;;) { + const event = await decodeTracksEventMaybe(reader, version); + if (!event) break; + await encodeTracksEvent(writer, event, version); + } + writer.close(); + await writer.closed; +} + +process.stdout.write(JSON.stringify(written)); diff --git a/test/justfile b/test/justfile index 51eb3ecf2a9..6c69dbc2774 100644 --- a/test/justfile +++ b/test/justfile @@ -38,10 +38,11 @@ harness: # Both runs first check js/net's varints against moq-net's encoder at every size boundary, # and on lite-06/07 messages (the filter also matches `lite_varint_interop`), then # moq-transport OBJECT_DATAGRAMs both ways and the JS answer to TRACK_STATUS on every -# draft, and Lite AUTH_ERROR NOT_SUPPORTED after a grant in both directions. +# draft, Lite AUTH_ERROR NOT_SUPPORTED after a grant in both directions, and lite-07 TRACKS +# listings in both directions. [positional-arguments] interop *args: - just rs test -p moq-net --lib varint_interop ietf_datagram_interop ietf_track_status_interop auth_unsupported_interop --run-ignored all + just rs test -p moq-net --lib varint_interop ietf_datagram_interop ietf_track_status_interop auth_unsupported_interop tracks_interop --run-ignored all ./interop/interop.sh "$@" # Nightly main vs newest installable registry releases: tokens, moq-lite session From e84f7472fdb0de37f8e34ad743b7171d366260fb Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 23:06:15 -0700 Subject: [PATCH 03/14] fix(js/net): refuse listings through a covering prefix route at the origin A route claiming a prefix names no broadcast, so the handle a dynamic route serves beneath its prefix lists nothing: tracks() throws NotFound on it and every clone, and an epoch-pinned front serving it aborts its listing with NotFound, as Rust's front does. Local reads now match sessions, and the publisher's separate covered() check is gone. Co-Authored-By: Claude Opus 5.5 --- js/net/src/broadcast.ts | 23 ++++++++++++-- js/net/src/internal.ts | 5 ++++ js/net/src/lite/publisher.ts | 3 -- js/net/src/lite/tracks.test.ts | 55 +++++++++++++++++++++++++++++++++- js/net/src/origin.ts | 21 +++++-------- js/net/src/resume.ts | 9 +++++- js/net/src/wire.ts | 3 -- 7 files changed, 95 insertions(+), 24 deletions(-) diff --git a/js/net/src/broadcast.ts b/js/net/src/broadcast.ts index c102b1a8ee0..6fb7aee7ca3 100644 --- a/js/net/src/broadcast.ts +++ b/js/net/src/broadcast.ts @@ -509,6 +509,7 @@ interface Shared { state: BroadcastState; path: Path.Valid; epoch?: Epoch.Valid; + unlisted?: boolean; } // Constructs a Consumer from within this module without exposing a public constructor @@ -527,6 +528,8 @@ export class Consumer { #state: BroadcastState; #path: Path.Valid; #epoch?: Epoch.Valid; + // Served on request through a route claiming a shorter prefix, which names no broadcast to list. + #unlisted: boolean; // Guards against a double close() on this handle over-decrementing the consumer count. #closed = false; @@ -535,6 +538,7 @@ export class Consumer { protected constructor(shared?: Shared) { this.#path = shared?.path ?? Path.empty(); this.#epoch = shared?.epoch; + this.#unlisted = shared?.unlisted ?? false; if (shared) { this.#state = shared.state; } else { @@ -549,7 +553,7 @@ export class Consumer { largest: (name) => largest(this.#state, name), fetchGroup: (name, sequence, options) => fetchGroup(this.#state, name, sequence, options), requested: () => requested(this.#state), - tracks: (owner) => listTracks(this.#state.listing, owner), + tracks: (owner) => this.#tracks(owner), lister: () => new Lister(this.#state.listing), }); } @@ -563,6 +567,9 @@ export class Consumer { hooks.stampEpoch = (target, epoch) => { target.#epoch = epoch; }; + hooks.stampUnlisted = (target) => { + target.#unlisted = true; + }; } /** @@ -614,7 +621,12 @@ export class Consumer { // Hand this consumer's backing state and path to a clone. Opaque (`never`) so the state type // stays unexported; a subclass passes it straight back into its own `super(...)`. protected shareState(): never { - return { state: this.#state, path: this.#path, epoch: this.#epoch } satisfies Shared as never; + return { + state: this.#state, + path: this.#path, + epoch: this.#epoch, + unlisted: this.#unlisted, + } satisfies Shared as never; } /** Get a lazy handle for a track on this broadcast. Repeat subscriptions dedupe onto one upstream subscription. */ @@ -634,7 +646,12 @@ export class Consumer { * has closed. */ tracks(): Tracks { - return listTracks(this.#state.listing); + return this.#tracks(); + } + + #tracks(owner?: object): Tracks { + if (this.#unlisted) throw new NotFound("a route claiming a prefix lists no tracks"); + return listTracks(this.#state.listing, owner); } /** diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index c342ed8218a..49ddf76f717 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -496,6 +496,8 @@ export const hooks: { stampPath: (target: BroadcastProducer | BroadcastConsumer, path: Path.Valid) => void; /** Name the epoch of the route an origin resolved a broadcast handle through. */ stampEpoch: (target: BroadcastConsumer, epoch: Epoch.Valid | undefined) => void; + /** Mark a broadcast handle served through a route claiming a shorter prefix: it lists nothing. */ + stampUnlisted: (target: BroadcastConsumer) => void; } = { receiveTrack: () => { throw new Error("track.ts not loaded"); @@ -647,6 +649,9 @@ export const hooks: { stampEpoch: () => { throw new Error("broadcast.ts not loaded"); }, + stampUnlisted: () => { + throw new Error("broadcast.ts not loaded"); + }, }; /** diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index 8ede3954838..0ab44fd0ff8 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -1453,9 +1453,6 @@ export class Publisher { try { if (watch.revoked()) throw unauthorized(msg.broadcast); const origin = this.#publish && wireOf(this.#publish); - // A route claiming a prefix names no broadcast, so a path it serves on request has - // nothing to list. - if (origin?.covered(msg.broadcast)) throw new NotFound(`broadcast ${msg.broadcast}`); const front = origin && (origin.local(msg.broadcast, msg.epoch) ?? diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts index b48c856e9b5..e69f9a573a5 100644 --- a/js/net/src/lite/tracks.test.ts +++ b/js/net/src/lite/tracks.test.ts @@ -3,7 +3,7 @@ import { Milli } from "@moq/clock"; import { Producer as BroadcastProducer, type Tracks } from "../broadcast.ts"; import { accept, connect } from "../connection/index.ts"; import * as Epoch from "../epoch.ts"; -import { fromTransport, SessionCode, StreamCode, StreamError } from "../error.ts"; +import { fromTransport, NotFound, SessionCode, StreamCode, StreamError } from "../error.ts"; import { randomHop } from "../hop.ts"; import { createMockTransportPair } from "../mock.ts"; import { Producer as OriginProducer } from "../origin.ts"; @@ -337,6 +337,38 @@ test("a path served through a covering prefix route is NOT_FOUND", async () => { dynamic.close(); }); +test("a local read through a covering prefix route is NOT_FOUND, and through its exact route lists", async () => { + const origin = new OriginProducer(); + for (const prefix of ["room", "room/bob"]) { + const dynamic = origin.dynamic(Path.from(prefix)); + void (async () => { + for await (const request of dynamic.requested()) { + const served = new BroadcastProducer(); + served.createTrack("video"); + request.accept(served); + } + })(); + } + const consumer = origin.consume(); + const alice = consumer.request(Path.from("room/alice")); + const bob = consumer.request(Path.from("room/bob")); + await settle(); + + const covered = alice.active.peek(); + if (!covered) throw new Error("expected the covering route to serve"); + expect(() => covered.tracks()).toThrow(NotFound); + // Every handle cloned from it refuses too. + expect(() => covered.clone().tracks()).toThrow(NotFound); + + const exact = bob.active.peek(); + if (!exact) throw new Error("expected the exact route to serve"); + const tracks = exact.tracks(); + expect(await next(tracks)).toBe("+video"); + expect(await next(tracks)).toBe("synced"); + tracks.close(); + origin.close(); +}); + test("a listing over a session carries each track's info, end to end", async () => { const pair = createMockTransportPair(ALPN_07_WIP); const origin = new OriginProducer(); @@ -398,6 +430,27 @@ test("an epoch-pinned front lists its serving route's tracks", async () => { server.abort(); }); +test("an epoch-pinned front served through a covering prefix route is NOT_FOUND", async () => { + const origin = new OriginProducer(); + const dynamic = origin.dynamic(Path.from("room"), { epoch: Epoch.parse("01900000-0000-7000-8000-000000000001") }); + void (async () => { + for await (const request of dynamic.requested()) { + const served = new BroadcastProducer(); + served.createTrack("video"); + request.accept(served); + } + })(); + const request = origin.consume().request(Path.from("room/alice")); + await settle(); + const front = request.active.peek(); + if (!front) throw new Error("expected the covering route to serve"); + expect(await next(front.tracks())).toBe(`error: ${StreamCode.NotFound}`); + // A later reader asks again, and is refused again. + expect(await next(front.tracks())).toBe(`error: ${StreamCode.NotFound}`); + request.close(); + origin.close(); +}); + test("lite-06 cannot list over a session", async () => { const pair = createMockTransportPair(ALPN_06); const origin = new OriginProducer(); diff --git a/js/net/src/origin.ts b/js/net/src/origin.ts index 66c328b3b52..61a01f3ecb1 100644 --- a/js/net/src/origin.ts +++ b/js/net/src/origin.ts @@ -376,11 +376,14 @@ function unroutable(): StreamError { /** A served route from {@link Producer.dynamic}: the queue a handler drains. */ class ServeState { readonly root: Path.Valid; + /** The prefix the route claims; every served path is at or beneath it. */ + readonly prefix: Path.Valid; /** False when a session announced the route: it may serve that session's broadcast. */ readonly originated: boolean; - constructor(root: Path.Valid, originated: boolean) { + constructor(root: Path.Valid, prefix: Path.Valid, originated: boolean) { this.root = root; + this.prefix = prefix; this.originated = originated; } @@ -437,6 +440,9 @@ class ServeState { this.settled.update((n) => n + 1); return; } + // A route claiming a prefix names no broadcast, so a path it serves on request has no + // inventory to list. + if (path !== this.prefix) hooks.stampUnlisted(front); this.served.set(path, front); void front.closed.then(() => { if (this.served.get(path) !== front) return; @@ -1326,7 +1332,7 @@ export class Producer implements Table { #insertRoute(prefix: Path.Valid, route: Route, originated: boolean): Dynamic { prefix = this.#scope.prefix(prefix); - const server = new ServeState(this.#scope.root, originated); + const server = new ServeState(this.#scope.root, prefix, originated); server.onChange = (path) => this.#state.refresh(path); const entry: RouteEntry = { identity: {}, @@ -1657,7 +1663,6 @@ export class Consumer { : new Derived([state.originated], (routes) => scope.projectRoutes(routes)), local: (path, epoch) => this.#local(scope.path(path), epoch), demand: (path, epoch, signal) => this.#demand(scope.path(path), epoch, signal), - covered: (path) => this.#covered(scope.path(path)), }); } @@ -2127,16 +2132,6 @@ export class Consumer { return local; } - /** - * Whether the originated route that would serve `path` claims it only through a shorter - * prefix. Such a route claims capability, not inventory, so it names no broadcast to list. - */ - #covered(path: Path.Valid): boolean { - if (this.#local(path)) return false; - const entry = this.#state.bestEntry(path, received); - return entry !== undefined && !this.#state.routes.peek()?.get(path)?.includes(entry); - } - /** * Resolve `path` for serving: an announced local broadcast, or wait for an originated * dynamic to accept it. Undefined when nothing here can serve the path. diff --git a/js/net/src/resume.ts b/js/net/src/resume.ts index f43086e2952..3b19de382e5 100644 --- a/js/net/src/resume.ts +++ b/js/net/src/resume.ts @@ -1019,9 +1019,16 @@ export class Front { listing.next = undefined; try { listing.reader = this.#source?.wire.tracks(); - } catch { + } catch (err) { // A closing source lists nothing more; the front moves off it soon. listing.reader = undefined; + // A live one refuses: served through a route claiming a prefix, it names + // nothing to list. A later reader asks again. + if (listing.source && listing.source.from.closed.peek() === undefined) { + stop(); + lister.abort(err instanceof Error ? err : new Error(String(err))); + continue; + } } } const reader = listing.reader; diff --git a/js/net/src/wire.ts b/js/net/src/wire.ts index c378f71ad47..709302ca558 100644 --- a/js/net/src/wire.ts +++ b/js/net/src/wire.ts @@ -106,9 +106,6 @@ export interface OriginConsumer { // pending: /quest/m1/public-model/answer.md /** Resolve `path` for serving; throws `Unroutable` when the route serves another `epoch`. */ demand(path: Path.Valid, epoch?: Epoch.Valid, signal?: AbortSignal): Promise; - // pending: /quest/m1/public-model/answer.md - /** Whether the route serving `path` claims it only through a shorter prefix, so it names no broadcast to list. */ - covered(path: Path.Valid): boolean; } // pending: /quest/m1/public-model/answer.md From c7eb3f5cbd184c2e17287967b8577f664524e935 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 23:15:18 -0700 Subject: [PATCH 04/14] fix(js/net): yield while draining a peer's TRACKS events A peer can buffer an unbounded churn of adds and removals of one name, which never reaches the announces cap and decodes without a task boundary. Poll a cooperative budget before each event, as stream writes do, so the flood cannot starve the host. Co-Authored-By: Claude Opus 5.5 --- js/net/src/lite/subscriber.ts | 7 +++++++ js/net/src/lite/tracks.test.ts | 31 +++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index a410b6163da..3fc3b1a6e3e 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -35,6 +35,7 @@ import { untilAborted } from "../util/abort.ts"; import { Deadline, REQUEST_TIMEOUT_MS, requestOptions } from "../util/deadline.ts"; import { abandoned, isAbandoned, unwanted } from "../util/demand.ts"; import { LIMITS, type Limits, SlotMap, Slots } from "../util/limits.ts"; +import { Budget } from "../util/yield.ts"; import { overrideBroadcastWire, wireOf } from "../wire.ts"; import { AnnounceHistory, @@ -79,6 +80,10 @@ import { waitsForSubscriberFin, } from "./version.ts"; +// A peer's buffered TRACKS events decode without reaching a task boundary, and a flood of adds +// and removals of one name never reaches the announces cap: yield so it cannot starve the host. +const listingBudget = new Budget(); + // Request streams refuse exhausted peer credit, with one deadline covering create and answer. export const SUBSCRIBE_SETUP_TIMEOUT_MS = REQUEST_TIMEOUT_MS; @@ -704,6 +709,8 @@ export class Subscriber { await new TracksMessage(broadcast, consumed.epoch).encode(stream.writer, this.version); let synced = false; for (;;) { + const pause = listingBudget.poll(); + if (pause !== undefined) await untilAborted(pause, stop); const event = await untilAborted(decodeTracksEventMaybe(stream.reader, this.version), stop); if (!event) { // The peer's broadcast ended, whether or not its retraction has arrived: end the diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts index e69f9a573a5..a39524847d4 100644 --- a/js/net/src/lite/tracks.test.ts +++ b/js/net/src/lite/tracks.test.ts @@ -240,6 +240,37 @@ test("a listing past the announces cap is reset, and the session stays", async ( again.close(); }); +test("a flood of a peer's listing churn yields to the host's tasks", async () => { + const { subscriber, listed } = handPlayed(); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + // Churn of one name never reaches the announces cap, so only the budget bounds it. + const flood = await bytes(async (w) => { + await encodeTracksEvent(w, { kind: "added", name: "first", info }, version); + for (let i = 0; i < 20_000; i++) { + await encodeTracksEvent(w, { kind: "added", name: "churn", info }, version); + await encodeTracksEvent(w, { kind: "removed", name: "churn" }, version); + } + await encodeTracksEvent(w, { kind: "added", name: "last", info }, version); + }); + await peer.stream.writer.write(flood); + expect(await next(tracks)).toBe("synced"); + expect(await next(tracks)).toBe("+first"); + // The receiver is mid-flood: a task queued now runs before it reaches the end. + let last = false; + let ranFirst: boolean | undefined; + setTimeout(() => { + ranFirst = !last; + }, 0); + let event: string; + do event = await next(tracks); + while (event !== "+last"); + last = true; + await settle(); + expect(ranFirst).toBe(true); + tracks.close(); +}); + test("a version without the Tracks Stream refuses the listing", async () => { const pair = createMockTransportPair(); const subscriber = new Subscriber(pair.client, Version.DRAFT_06, randomHop()); From e97b166de8ffeb5ddf81304471fac1cd416448a4 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 20 Oct 2026 23:21:51 -0700 Subject: [PATCH 05/14] fix(js/net): close the session on data after TRACKS, and FIN a finished listing Bytes after a TRACKS request are a protocol violation that closes the session, as in Rust, rather than only resetting the stream. A listing the peer ended with FIN now FINs the requester's half too, so the stream does not linger half-open holding stream credit. Co-Authored-By: Claude Opus 5.5 --- js/net/src/lite/publisher.ts | 9 +++++++-- js/net/src/lite/subscriber.ts | 3 ++- js/net/src/lite/tracks.test.ts | 20 +++++++++++++++++++- 3 files changed, 28 insertions(+), 4 deletions(-) diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index 0ab44fd0ff8..59969531f6e 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -1444,8 +1444,13 @@ export class Publisher { // TRACKS is the requester's only message: its FIN or reset ends the listing. void stream.reader.done().then( (fin) => { - if (!fin) stream.abort(new ProtocolViolation("data after TRACKS")); - cancel.abort(new StreamError(StreamCode.Cancel, { message: "listing cancelled" })); + if (fin) { + cancel.abort(new StreamError(StreamCode.Cancel, { message: "listing cancelled" })); + return; + } + const violation = new ProtocolViolation("data after TRACKS"); + closeViolation(this.#quic, violation); + cancel.abort(violation); }, (err: unknown) => cancel.abort(error(err)), ); diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 3fc3b1a6e3e..61729d910bf 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -714,7 +714,8 @@ export class Subscriber { const event = await untilAborted(decodeTracksEventMaybe(stream.reader, this.version), stop); if (!event) { // The peer's broadcast ended, whether or not its retraction has arrived: end the - // listing and let go of what it listed. + // listing and let go of what it listed. FIN our half too, so the stream is done. + stream.writer.close(); lister.finish(); return; } diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts index a39524847d4..bcf1f812c75 100644 --- a/js/net/src/lite/tracks.test.ts +++ b/js/net/src/lite/tracks.test.ts @@ -153,6 +153,8 @@ test("a session syncs at once and forwards the peer's tracks as live adds", asyn peer.stream.writer.close(); expect(await next(tracks)).toBe("end"); expect(remote.closed.peek()).toBeUndefined(); + // The requester FINs its half too, so the stream does not linger half-open. + expect(await peer.stream.reader.done()).toBe(true); }); test("every listing of a broadcast shares one TRACKS stream, cancelled once unused", async () => { @@ -281,6 +283,7 @@ test("a version without the Tracks Stream refuses the listing", async () => { /** A lite-07 publisher serving `origin`, answering TRACKS streams the test writes by hand. */ function publisher(origin: OriginProducer) { const pair = createMockTransportPair(ALPN_07_WIP); + const closing = spyOn(pair.server, "close"); const publisher = new Publisher(pair.server, version, randomHop(), origin.consume()); const list = async (broadcast: string) => { const requester = await Stream.open(pair.client, { version }); @@ -309,9 +312,24 @@ function publisher(origin: OriginProducer) { }, }; }; - return { list }; + return { list, closing }; } +test("data after a TRACKS request closes the session", async () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room")); + broadcast.announce(); + broadcast.createTrack("a"); + const { list, closing } = publisher(origin); + const listing = await list("room"); + expect(await listing.next()).toBe("+a"); + expect(await listing.next()).toBe("synced"); + await listing.requester.writer.u53(0); + expect(await listing.next()).toMatch(/^reset: /); + expect(closing).toHaveBeenCalledWith(expect.objectContaining({ closeCode: SessionCode.ProtocolViolation })); + broadcast.close(); +}); + test("a publisher lists its broadcast's own tracks, never one served on request", async () => { const origin = new OriginProducer(); const broadcast = origin.createBroadcast(Path.from("room")); From c4af6830204aee8934dea9275570d04c5c2076dc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 08:11:04 -0700 Subject: [PATCH 06/14] test(net): set the TRACK_INFO compression codec in the TRACKS interop echo main added TrackInfo::compress; TRACKS is lite-07 only, so the echo carries the codec byte across Rust and JS. Co-Authored-By: Claude Opus 5.5 --- rs/moq-net/src/test_interop.rs | 1 + 1 file changed, 1 insertion(+) diff --git a/rs/moq-net/src/test_interop.rs b/rs/moq-net/src/test_interop.rs index a3eca35b492..ce497fc9a0b 100644 --- a/rs/moq-net/src/test_interop.rs +++ b/rs/moq-net/src/test_interop.rs @@ -415,6 +415,7 @@ fn lite_tracks_interop() { priority: 3, max_age: Some(std::time::Duration::from_secs(2)), timescale: Some(crate::Timescale::MICRO), + compress: true, }; let mut bytes = Vec::new(); let mut w = Encoder::new(&mut bytes, version.into()); From e93d1d4a442b7294a08a8262ad48fb9ae01a7dce Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 08:56:58 -0700 Subject: [PATCH 07/14] fix(js/net): drop a listing answer from a source replaced as it arrived Co-Authored-By: Claude Opus 5.5 --- js/net/src/resume.test.ts | 39 +++++++++++++++++++++++++++++++++++++++ js/net/src/resume.ts | 2 ++ 2 files changed, 41 insertions(+) diff --git a/js/net/src/resume.test.ts b/js/net/src/resume.test.ts index af0636635ff..cfa43a5ed94 100644 --- a/js/net/src/resume.test.ts +++ b/js/net/src/resume.test.ts @@ -4,6 +4,7 @@ import { Producer as BroadcastProducer } from "./broadcast.ts"; import * as Epoch from "./epoch.ts"; import { NotFound, StreamCode, StreamError } from "./error.ts"; import { Producer as GroupProducer } from "./group.ts"; +import type { Lister } from "./listing.ts"; import { Route } from "./hop.ts"; import { hooks } from "./internal.ts"; import { Producer } from "./origin.ts"; @@ -1610,3 +1611,41 @@ test.each([Milli(100), Milli(3_600_000)])( } }, ); + +test.each([ + ["fails", (answer: Lister) => answer.abort(new Error("old route failed"))], + ["ends", (answer: Lister) => answer.finish()], + ["changes", (answer: Lister) => answer.add("gone", { priority: 0 })], +])("a listing follows the source that took over as the old one's listing %s", async (_, end) => { + const front = new Front(); + const a = new BroadcastProducer(); + const b = new BroadcastProducer(); + const from = a.consume(); + // The old source's listing is answered, as a session answers its peer's. + const answer = wireOf(from).lister(); + answer.synced(); + b.createTrack("seq"); + front.serve(from); + const tracks = front.consumer.tracks(); + try { + expect(await tracks.next()).toMatchObject({ kind: "synced" }); + + // The new source takes over just as the old source's listing settles, which a later + // reader of it sees at the same moment the front's reader does. + const witness = from.tracks(); + expect(await witness.next()).toMatchObject({ kind: "synced" }); + const takeover = () => front.serve(b.consume()); + const settled = witness.next().then(takeover, takeover); + end(answer); + await settled; + expect(await tracks.next()).toMatchObject({ kind: "added", name: "seq" }); + b.createTrack("late"); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "late" }); + } finally { + tracks.close(); + answer.close(); + front.consumer.close(); + a.close(); + b.close(); + } +}); diff --git a/js/net/src/resume.ts b/js/net/src/resume.ts index 6f883bf0939..14c48f25373 100644 --- a/js/net/src/resume.ts +++ b/js/net/src/resume.ts @@ -1173,6 +1173,8 @@ export class Front { const result = await race([listing.next, wait().then(() => undefined)]); if (!result) continue; listing.next = undefined; + // The source was replaced as its answer arrived: the new source's listing supersedes it. + if (listing.source !== this.#source) continue; const { event, err } = result; if (err) { // Its readers fail, and a later reader asks again. From 9d8782485d2db8ec468e4c6eda4ad1fd41667b29 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 09:06:36 -0700 Subject: [PATCH 08/14] style(js/net): sort resume test imports Co-Authored-By: Claude Opus 5.5 --- js/net/src/resume.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/js/net/src/resume.test.ts b/js/net/src/resume.test.ts index cfa43a5ed94..516eec3edf6 100644 --- a/js/net/src/resume.test.ts +++ b/js/net/src/resume.test.ts @@ -4,9 +4,9 @@ import { Producer as BroadcastProducer } from "./broadcast.ts"; import * as Epoch from "./epoch.ts"; import { NotFound, StreamCode, StreamError } from "./error.ts"; import { Producer as GroupProducer } from "./group.ts"; -import type { Lister } from "./listing.ts"; import { Route } from "./hop.ts"; import { hooks } from "./internal.ts"; +import type { Lister } from "./listing.ts"; import { Producer } from "./origin.ts"; import * as Path from "./path.ts"; import { Front } from "./resume.ts"; From 31a04c3ba1cddf66b1e0027e1e2a13a4d2e7c981 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 09:30:42 -0700 Subject: [PATCH 09/14] fix(js/net): refuse a peer's listing that names a compressed track Co-Authored-By: Claude Opus 5.5 --- js/net/src/lite/subscriber.ts | 3 +++ js/net/src/lite/tracks.test.ts | 13 +++++++++++++ 2 files changed, 16 insertions(+) diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index e984f135acd..30e6af37711 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -721,6 +721,9 @@ export class Subscriber { } if (event.kind === "added") { if (listed.has(event.name)) throw new ProtocolViolation(`track listed twice: ${event.name}`); + // Until js/net inflates, a compressed track is refused, as its subscription is, + // rather than listed as a plain one. + if (event.info.compress) throw new Error("compressed tracks are not supported"); if (!this.#announces.tryAcquire()) { throw new SessionError(SessionCode.TooManyRequests, { reason: "too many listed tracks" }); } diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts index bcf1f812c75..d0b81073d44 100644 --- a/js/net/src/lite/tracks.test.ts +++ b/js/net/src/lite/tracks.test.ts @@ -242,6 +242,19 @@ test("a listing past the announces cap is reset, and the session stays", async ( again.close(); }); +test("a listing naming a compressed track is refused, and the session stays", async () => { + const { subscriber, listed, closing } = handPlayed(); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + await peer.send({ kind: "added", name: "a", info }); + await peer.send({ kind: "added", name: "z", info: new TrackInfo({ priority: 2, compress: true }) }); + expect(await next(tracks)).toBe("synced"); + expect(await next(tracks)).toBe("+a"); + expect(await next(tracks)).toMatch(/^error: .*compressed tracks are not supported/); + await settle(); + expect(closing).not.toHaveBeenCalled(); +}); + test("a flood of a peer's listing churn yields to the host's tasks", async () => { const { subscriber, listed } = handPlayed(); const tracks = subscriber.consume(Path.from("room")).tracks(); From 923f83492c5a96852bb4460411cdf2d7850ab101 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 09:50:24 -0700 Subject: [PATCH 10/14] test(js/net): serve covering-prefix listing tests through dynamicRoute main's #5678 makes a dynamic claim answer by publishing, so the tests that need a covering prefix route serving on request use the internal route. Co-Authored-By: Claude Opus 5.5 --- js/net/src/lite/tracks.test.ts | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts index d0b81073d44..2dff49f4d24 100644 --- a/js/net/src/lite/tracks.test.ts +++ b/js/net/src/lite/tracks.test.ts @@ -385,7 +385,7 @@ test("a newer listing of a broadcast on one session replaces the older", async ( test("a path served through a covering prefix route is NOT_FOUND", async () => { const origin = new OriginProducer(); - const dynamic = origin.dynamic(Path.from("room")); + const dynamic = origin.dynamicRoute(Path.from("room")); void (async () => { for await (const request of dynamic.requested()) { const served = new BroadcastProducer(); @@ -402,7 +402,7 @@ test("a path served through a covering prefix route is NOT_FOUND", async () => { test("a local read through a covering prefix route is NOT_FOUND, and through its exact route lists", async () => { const origin = new OriginProducer(); for (const prefix of ["room", "room/bob"]) { - const dynamic = origin.dynamic(Path.from(prefix)); + const dynamic = origin.dynamicRoute(Path.from(prefix)); void (async () => { for await (const request of dynamic.requested()) { const served = new BroadcastProducer(); @@ -494,7 +494,9 @@ test("an epoch-pinned front lists its serving route's tracks", async () => { test("an epoch-pinned front served through a covering prefix route is NOT_FOUND", async () => { const origin = new OriginProducer(); - const dynamic = origin.dynamic(Path.from("room"), { epoch: Epoch.parse("01900000-0000-7000-8000-000000000001") }); + const dynamic = origin.dynamicRoute(Path.from("room"), { + epoch: Epoch.parse("01900000-0000-7000-8000-000000000001"), + }); void (async () => { for await (const request of dynamic.requested()) { const served = new BroadcastProducer(); From aa4102f07cdcde07741e2372348494b722c9d535 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 21 Oct 2026 11:32:15 -0700 Subject: [PATCH 11/14] fix(js/net): refuse listing a broadcast consumed over moq-transport Co-Authored-By: Claude Opus 5.5 --- js/net/src/ietf/subscriber.test.ts | 12 ++++++++++++ js/net/src/ietf/subscriber.ts | 19 +++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/js/net/src/ietf/subscriber.test.ts b/js/net/src/ietf/subscriber.test.ts index c9f960a3adc..e9441b33b24 100644 --- a/js/net/src/ietf/subscriber.test.ts +++ b/js/net/src/ietf/subscriber.test.ts @@ -867,6 +867,18 @@ test("a PUBLISH_NAMESPACE update that changes the publisher applies in place", a expect(await announced.next()).toMatchObject({ prefix: Path.from("theirs"), kind: "end" }); }); +test("a consumed broadcast refuses to list its tracks rather than claim it has none", async () => { + const pair = createMockTransportPair(ALPN.DRAFT_19); + const session = new NativeSession(pair.server, VERSION, true); + const subscriber = new Subscriber({ session, cluster: { self: SELF, peer: PEER } }); + const consumer = subscriber.consume(Path.from("theirs")); + await expect(consumer.tracks().next()).rejects.toThrow("track listing requires moq-lite-07"); + // A later reader asks again, and is refused again. + await expect(consumer.tracks().next()).rejects.toThrow("track listing requires moq-lite-07"); + consumer.close(); + subscriber.close(); +}); + /** * A second PUBLISH_NAMESPACE on the stream that already carries one is no longer an * update: it is the base draft's duplicate request, a protocol violation. diff --git a/js/net/src/ietf/subscriber.ts b/js/net/src/ietf/subscriber.ts index ce472b2678c..54d5572f289 100644 --- a/js/net/src/ietf/subscriber.ts +++ b/js/net/src/ietf/subscriber.ts @@ -29,6 +29,7 @@ import { scopeHead, scopeOverlaps, } from "../internal.ts"; +import type { Lister } from "../listing.ts"; import * as Path from "../path.ts"; import { type Cursor, Reader, type Stream } from "../stream.ts"; import { Tail } from "../tail.ts"; @@ -726,6 +727,8 @@ export class Subscriber { #createConsume(path: Path.Valid, instance: ReturnType): broadcast.Consumer { // moq-transport resolves metadata through SUBSCRIBE and has no one-shot group fetch here. const consumer = new ConsumeBroadcast(); + // Registered before anyone can list the broadcast, so no reader syncs on an empty answer. + void this.#refuseListing(wireOf(consumer).lister()); void (async () => { for (;;) { const request = await wireOf(consumer).requested(); @@ -738,6 +741,22 @@ export class Subscriber { return this.#consumes.insert(path, instance, consumer); } + // moq-transport lists no tracks here: refuse each listing until the broadcast closes, since an + // empty answer would claim the peer lists nothing. + async #refuseListing(lister: Lister): Promise { + try { + for (;;) { + while (!lister.used.peek()) { + if (lister.ended.peek() !== undefined) return; + await race([Signal.race(lister.used, lister.ended)]); + } + lister.abort(new Error("track listing requires moq-lite-07 or newer")); + } + } finally { + lister.close(); + } + } + // The adapter is gone. If the transport has already closed, that close is the // error. A still-open transport, such as a GOAWAY drain, has no peer code yet, // so this does not wait for it. From daf62eec070450f8cc964011ba52a8c9413def78 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 05:37:36 -0700 Subject: [PATCH 12/14] fix(net): authorize a TRACKS listing like a subscribe A listing opened the peer's TRACKS stream without the gate subscriptions pass: it never waited for an in-band token that could cover the broadcast (so the peer refused it), never checked our grant, and outlived a grant that stopped covering it. Both the JS and Rust subscribers now wait for the token before syncing readers or asking the peer, refuse a broadcast outside the grant with Unauthorized, and reset a live listing with UNAUTHORIZED once the grant shrinks, leaving the session up. Co-Authored-By: Claude Opus 5.5 --- doc/lib/js/net.md | 2 +- js/net/src/auth.test.ts | 34 ++++++++ js/net/src/lite/subscriber.ts | 46 +++++++++-- js/net/src/lite/tracks.test.ts | 39 +++++++++- rs/moq-net/src/lite/subscriber.rs | 125 +++++++++++++++++++++++++++++- 5 files changed, 232 insertions(+), 14 deletions(-) diff --git a/doc/lib/js/net.md b/doc/lib/js/net.md index 15b0f46e326..3b0e0c34cd6 100644 --- a/doc/lib/js/net.md +++ b/doc/lib/js/net.md @@ -47,7 +47,7 @@ broadcast.announce(); - **Timing is per track, with no default.** A track that declares a `timescale` carries a `timestamp` on every frame; one without is [untimed](/concept/moq-lite#subscriptions) and its frames carry none. Omitting it means untimed, as in Rust. Nothing stamps a frame for you: the publisher passes the timestamp, and the `Timed` type (`{ value, at }`) carries one into the `@moq/json` and `@moq/flate` producers. - **Subscriber staleness is `maxDelay`.** It counts from a group's successor, by media time and by wall clock, whichever reaches it first. Passing the old `maxAge` key throws a `TypeError` naming `maxDelay`. Publisher retention is the separate `Track.Info.maxAge`. - **Hidden paths** stay out of discovery unless the announce request opts in. See [hidden broadcasts](/concept/moq-lite#hidden-broadcasts). -- **Authorization is in band.** On moq-lite 07 (opt in with `versions: ["moq-lite-07-wip", "moq-lite-06"]`, listing every fallback you still want) and on moq-transport draft-17+ with the [MoQ Auth extension](/draft/moq-auth), `auth.grant` watches what the relay lets this side publish and subscribe to, and `auth.add(token)` presents another token without reconnecting. Publishing outside the grant closes the connection with `SessionCode.Unauthorized`, naming the path. On moq-lite, a subscription the grant stops covering resets with `StreamCode.Unauthorized` and the session stays up. On moq-transport, `auth.add(token)` keeps the token even without the extension, and a broadcast announced with the `Auth.Auth` it returns (`broadcast.announce({ auth })`) carries it on that announcement's `PUBLISH_NAMESPACE`; announcing again with another renews the request. An `Auth.Auth` belongs to the session that added it, so after a reconnect, add the token again and re-announce. +- **Authorization is in band.** On moq-lite 07 (opt in with `versions: ["moq-lite-07-wip", "moq-lite-06"]`, listing every fallback you still want) and on moq-transport draft-17+ with the [MoQ Auth extension](/draft/moq-auth), `auth.grant` watches what the relay lets this side publish and subscribe to, and `auth.add(token)` presents another token without reconnecting. Publishing outside the grant closes the connection with `SessionCode.Unauthorized`, naming the path. On moq-lite, a subscription or track listing the grant stops covering resets with `StreamCode.Unauthorized` and the session stays up. On moq-transport, `auth.add(token)` keeps the token even without the extension, and a broadcast announced with the `Auth.Auth` it returns (`broadcast.announce({ auth })`) carries it on that announcement's `PUBLISH_NAMESPACE`; announcing again with another renews the request. An `Auth.Auth` belongs to the session that added it, so after a reconnect, add the token again and re-announce. - **A graceful close waits.** `await connection.close()` withdraws announcements and drains for up to one second. Application delivery requires explicit `moq-lite-07-wip`; older Lite versions wait only for transport ACKs. IETF media streams are not drained. `abort()` ends immediately. Examples: diff --git a/js/net/src/auth.test.ts b/js/net/src/auth.test.ts index 27a50d510b2..575fc5b1163 100644 --- a/js/net/src/auth.test.ts +++ b/js/net/src/auth.test.ts @@ -623,6 +623,40 @@ test("a fetch abandoned while its token waits never reaches the wire", async () } }); +// The peer refuses a listing its token has not yet authorized, so the request waits for it. +test("a listing waits for its token before reaching the wire", async () => { + const origin = new OriginProducer(); + const broadcast = origin.createBroadcast(Path.from("room/x")); + broadcast.announce(); + broadcast.createTrack("video"); + const { client, server, transport } = await connect({ serverPublish: origin, protocol: Lite.ALPN_07_WIP }); + const requests = server.auth.requests(); + const setup = await requests.next(); + if (!setup) throw new Error("missing setup token"); + const initial = setup.accept(grant([], [])); + await waitFor(client.auth.grant, (g) => g !== undefined); + const adding = client.auth.add("first"); + try { + const unanswered = await requests.next(); + if (!unanswered) throw new Error("missing token"); + const opened = transport.sendStreams.bidi.length; + const tracks = wireOf(client).consume(Path.from("room/x")).tracks(); + for (let i = 0; i < 10; i++) await nextTask(); + expect(transport.sendStreams.bidi.length).toBe(opened); + const issued = unanswered.accept(grant([], ["room"])); + await adding; + expect(await tracks.next()).toEqual({ kind: "synced" }); + expect(await tracks.next()).toMatchObject({ kind: "added", name: "video" }); + tracks.close(); + issued.close(); + } finally { + initial.close(); + client.abort(); + server.abort(); + await adding.catch(() => void 0); + } +}); + // The window between the wait seeing the caller leave and acting on it spans a few // microtasks, so sweep where the new caller lands. test.each(Array.from({ length: 17 }, (_, gap) => gap))( diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 3ebf23aac12..64493c0e3f9 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -664,22 +664,31 @@ export class Subscriber { // Answer listings of a consumed broadcast from the peer's TRACKS while anyone lists it, until // the broadcast closes. Every listing of the broadcast shares one TRACKS stream. async #runListing(broadcast: Path.Valid, consumed: Consumed, lister: Lister) { + const unlisted = async () => { + while (lister.used.peek() && lister.ended.peek() === undefined) { + await Signal.race(lister.used, lister.ended); + } + }; try { for (;;) { - while (!lister.used.peek()) { - if (lister.ended.peek() !== undefined || this.#closed.signal.aborted) return; + if (lister.ended.peek() !== undefined || this.#closed.signal.aborted) return; + if (!lister.used.peek()) { await untilAborted(race([Signal.race(lister.used, lister.ended)]), this.#closed.signal).catch( () => {}, ); + continue; } if (!hasTracks(this.version)) { // An empty answer would claim the peer lists nothing. lister.abort(new Error("track listing requires moq-lite-07 or newer")); continue; } - // Synced with this hop, which holds nothing yet: a relay never waits on its upstream, - // so the peer's tracks follow as live adds. - lister.synced(); + // A listing is a subscribe: one opened before a token that covers it is answered + // would be refused by the peer. + const ready = await this.#auth + ?.ready(broadcast, "subscribe", { signal: this.#closed.signal, until: unlisted }) + .catch(() => false); + if (ready === false) continue; await this.#listTracks(broadcast, consumed, lister); } } finally { @@ -688,12 +697,27 @@ export class Subscriber { } // Forward the peer's TRACKS listing of `broadcast` into `lister` until the peer ends it, it - // fails, or nobody lists the broadcast any more. + // fails, our grant stops covering the broadcast, or nobody lists the broadcast any more. async #listTracks(broadcast: Path.Valid, consumed: Consumed, lister: Lister): Promise { + // Armed before the first check and held to the end, so a shrink is never missed. + const refused = unauthorized(broadcast); + const revoked = new AbortController(); + const disposeGrant = this.#grant?.subscribe(() => { + if (this.#denied(broadcast)) revoked.abort(refused); + }); + if (this.#denied(broadcast)) { + disposeGrant?.(); + lister.abort(refused); + return; + } + // Synced with this hop, which holds nothing yet: a relay never waits on its upstream, + // so the peer's tracks follow as live adds. + lister.synced(); + // Each listed track holds an announces slot: discovery state the peer makes us hold. const listed = new Set(); const unused = new AbortController(); - const stop = AbortSignal.any([unused.signal, this.#closed.signal]); + const stop = AbortSignal.any([unused.signal, revoked.signal, this.#closed.signal]); const disposeUsed = lister.used.subscribe((used) => { if (!used) unused.abort(new StreamError(StreamCode.Cancel, { message: "listing unused" })); }); @@ -740,6 +764,13 @@ export class Subscriber { } } } catch (err) { + if (revoked.signal.aborted) { + // Reset rather than cancel, so the peer reads a revocation instead of a routine end. + console.debug(`tracks revoked: broadcast=${broadcast}`); + stream?.abort(refused); + lister.abort(refused); + return; + } if (unused.signal.aborted) { // Nobody lists the broadcast any more: cancel the peer's listing. stream?.close(); @@ -752,6 +783,7 @@ export class Subscriber { stream?.abort(e); lister.abort(e); } finally { + disposeGrant?.(); disposeUsed(); this.#announces.release(listed.size); } diff --git a/js/net/src/lite/tracks.test.ts b/js/net/src/lite/tracks.test.ts index 517e3df5238..36a08cbe7bc 100644 --- a/js/net/src/lite/tracks.test.ts +++ b/js/net/src/lite/tracks.test.ts @@ -1,5 +1,7 @@ import { afterEach, expect, mock, spyOn, test } from "bun:test"; import { Milli } from "@moq/clock"; +import { type Getter, Signal } from "@moq/signals"; +import type { Grant } from "../auth.ts"; import { Producer as BroadcastProducer, type Tracks } from "../broadcast.ts"; import { accept, connect } from "../connection/index.ts"; import * as Epoch from "../epoch.ts"; @@ -10,6 +12,7 @@ import { Producer as OriginProducer } from "../origin.ts"; import * as Path from "../path.ts"; import { Reader, Stream, Writer } from "../stream.ts"; import { Timescale } from "../time.ts"; +import type { Limits } from "../util/limits.ts"; import { wireOf } from "../wire.ts"; import { Publisher } from "./publisher.ts"; import { StreamId } from "./stream.ts"; @@ -103,7 +106,7 @@ test("older versions have no tracks stream", async () => { }); /** A lite-07 subscriber whose publisher the test plays by hand. */ -function handPlayed(limits?: { announces: number; subscriptions: number; fetches: number }) { +function handPlayed({ limits, grant }: { limits?: Limits; grant?: Getter } = {}) { const pair = createMockTransportPair(ALPN_07_WIP); const closing = spyOn(pair.client, "close"); const subscriber = new Subscriber( @@ -113,7 +116,7 @@ function handPlayed(limits?: { announces: number; subscriptions: number; fetches undefined, undefined, undefined, - undefined, + grant, limits, ); // The TRACKS request the subscriber opens next. @@ -218,7 +221,7 @@ test("a peer's malformed listing closes the session", async () => { test("a listing past the announces cap is reset, and the session stays", async () => { const cap = 3; - const { subscriber, listed, closing } = handPlayed({ announces: cap, subscriptions: 10, fetches: 10 }); + const { subscriber, listed, closing } = handPlayed({ limits: { announces: cap, subscriptions: 10, fetches: 10 } }); const tracks = subscriber.consume(Path.from("room")).tracks(); const peer = await listed(); for (let i = 0; i <= cap; i++) await peer.send({ kind: "added", name: `t${i}`, info }); @@ -280,6 +283,36 @@ test("a version without the Tracks Stream refuses the listing", async () => { expect(await next(tracks)).toBe("error: Error: track listing requires moq-lite-07 or newer"); }); +function scoped(prefix: string): Grant { + return { publish: new Path.Patterns(), subscribe: new Path.Patterns([Path.Pattern.subtree(prefix)]) }; +} + +test("a listing our grant does not cover is refused, and never reaches the wire", async () => { + const { subscriber, pair } = handPlayed({ grant: new Signal(scoped("other")) }); + const tracks = subscriber.consume(Path.from("room")).tracks(); + expect(await next(tracks)).toBe(`error: ${StreamCode.Unauthorized}`); + expect(pair.client.sendStreams.bidi).toHaveLength(0); +}); + +test("a grant that stops covering a listing resets it with UNAUTHORIZED, and the session stays", async () => { + const grant = new Signal(scoped("room")); + const { subscriber, listed, closing } = handPlayed({ grant }); + const tracks = subscriber.consume(Path.from("room")).tracks(); + const peer = await listed(); + await peer.send({ kind: "added", name: "a", info }); + expect(await next(tracks)).toBe("synced"); + expect(await next(tracks)).toBe("+a"); + + grant.set(scoped("other")); + expect(await next(tracks)).toBe(`error: ${StreamCode.Unauthorized}`); + const reset = await peer.stream.reader.done().then( + () => undefined, + (err: unknown) => fromTransport(err), + ); + expect((reset as StreamError).code).toBe(StreamCode.Unauthorized); + expect(closing).not.toHaveBeenCalled(); +}); + /** A lite-07 publisher serving `origin`, answering TRACKS streams the test writes by hand. */ function publisher(origin: OriginProducer) { const pair = createMockTransportPair(ALPN_07_WIP); diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 2f0a7262164..86e9fb862bb 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -1837,11 +1837,25 @@ struct TracksRecv { /// Each event fans out to every reader, so a peer streaming add/remove churn must not /// hold the thread for as long as its bytes keep arriving. budget: kio::coop::Budget, + /// A listing is a subscribe: it waits for a token that may cover the broadcast, and + /// ends once the session stops allowing it. + gate: crate::auth::Gate, + /// The gate let the listing through, and this hop synced its readers. + admitted: bool, } impl TracksRecv { /// Apply the peer's events until it ends the listing, or the error that ends it. fn poll(&mut self, source: &SourceServe, waiter: &kio::Waiter) -> Poll> { + ready!(self.gate.poll_ready(waiter)); + if self.gate.poll_denied(waiter).is_ready() { + return Poll::Ready(Err(Error::Unauthorized)); + } + if !std::mem::replace(&mut self.admitted, true) { + // Synced with this hop, which holds nothing yet: a relay never waits on its + // upstream, so the peer's tracks follow as live adds. + source.lister.synced(); + } let mut cx = waiter.context(); let version = source.subscriber.version; let stream = match &mut self.stream { @@ -1926,15 +1940,18 @@ impl SourceServe { self.lister.abort(Error::Unsupported); continue; } - // Synced with this hop, which holds nothing yet: a relay never waits on its - // upstream, so the peer's tracks follow as live adds. - self.lister.synced(); listing = Some(TracksRecv { session: self.subscriber.session.clone(), stream: None, listed: HashMap::new(), synced: false, budget: kio::coop::Budget::new(32), + gate: crate::auth::Gate::new( + self.subscriber.auth.clone(), + self.path.clone(), + crate::auth::Direction::Subscribe, + ), + admitted: false, }); } Some(_) if self.lister.poll_unused(waiter).is_ready() => { @@ -3613,6 +3630,108 @@ mod tests { } } + /// A lite-07 source of `room` over `session`, a consumer to list it through, and the + /// route that keeps it served. + fn listed_source( + session: SinkSession, + auth: crate::auth::Handle, + ) -> (SourceServe, crate::broadcast::Consumer, kio::Producer<()>) { + let subscriber = Subscriber::new(SubscriberConfig { + runtime: crate::time::sim(), + session, + origin: origin::Config::new(crate::Hop::new(1).unwrap()).produce(), + recv_bandwidth: None, + version: Version::Lite07, + peer_setup: crate::lite::PeerSetup::new(false, crate::transport::Binding::InBand), + peer_hop: None, + cost: None, + going_away: Default::default(), + auth, + }); + let broadcast = crate::broadcast::Info::new().produce(); + let consumer = broadcast.consume(); + let dynamic = broadcast.dynamic(); + let route = kio::Producer::new(()); + let serve = SourceServe::new( + subscriber, + MintedSource { + validity: Default::default(), + path: Path::new("room").to_owned(), + epoch: None, + source: crate::model::broadcast::SourceGuard::new(broadcast), + dynamic, + route: route.consume(), + }, + ); + (serve, consumer, route) + } + + /// A listing is a subscribe: it waits for a token that may cover the broadcast before + /// syncing its readers or asking the peer, which would refuse it until then. + #[moq_clock::test] + async fn a_listing_waits_for_its_token() { + let auth = crate::auth::Handle::new(true); + let token = auth.present(bytes::Bytes::from_static(b"new"), false).unwrap(); + let open = kio::Producer::new(true); + let session = SinkSession::gated_bi(open.consume()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth); + let mut tracks = consumer.tracks().unwrap(); + + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert_eq!(session.log.bi_opens(), 0, "asked the peer before the token settled"); + assert!( + tracks.next().now_or_never().is_none(), + "synced before the token settled" + ); + + drop(token); + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert_eq!(session.log.bi_opens(), 1); + assert!(matches!( + tracks.next().now_or_never(), + Some(Ok(Some(crate::broadcast::TracksEvent::Synced))) + )); + } + + /// A listing outside the session's scope is refused before it reaches the wire. + #[moq_clock::test] + async fn a_listing_outside_the_scope_is_refused() { + let auth = crate::auth::Handle::new(false); + auth.authorize(&crate::auth::Grant::default()); + let open = kio::Producer::new(true); + let session = SinkSession::gated_bi(open.consume()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth); + let mut tracks = consumer.tracks().unwrap(); + + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert!(matches!(tracks.next().now_or_never(), Some(Err(Error::Unauthorized)))); + assert_eq!(session.log.bi_opens(), 0); + } + + /// A narrower scope resets a live listing with UNAUTHORIZED rather than keep listing + /// what the session no longer allows, and leaves the session up. + #[moq_clock::test] + async fn a_narrower_limit_resets_a_listing() { + let auth = crate::auth::Handle::new(false); + let open = kio::Producer::new(true); + let session = SinkSession::gated_bi(open.consume()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth.clone()); + let mut tracks = consumer.tracks().unwrap(); + + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert_eq!(session.log.bi_opens(), 1); + assert!(matches!( + tracks.next().now_or_never(), + Some(Ok(Some(crate::broadcast::TracksEvent::Synced))) + )); + + auth.authorize(&crate::auth::Grant::default()); + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert!(matches!(tracks.next().now_or_never(), Some(Err(Error::Unauthorized)))); + assert_eq!(session.log.resets(), vec![crate::StreamError::Unauthorized.to_code()]); + assert!(session.log.closes().is_empty()); + } + /// A track waiting on an unanswered token ends once nobody wants it, rather than /// holding its task and track until a token that may never be answered. #[moq_clock::test] From 86558abcc32096b8c5b7101d8eb02fc0053b4efa Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 05:57:13 -0700 Subject: [PATCH 13/14] fix(net): never list a retired epochless instance A TRACKS request names no epoch for an epochless route, so a listing set up on an instance that was replaced while it waited for its token would list the replacement's tracks. Until the request is out, both the JS and Rust subscribers now check the consumed instance's validity, as track setup does, and fail the listing with Unroutable once it retires. Co-Authored-By: Claude Opus 5.5 --- js/net/src/lite/subscriber.test.ts | 64 ++++++++++++++++++++++++++++++ js/net/src/lite/subscriber.ts | 39 ++++++++++++------ rs/moq-net/src/lite/subscriber.rs | 37 +++++++++++++++-- 3 files changed, 123 insertions(+), 17 deletions(-) diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index fc8efae24e2..5e235670ac1 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -2,6 +2,7 @@ import { expect, jest, spyOn, test } from "bun:test"; import { Manual, Milli, Nano } from "@moq/clock"; import { Signal } from "@moq/signals"; import type { Grant } from "../auth.ts"; +import type { AuthSession } from "../auth_session.ts"; import type { Probe as ProbeStats } from "../connection/stats.ts"; import * as Epoch from "../epoch.ts"; import { @@ -1448,6 +1449,69 @@ test.each([Version.DRAFT_05, Version.DRAFT_06, Version.DRAFT_07])( }, ); +// A TRACKS request carries no epoch for an epochless route, so it would list the replacement. +test("a listing of an epochless handle retired while its token waits never reaches the wire", async () => { + const { quic, streams, opened } = fakeSession(); + let release!: () => void; + const released = new Promise((resolve) => (release = resolve)); + const auth = { + ready: (_path: Path.Valid, _direction: string, opts?: { signal?: AbortSignal }) => + new Promise((resolve, reject) => { + void released.then(() => resolve(true)); + opts?.signal?.addEventListener("abort", () => reject(opts.signal?.reason), { once: true }); + }), + } as unknown as AuthSession; + const version = Version.DRAFT_07; + const subscriber = new Subscriber( + quic, + version, + HopSchema.parse(1n), + undefined, + undefined, + undefined, + undefined, + undefined, + auth, + ); + const announced = subscriber.announced(); + await opened(0); + const send = async (encode: (writer: Writer) => Promise) => { + const writer = new Writer( + new WritableStream({ + write: (chunk) => streams[0].inbound.enqueue(new Uint8Array(chunk)), + }), + version, + ); + await encode(writer); + }; + const path = Path.from("pool"); + await send((writer) => new AnnounceOk(PEER, 0).encode(writer, version)); + await send((writer) => + encodeAnnounceBroadcast(writer, { status: "active", suffix: path, hops: [PUBLISHER_A] }, version), + ); + await announced.next(); + const held = subscriber.consume(Path.from("pool/job")); + const tracks = held.tracks(); + const event = tracks.next().catch((err: unknown) => err); + + await send((writer) => encodeAnnounceBroadcast(writer, { status: "endedId", id: 0n }, version)); + await announced.next(); + await send((writer) => + encodeAnnounceBroadcast(writer, { status: "active", suffix: path, hops: [PUBLISHER_B] }, version), + ); + await announced.next(); + release(); + + const err = await event; + expect(err).toBeInstanceOf(StreamError); + expect((err as StreamError).code).toBe(StreamCode.Unroutable); + expect(streams.length).toBe(1); + tracks.close(); + held.close(); + announced.close(); + subscriber.close(); +}); + test.each(["track", "fetch", "open"] as const)( "retirement settles coalesced fetches during %s setup", async (stage) => { diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 64493c0e3f9..45fd870d905 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -664,11 +664,6 @@ export class Subscriber { // Answer listings of a consumed broadcast from the peer's TRACKS while anyone lists it, until // the broadcast closes. Every listing of the broadcast shares one TRACKS stream. async #runListing(broadcast: Path.Valid, consumed: Consumed, lister: Lister) { - const unlisted = async () => { - while (lister.used.peek() && lister.ended.peek() === undefined) { - await Signal.race(lister.used, lister.ended); - } - }; try { for (;;) { if (lister.ended.peek() !== undefined || this.#closed.signal.aborted) return; @@ -683,12 +678,6 @@ export class Subscriber { lister.abort(new Error("track listing requires moq-lite-07 or newer")); continue; } - // A listing is a subscribe: one opened before a token that covers it is answered - // would be refused by the peer. - const ready = await this.#auth - ?.ready(broadcast, "subscribe", { signal: this.#closed.signal, until: unlisted }) - .catch(() => false); - if (ready === false) continue; await this.#listTracks(broadcast, consumed, lister); } } finally { @@ -698,7 +687,30 @@ export class Subscriber { // Forward the peer's TRACKS listing of `broadcast` into `lister` until the peer ends it, it // fails, our grant stops covering the broadcast, or nobody lists the broadcast any more. + // Returns without answering when setup was cut short, for the caller to look again. async #listTracks(broadcast: Path.Valid, consumed: Consumed, lister: Lister): Promise { + // Until the request is out, the consumed instance must still be the selected one: a TRACKS + // request without an epoch would list its replacement. + const valid = consumed.valid; + if (valid?.aborted) { + lister.abort(error(valid.reason)); + return; + } + const setup = valid ? AbortSignal.any([this.#closed.signal, valid]) : this.#closed.signal; + // A listing is a subscribe: one opened before a token that covers it is answered would be + // refused by the peer. + const ready = await this.#auth + ?.ready(broadcast, "subscribe", { + signal: setup, + until: async () => { + while (lister.used.peek() && lister.ended.peek() === undefined) { + await Signal.race(lister.used, lister.ended); + } + }, + }) + .catch(() => false); + if (ready === false || setup.aborted) return; + // Armed before the first check and held to the end, so a shrink is never missed. const refused = unauthorized(broadcast); const revoked = new AbortController(); @@ -724,13 +736,14 @@ export class Subscriber { let stream: Stream | undefined; try { const opening = Stream.open(this.#quic, { version: this.version, ...requestOptions }); + const cut = valid ? AbortSignal.any([stop, valid]) : stop; void opening.then( (late) => { - if (stop.aborted) late.abort(error(stop.reason)); + if (cut.aborted) late.abort(error(cut.reason)); }, () => {}, ); - stream = await untilAborted(opening, stop); + stream = await untilAborted(opening, cut); await stream.writer.u53(StreamId.Tracks); await new TracksMessage(broadcast, consumed.epoch).encode(stream.writer, this.version); let synced = false; diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 86e9fb862bb..4f8210b1033 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -1842,11 +1842,17 @@ struct TracksRecv { gate: crate::auth::Gate, /// The gate let the listing through, and this hop synced its readers. admitted: bool, + /// Until the request is out, the source must still be the selected instance: a TRACKS + /// request without an epoch would list its replacement. + validity: crate::model::broadcast::RequestValidity, } impl TracksRecv { /// Apply the peer's events until it ends the listing, or the error that ends it. fn poll(&mut self, source: &SourceServe, waiter: &kio::Waiter) -> Poll> { + if self.stream.is_none() { + self.validity.check(waiter)?; + } ready!(self.gate.poll_ready(waiter)); if self.gate.poll_denied(waiter).is_ready() { return Poll::Ready(Err(Error::Unauthorized)); @@ -1952,6 +1958,7 @@ impl SourceServe { crate::auth::Direction::Subscribe, ), admitted: false, + validity: self.validity.start(), }); } Some(_) if self.lister.poll_unused(waiter).is_ready() => { @@ -3635,6 +3642,7 @@ mod tests { fn listed_source( session: SinkSession, auth: crate::auth::Handle, + validity: crate::model::broadcast::RequestValidity, ) -> (SourceServe, crate::broadcast::Consumer, kio::Producer<()>) { let subscriber = Subscriber::new(SubscriberConfig { runtime: crate::time::sim(), @@ -3655,7 +3663,7 @@ mod tests { let serve = SourceServe::new( subscriber, MintedSource { - validity: Default::default(), + validity, path: Path::new("room").to_owned(), epoch: None, source: crate::model::broadcast::SourceGuard::new(broadcast), @@ -3674,7 +3682,7 @@ mod tests { let token = auth.present(bytes::Bytes::from_static(b"new"), false).unwrap(); let open = kio::Producer::new(true); let session = SinkSession::gated_bi(open.consume()); - let (mut serve, consumer, _route) = listed_source(session.clone(), auth); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth, Default::default()); let mut tracks = consumer.tracks().unwrap(); assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); @@ -3693,6 +3701,27 @@ mod tests { )); } + /// An instance retired while its listing waits for a token lists nothing: a TRACKS + /// request without an epoch would list its replacement. + #[moq_clock::test] + async fn a_listing_of_a_retired_instance_never_reaches_the_wire() { + let auth = crate::auth::Handle::new(true); + let token = auth.present(bytes::Bytes::from_static(b"new"), false).unwrap(); + let open = kio::Producer::new(true); + let session = SinkSession::gated_bi(open.consume()); + let selected = kio::Producer::new(()); + let validity = crate::model::broadcast::RequestValidity::Route(selected.consume()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth, validity); + let mut tracks = consumer.tracks().unwrap(); + + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + drop(selected); + drop(token); + assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); + assert!(matches!(tracks.next().now_or_never(), Some(Err(Error::Unroutable)))); + assert_eq!(session.log.bi_opens(), 0); + } + /// A listing outside the session's scope is refused before it reaches the wire. #[moq_clock::test] async fn a_listing_outside_the_scope_is_refused() { @@ -3700,7 +3729,7 @@ mod tests { auth.authorize(&crate::auth::Grant::default()); let open = kio::Producer::new(true); let session = SinkSession::gated_bi(open.consume()); - let (mut serve, consumer, _route) = listed_source(session.clone(), auth); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth, Default::default()); let mut tracks = consumer.tracks().unwrap(); assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); @@ -3715,7 +3744,7 @@ mod tests { let auth = crate::auth::Handle::new(false); let open = kio::Producer::new(true); let session = SinkSession::gated_bi(open.consume()); - let (mut serve, consumer, _route) = listed_source(session.clone(), auth.clone()); + let (mut serve, consumer, _route) = listed_source(session.clone(), auth.clone(), Default::default()); let mut tracks = consumer.tracks().unwrap(); assert!(kio::Task::poll(&mut serve, &kio::Waiter::noop()).is_pending()); From af41d8bb9fd2a18187c011df69c5b6f52a1da574 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 05:59:22 -0700 Subject: [PATCH 14/14] fix(js/net): drop a listing whose readers left as its token settled The unused watch is armed after the auth wait, and a signal subscription does not replay its value, so readers leaving in that gap went unseen and the listing opened TRACKS for nobody. Recheck after the wait. The docs now say a session listing syncs with what this side holds, as tracks() does. Co-Authored-By: Claude Opus 5.5 --- doc/lib/js/net.md | 2 +- js/net/src/lite/subscriber.test.ts | 35 ++++++++++++++++++++++++++++++ js/net/src/lite/subscriber.ts | 3 ++- 3 files changed, 38 insertions(+), 2 deletions(-) diff --git a/doc/lib/js/net.md b/doc/lib/js/net.md index 3b0e0c34cd6..f0c964bc2d7 100644 --- a/doc/lib/js/net.md +++ b/doc/lib/js/net.md @@ -42,7 +42,7 @@ broadcast.announce(); - **Connections share by URL and versions.** `new Connection({ url })` pools and reconnects with backoff, which the elements use. An ordered `versions` list, spelled like `moq-lite-06` as in Rust, applies to both WebTransport and WebSocket; equivalent lists share, while a different order or an omitted list gets its own pool entry. Supplying your own transport options or origin selects a private loop. Receive estimates are on demand: observe `connection.probe` reactively or subscribe while you need them; `peek()` alone opens no PROBE stream. - **GOAWAY moves the session.** The connection dials the replacement at once while the old session keeps serving its groups, up to `goaway.handover` (default 10s, or the relay's deadline when that is sooner). New requests keep opening on the old session until the replacement's route outranks it, which is a restart without an epoch: an unpinned announced request follows the replacement, while a blind or epoch-pinned request stays on its accepted instance. With an epoch, which travels only when `moq-lite-07-wip` is offered, the replacement serves the same broadcast, so open subscriptions move across without missing a group. An empty GOAWAY redials the same URL. A redirect stays on the same host unless `goaway.redirect` is `"follow"`. - **One send estimate per connection.** `Bandwidth.Allocator` divides it by track priority, max-min fair within a tier. An idle track claims nothing. Publishers reserve against it so their targets sum to the estimate. -- **List a broadcast's tracks.** `for await (const event of broadcast.tracks())` reports each track the broadcast lists (`added`, with its `Track.Info`), one `synced`, then `added` and `removed` as tracks come and go, until the broadcast ends; leaving the loop stops listing. Over a session it needs `moq-lite-07-wip`, syncs at once with nothing, and the peer's tracks follow as `added`. A track served only on request is never listed. See [discovery](/concept/moq-lite#discovery). +- **List a broadcast's tracks.** `for await (const event of broadcast.tracks())` reports each track the broadcast lists (`added`, with its `Track.Info`), one `synced`, then `added` and `removed` as tracks come and go, until the broadcast ends; leaving the loop stops listing. Over a session it needs `moq-lite-07-wip`, syncs at once with what this side holds, and the peer's tracks follow as `added`. A track served only on request is never listed. See [discovery](/concept/moq-lite#discovery). - **One track per name.** Concurrent subscriptions share one request and producer. `createTrack` on a name with a request still queued answers it. The application owns sequence numbers: a track that replaces an ended one, created or requested, throws on an append that would reuse a sequence the name already wrote, so it writes its first group with an explicit sequence. Only a new broadcast starts over. - **Timing is per track, with no default.** A track that declares a `timescale` carries a `timestamp` on every frame; one without is [untimed](/concept/moq-lite#subscriptions) and its frames carry none. Omitting it means untimed, as in Rust. Nothing stamps a frame for you: the publisher passes the timestamp, and the `Timed` type (`{ value, at }`) carries one into the `@moq/json` and `@moq/flate` producers. - **Subscriber staleness is `maxDelay`.** It counts from a group's successor, by media time and by wall clock, whichever reaches it first. Passing the old `maxAge` key throws a `TypeError` naming `maxDelay`. Publisher retention is the separate `Track.Info.maxAge`. diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index 5e235670ac1..3c97d3bc1ed 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -1512,6 +1512,41 @@ test("a listing of an epochless handle retired while its token waits never reach subscriber.close(); }); +test("a listing whose readers leave as its token settles never reaches the wire", async () => { + const { quic, streams } = fakeSession(); + let release!: () => void; + const released = new Promise((resolve) => (release = resolve)); + let ask!: () => void; + const asked = new Promise((resolve) => (ask = resolve)); + // Ignores `until`, so the wait settles admitted after the readers left. + const auth = { + ready: () => { + ask(); + return released.then(() => true); + }, + } as unknown as AuthSession; + const subscriber = new Subscriber( + quic, + Version.DRAFT_07, + HopSchema.parse(1n), + undefined, + undefined, + undefined, + undefined, + undefined, + auth, + ); + const held = subscriber.consume(Path.from("room")); + const tracks = held.tracks(); + await asked; + release(); + tracks.close(); + await new Promise((resolve) => setTimeout(resolve, 0)); + expect(streams.length).toBe(0); + held.close(); + subscriber.close(); +}); + test.each(["track", "fetch", "open"] as const)( "retirement settles coalesced fetches during %s setup", async (stage) => { diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 45fd870d905..cdffbf67c95 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -709,7 +709,8 @@ export class Subscriber { }, }) .catch(() => false); - if (ready === false || setup.aborted) return; + // The readers may have left while the wait settled, before the unused watch below is armed. + if (ready === false || setup.aborted || !lister.used.peek()) return; // Armed before the first check and held to the end, so a shrink is never missed. const refused = unauthorized(broadcast);