fix(publish): mint a new track name when a rendition's decoder identity changes

doc/concept/hang.md

@@ -51,21 +51,20 @@ VP9, AV1, AAC, and Opus are what the tools produce today. Properties shared by
 every video rendition (display size, rotation, flip) sit at the section root.
 
 A track name always means the same content, so a catalog update never
-redefines a track. Its codec, description, container, sample rate, and channel
-count are fixed. `codedWidth` and `codedHeight` are ceilings: the picture
-resizes in band at a keyframe below them, unless a `description` pins it.
-`bitrate` and `framerate` are maxima that only rise, like `jitter` and `delay`.
-Everything else, such as `enabled` and the section's display size, updates
-freely. A
-publisher that needs a new identity publishes a new track name, or restarts the
-broadcast under a new [epoch](/concept/moq-lite#publisher-epochs). A player can
-configure its decoder once per track.
+redefines a track. The fields a decoder can't change without reinitializing
+are fixed: codec, description, container, display aspect, sample rate, and
+channel count. Everything else, such as the coded size, `bitrate`,
+`framerate`, and `enabled`, updates in place; the picture resizes in band at a
+keyframe unless a `description` pins it. A publisher that needs a new identity
+publishes a new track name, or restarts the broadcast under a new
+[epoch](/concept/moq-lite#publisher-epochs). A player can configure its
+decoder once per track.
 
 A few things the catalog can express beyond decoder config:
 
 - **Labels.** Any rendition may carry a human-readable `label` for a track picker. The map key stays the rendition's ID, so labels need not be unique and renaming one doesn't rename the track.
 - **Renditions in another broadcast.** A rendition may point at a relative broadcast path, so a transcoder can publish a ladder that adds low rungs and references the source's original rendition without re-publishing its bytes. The path resolves against where the consumer found the catalog, so a reference that escapes above the root names nothing and the catalog is rejected.
-- **Track names.** The map key is a rendition's ID and, by default, its track name. A rendition sets `track` only when the two must differ, such as two renditions of same-named tracks in different broadcasts; readers subscribe to `track` when present. An ID can't name both a video and an audio rendition.
+- **Track names.** The map key is a rendition's ID and, by default, its track name. A rendition sets `track` only when the two must differ, such as two renditions of same-named tracks in different broadcasts, or a rendition whose media changed and so needs a new name; readers subscribe to `track` when present. An ID can't name both a video and an audio rendition.
 - **Jitter.** A rendition can say how far its frames fell behind the media clock before the publisher flushed them, in whole milliseconds rounded up. Encoders report the spread of lateness above each rendition's own recent minimum, so a constant encoder delay is not jitter; container imports estimate batch spans without counting ingest delay. It describes the publisher, never the network, only grows over the life of a stream, and a player sizes its buffer to at least this much. A `0` is read as absent.
 - **Delay.** A rendition can also say how far its frames reach the transport behind the broadcast's earliest rendition, measured the same way from each rendition's minimum lateness, so a video encoder running 200 ms behind audio advertises `delay: 200` on video. It follows the same rules as jitter. A player holds the largest `delay + jitter` among the renditions it subscribes to, and never subtracts one rendition's `delay` from another's.
 - **Disabled renditions.** An audio or video rendition with `enabled: false` has no frames coming, and a player must not select it. Pausing a rendition in `@moq/publish` keeps it in the catalog this way rather than removing it, so a mute is a one-field update; a disabled video rendition encodes one black keyframe first, so a player that predates the field shows black instead of a frozen picture. The field is written only when false. The older video `stalled` field is no longer written, and readers ignore it.

doc/lib/js/publish.md

@@ -49,7 +49,10 @@ for a permission, device, or settings change instead of prompting again.
 The video encoder follows its share of the connection's send-rate estimate, so
 several publishers on one connection do not each target the whole uplink.
 Codec, resolution, framerate, and bitrate are tunable through
-`el.video.config`, and `el.video.cut()` asks for an extra keyframe. For
+`el.video.config`, and `el.video.cut()` asks for an extra keyframe. Size,
+bitrate, and framerate change in place, while a codec or audio format change
+keeps the rendition's catalog ID under a fresh `track` name that viewers
+follow through the catalog. For
 simulcast, drop the element and register several encoders on a
 `Publish.Broadcast`, as below.
 
@@ -68,8 +71,10 @@ timeline. The catalog's `clock` maps it to wall time. Stamp your own tracks
 `broadcast.net` is the underlying `Moq.Broadcast.Producer`, so an application
 can serve its own tracks alongside the media. It is recreated when the origin, name,
 `enabled`, or `maxAge` changes, so acquire it from an effect and create each
-track once per `net`. A track name always means the same content, so a track
-re-created on the same `net` throws on an append that would reuse its sequences:
+track once per `net`. Names under a rendition's ID, such as `video/1`, are
+reserved for that rendition's replacement tracks. A track name always means
+the same content, so a track re-created on the same `net` throws on an append
+that would reuse its sequences:
 
 ```ts
 import * as Json from "@moq/json";

drafts/draft-lcurley-moq-hang.md

@@ -166,14 +166,13 @@ A discontinuity marker is a delivery event, not a new epoch, and a system-clock
 
 ## Track Identity {#identity}
 A track name always means the same content, so a catalog update never redefines an audio or video track.
-Each rendition field falls in one of three classes:
+A track's identity is the fields a consumer cannot change without reinitializing its decoder or demuxer: `codec`, `description`, and `container`, plus `displayAspectWidth` and `displayAspectHeight` for video and `sampleRate` and `numberOfChannels` for audio.
+A publisher MUST NOT change them for a track once a catalog lists it.
 
-- Identity: `codec`, `description`, and `container`, plus `sampleRate` and `numberOfChannels` for audio. A publisher MUST NOT change them for a track once a catalog lists it.
-- Ceilings: video `codedWidth` and `codedHeight` bound every picture the track carries, and `codec` names a profile and level that covers them. The picture size MAY change at any keyframe within both bounds, which a `description` carrying parameter sets rules out. A publisher MAY fill an absent ceiling once, and MUST NOT change it afterward.
-- Maxima: `bitrate` and `framerate` are the most the track has carried or is configured to carry. A publisher MAY raise them and MUST NOT lower them.
-
-Every other rendition field is live state that MAY change in any update within its own rules (`jitter` and `delay` never fall), as MAY the video section's `display`, `rotation`, and `flip`, which describe presentation rather than decoding.
-A publisher that changes an identity field or exceeds a ceiling MUST publish the media under a new track name, or as a new broadcast under a new Epoch {{moql}}.
+Every other rendition field is live state that MAY change in any update within its own rules (`jitter` and `delay` never fall), as MAY the video section's `display`, `rotation`, and `flip`.
+The picture size MAY change at any keyframe, which a `description` carrying parameter sets rules out, and `codec` names a profile and level that covers every picture the track carries.
+A publisher that changes an identity field MUST publish the media under a new track name, or as a new broadcast under a new Epoch {{moql}}.
+A rendition MAY keep its ID and name the new track with `track` ({{field-track}}), so a consumer follows the new name rather than reusing a decoder configured for the ID.
 A track name removed from the catalog MUST NOT return with a different identity.
 A consumer MAY configure a decoder once per track from the catalog.
 
@@ -1123,7 +1122,7 @@ This document has no IANA actions.
 ## moq-hang-04
 {:numbered="false"}
 
-- Fixed a track's identity for its name ({{identity}}): codec, description, container, sample rate, and channel count never change; `codedWidth` and `codedHeight` are ceilings below which the picture resizes in band; `bitrate` and `framerate` are maxima that only rise. A change of identity takes a new track name or a new Epoch.
+- Fixed a track's identity for its name ({{identity}}): codec, description, container, display aspect, sample rate, and channel count never change, while every other field, including size, bitrate, and framerate, updates in place. A change of identity takes a new track name, which a rendition may name with `track` while keeping its ID, or a new Epoch.
 - Specified MSF text metadata preservation and refusal of unrepresentable text packaging during gateway conversion.
 - Specified conversion between relative broadcast references and MSF track namespaces without changing track identity, reading a namespace aligned with the catalog path's trailing segments as the catalog itself.
 

js/hang/src/catalog/audio.ts

@@ -43,7 +43,7 @@ export const AudioConfigSchema = z.object({
 	// The number of channels in the audio
 	numberOfChannels: u53Schema,
 
-	// The maximum bitrate of the audio in bits per second. It only rises.
+	// The maximum bitrate of the audio in bits per second.
 	// TODO: Support up to Number.MAX_SAFE_INTEGER
 	bitrate: z.optional(u53Schema),
 

js/hang/src/catalog/video.ts

@@ -44,10 +44,10 @@ export const VideoConfigSchema = z.object({
 	displayAspectWidth: z.optional(u53Schema),
 	displayAspectHeight: z.optional(u53Schema),
 
-	// The maximum frame rate of the video in frames per second. It only rises.
+	// The maximum frame rate of the video in frames per second.
 	framerate: z.optional(z.number()),
 
-	// The maximum bitrate of the video in bits per second. It only rises.
+	// The maximum bitrate of the video in bits per second.
 	// TODO: Support up to Number.MAX_SAFE_INTEGER
 	bitrate: z.optional(u53Schema),
 

js/publish/src/audio/capture.test.ts

@@ -504,6 +504,10 @@ function installFailableAudioEncoder() {
 			this.state = "configured";
 		}
 		encode(): void {}
+		async flush(): Promise<void> {
+			// The throwaway description probe is not a live capture encoder.
+			instances.splice(instances.indexOf(this), 1);
+		}
 		close(): void {
 			this.state = "closed";
 		}
@@ -572,6 +576,7 @@ test("a failed encoder stays failed until the capture source changes", async ()
 
 	try {
 		await render();
+		if (!encoder.out.catalog.peek()) await encoder.out.catalog.changed();
 		expect(codecs.instances).toHaveLength(1);
 		const failure = new Error("audio encoder failed");
 		codecs.instances[0].fail(failure);

js/publish/src/audio/encoder.test.ts

@@ -1,5 +1,5 @@
 import { describe, expect, mock, spyOn, test } from "bun:test";
-import type * as Catalog from "@moq/hang/catalog";
+import * as Catalog from "@moq/hang/catalog";
 import * as Json from "@moq/json";
 import * as Moq from "@moq/net";
 import { Time } from "@moq/net";
@@ -70,6 +70,8 @@ describe("toEncoderConfig", () => {
 // input timestamps never reaches the output.
 class LaggingAudioEncoder {
 	static instances: LaggingAudioEncoder[] = [];
+	static probes: AudioEncoderConfig[] = [];
+	static description: ((config: AudioEncoderConfig) => Uint8Array) | undefined;
 	static readonly LAG = 2;
 
 	// Called on configure; the encoder publishes its pipeline synchronously right after.
@@ -80,6 +82,8 @@ class LaggingAudioEncoder {
 	#held: { timestamp: number; duration: number }[] = [];
 	#base: number | undefined;
 	#encoded = 0;
+	#probe = false;
+	#config?: AudioEncoderConfig;
 	#error: AudioEncoderInit["error"];
 
 	constructor(init: AudioEncoderInit) {
@@ -95,7 +99,10 @@ class LaggingAudioEncoder {
 
 	configure(config: AudioEncoderConfig): void {
 		this.state = "configured";
-		LaggingAudioEncoder.onConfigure?.(config);
+		this.#config = config;
+		queueMicrotask(() => {
+			if (!this.#probe) LaggingAudioEncoder.onConfigure?.(config);
+		});
 	}
 
 	encode(data: AudioData): void {
@@ -104,18 +111,36 @@ class LaggingAudioEncoder {
 		this.#held.push({ timestamp: this.#base + this.#encoded, duration });
 		this.#encoded += duration;
 		while (this.#held.length > LaggingAudioEncoder.LAG) {
-			const { timestamp, duration } = this.#held.shift() as { timestamp: number; duration: number };
-			const chunk = {
-				type: "key",
-				timestamp,
-				duration,
-				byteLength: 1,
-				copyTo: (dest: Uint8Array) => dest.set([1]),
-			};
-			this.#output(chunk as unknown as EncodedAudioChunk);
+			this.#emit(this.#held.shift() as { timestamp: number; duration: number });
 		}
 	}
 
+	#emit({ timestamp, duration }: { timestamp: number; duration: number }): void {
+		const chunk = { type: "key", timestamp, duration, byteLength: 1, copyTo: (dest: Uint8Array) => dest.set([1]) };
+		const config = this.#config;
+		const description = config && LaggingAudioEncoder.description?.(config);
+		const metadata =
+			config && description
+				? {
+						decoderConfig: {
+							codec: config.codec,
+							sampleRate: config.sampleRate,
+							numberOfChannels: config.numberOfChannels,
+							description,
+						},
+					}
+				: undefined;
+		this.#output(chunk as unknown as EncodedAudioChunk, metadata);
+	}
+
+	async flush(): Promise<void> {
+		this.#probe = true;
+		if (this.#config) LaggingAudioEncoder.probes.push(structuredClone(this.#config));
+		const index = LaggingAudioEncoder.instances.indexOf(this);
+		if (index !== -1) LaggingAudioEncoder.instances.splice(index, 1);
+		for (const chunk of this.#held.splice(0)) this.#emit(chunk);
+	}
+
 	reset(): void {
 		this.state = "unconfigured";
 		this.#held = [];
@@ -145,6 +170,8 @@ class FakeAudioData {
 
 function installFakeWebCodecs() {
 	LaggingAudioEncoder.instances = [];
+	LaggingAudioEncoder.probes = [];
+	LaggingAudioEncoder.description = undefined;
 	const names = ["AudioEncoder", "AudioDecoder", "AudioData"] as const;
 	const originals = names.map((name) => Object.getOwnPropertyDescriptor(globalThis, name));
 	const fakes = [LaggingAudioEncoder, class {}, FakeAudioData];
@@ -573,6 +600,7 @@ test("a rendition trailing the broadcast's earliest advertises delay", async ()
 // Regression: codec settings that can't resolve left the encoder unsettled, so `<moq-publish>` never
 // announced and withheld every other rendition with it.
 test("settles when the codec settings can't resolve", async () => {
+	using _webcodecs = installFakeWebCodecs();
 	const error = spyOn(console, "error").mockImplementation(() => {});
 	const capture = {
 		in: { source: new Signal(undefined) },
@@ -595,7 +623,9 @@ test("settles when the codec settings can't resolve", async () => {
 		expect(error).toHaveBeenCalled();
 
 		// A valid duration resolves, and the config keeps it settled.
+		const ready = encoder.out.catalog.changed();
 		encoder.codec.set({ mime: "opus", frameDuration: Time.Milli(20) });
+		await ready;
 		await settle();
 		expect(encoder.out.catalog.peek()).toBeDefined();
 		expect(encoder.settled.peek()).toBe(true);
@@ -607,6 +637,7 @@ test("settles when the codec settings can't resolve", async () => {
 
 // A fade the gain can't ramp over is refused like any other bad setting, rather than ramping wrong.
 test("refuses the rendition while the fade is invalid", async () => {
+	using _webcodecs = installFakeWebCodecs();
 	const error = spyOn(console, "error").mockImplementation(() => {});
 	const capture = {
 		in: { source: new Signal(undefined) },
@@ -625,8 +656,9 @@ test("refuses the rendition while the fade is invalid", async () => {
 		expect(encoder.settled.peek()).toBe(true);
 		expect(error).toHaveBeenCalled();
 
+		const ready = encoder.out.catalog.changed();
 		encoder.fade.set(Time.Milli(0));
-		await settle();
+		await ready;
 		expect(encoder.out.catalog.peek()).toBeDefined();
 
 		encoder.fade.set(Time.Milli(Number.NaN));
@@ -767,6 +799,7 @@ test("audio failure stays local through demand changes and capture format re-arm
 		const config = (await catalog.latest())?.value.audio?.renditions.audio;
 		if (!config?.track) throw new Error("expected recovered track name");
 		expect(config.track).not.toBe("audio");
+		expect(config.numberOfChannels).toBe(Catalog.u53(2));
 		expect(config.enabled).not.toBe(false);
 		recovered = net.track(config.track).subscribe();
 		await settle();
@@ -865,3 +898,26 @@ test.each(["failure first", "config first"])(
 		}
 	},
 );
+
+test("Opus description is probed with the exact live config before advertising", async () => {
+	using _codecs = installFakeWebCodecs();
+	LaggingAudioEncoder.description = (config) => new TextEncoder().encode(JSON.stringify(config));
+	using env = await setup(new Baseline(), {
+		mime: "opus",
+		frameDuration: Time.Milli(2.5),
+		complexity: 3,
+		packetlossperc: 7,
+		useinbandfec: true,
+		bitrate: 64000,
+	});
+	expect(env.encoder.out.catalog.peek()?.description).toBe(Buffer.from(JSON.stringify(env.config)).toString("hex"));
+	expect(LaggingAudioEncoder.probes).toEqual([env.config]);
+	expect(LaggingAudioEncoder.instances).toHaveLength(1);
+	expect(env.written).toHaveLength(0);
+	const configuring = configured();
+	env.capture.in.source.set({ track: {}, kind: "voice" } as never);
+	const voice = await configuring;
+	expect(voice.opus).toMatchObject({ application: "voip", signal: "voice" });
+	expect(LaggingAudioEncoder.probes.at(-1)).toEqual(voice);
+	expect(env.encoder.out.catalog.peek()?.description).toBe(Buffer.from(JSON.stringify(voice)).toString("hex"));
+});