From 393efb667494a2290b1ae39e741e38f2b4b9edb5 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Tue, 27 Oct 2026 23:55:19 -0700 Subject: [PATCH 1/5] quest: cut records-free HLS on a broadcast join interval The time-addressed HLS export numbers segments from a catalog-wide media.join.interval grid, keeps whole groups, and fills pauses and overruns with EXT-X-GAP. Drop the stale opt-in wording for hls records. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 2 +- quest/m1/archive/README.md | 6 ++-- quest/m1/time-addressing/consumers.md | 51 ++++++++++++++++++++++----- 3 files changed, 46 insertions(+), 13 deletions(-) diff --git a/quest/m1/README.md b/quest/m1/README.md index 8990c91aa21..b41d1a83af1 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -125,7 +125,7 @@ surface and zero-copy texture import, land before the cut. - [Announcement takeover](/quest/m1/announce-takeover.md) - a new independent route explicitly continues instances served through its predecessor, with shared Rust/JS semantics and lite-07 wire - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Drop set](/quest/m1/drop-set.md) - each cursor judges missing groups against its own budget, so a narrow reader drops promptly and a widening re-offers groups already reported dropped on lite-07 -- [Broadcast archives](/quest/m1/archive/README.md) - ANNOUNCE, TRACKS, and a SUBSCRIBE per track record every track under a prefix to any object_store, a host serves a stored broadcast back, and the catalog drops its archive pointer for an opt-in `hls` field +- [Broadcast archives](/quest/m1/archive/README.md) - ANNOUNCE, TRACKS, and a SUBSCRIBE per track record every track under a prefix to any object_store, a host serves a stored broadcast back, and the catalog drops its archive pointer for a per-rendition `hls` field - [Compressed tracks](/quest/m1/compress/README.md) - a track flagged `compress` is DEFLATEd per group by moq-net, forwarded compressed, and inflated once on the relay for peers that cannot signal it; `catalog.json.z` and moq-flate go away - [Finalize moq-lite-07](/quest/m1/lite07-finalize.md) - when the maintainer cuts it, lite-07 negotiates as `moq-lite-07` and the next release ships it - [Moxygen minimal interop](/quest/m1/smoke-moxygen.md) - waiting on Moxygen to send a registered filter type, then the smoke lane stops hiding its failure diff --git a/quest/m1/archive/README.md b/quest/m1/archive/README.md index 9b9d927f841..b62432a2fc2 100644 --- a/quest/m1/archive/README.md +++ b/quest/m1/archive/README.md @@ -11,8 +11,8 @@ host publishes at a path of its choosing. Track, group, and frame structure and payloads survive byte for byte, so any consumer of the live broadcast, HLS included, works on the stored one. -The catalog carries no archive pointer. Timelines survive only as an opt-in -`hls` field for HLS compatibility. +The catalog carries no archive pointer. Timelines survive only as a +per-rendition `hls` field for HLS compatibility. ## Plan @@ -68,7 +68,7 @@ archive (per-track `.timeline.z` records named by the catalog's root - **Seek by time.** A `Time(pts)` range end ([Subscribe ranges](/quest/m1/subscribe-ranges/README.md)) is resolved by whoever holds the track, the cache or an archive consumer. DVR seeks with it. -- **`hls` is for HLS only, and opt-in.** A per-rendition catalog field names a +- **`hls` is for HLS only.** A per-rendition catalog field names a track whose (first group, pts) records mark each HLS segment, each rendition cutting its own on a pts grid so they line up wherever keyframes do (Catalog HLS field, landed). The root `archive` diff --git a/quest/m1/time-addressing/consumers.md b/quest/m1/time-addressing/consumers.md index a67af766638..cb7aa33256e 100644 --- a/quest/m1/time-addressing/consumers.md +++ b/quest/m1/time-addressing/consumers.md @@ -4,8 +4,9 @@ Every first-party consumer that numbered groups works by start timestamp, in Rust and JS, and hang stops writing duration and discontinuity markers -that group ends and gaps now carry. The HLS/DASH export lists segments -straight from group starts and ends, so the HLS records go away. +that group ends and gaps now carry. The HLS/DASH export cuts segments +from group starts and ends on the broadcast's declared join interval, so the +HLS records go away. ## Plan @@ -21,9 +22,40 @@ Decided 2026-10-25 (maintainer), from the spike: pause markers, which they already tolerate being absent. Decided 2026-10-27 (maintainer): moq-mux publishes HLS records by default -until this lands. Then the `.hls` record tracks, the catalog `hls` -field (hang draft and both catalog libraries), the `catalog::Config::with_hls` -opt-out, and any A/V segment alignment go away. +until this lands. Then the `.hls` record tracks, the per-rendition +`hls` catalog field, and any A/V segment alignment go away. + +Decided 2026-10-27 (maintainer, from moq.pro's HLS audit): how the export +cuts segments without records. HLS numbers segments by position, so every +edge and every reload must derive the same consecutive numbers from +timestamps alone, and a pts grid is the only stateless rule that does. +Rejected: one platform grid (long-GOP publishers overrun it constantly), +per-edge counting (failover renumbers), and a publisher counter (per-producer +state again). + +- **A catalog `media` section** holds what applies to every audio and video + track. `media.join.interval` promises that each one starts a group at least + that often, and is the export's grid, so every rendition shares segment + numbers. It is fixed for the broadcast instance, since the numbering + depends on it. The root `clock` moves to `media.clock` in the same break. Nested + so a later join mechanism that is not a fixed grid can sit beside it. + moq-mux declares the interval from its GOP hint, and `@moq/publish` does + too. A broadcast without one is not HLS-renderable. +- **Segment N starts at the group containing pts N × interval**, the group a + `Time` start at that pts resolves to, so an edge serves a segment from its + number alone. It runs up to the group containing the next line, which the + edge trims itself; ranges need no exclusive time end. The media sequence + number is N. +- **Whole groups only.** Never split a group: a segment starting mid-GOP is + not independent. +- **A line inside a pause or gap** is an `EXT-X-GAP` segment. +- **A group containing more than one line** (it overran the interval) is + listed at its first line, and each line it swallows is an `EXT-X-GAP` + segment, so numbering never skips. Its `EXTINF` understates the media; + document the deviation beside + [fixed target duration](/quest/m1/archive/hls-target.md)'s overrun rule, + whose target then comes from the interval. +- DASH lists the same segments by `$Time$`. Convert each consumer that numbers groups: hang and moq-mux timelines, HLS/DASH export, `seek` for GOP-aligned failover in moq-mux, HLS import's @@ -33,10 +65,11 @@ JS container consumers and `js/watch`. Update the hang draft and the docs each one makes stale; rename CLI flags per the repo's CLI rule. Public API: breaking where a consumer exposes group numbers (CLI -`--group`, archive keys), and `moq_mux::hls`, `hang::hls`, and -`catalog::Config::with_hls` are deleted. Wire: hang drops empty duration -frames, discontinuity marker groups, the `hls` catalog field, and the record -tracks. +`--group`, archive keys), `hang::hls` and the per-rendition `hls` field are +deleted, and `moq_mux::hls` shrinks to declaring the interval. Wire: hang +drops empty duration frames, discontinuity marker groups, the `hls` field, +and the record tracks, adds `media.join.interval`, and moves `clock` to +`media.clock`. ## Required From a239bf8c864b4d6cd2089df26bb297754f282047 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 00:02:19 -0700 Subject: [PATCH 2/5] quest: keep resumed groups and bound segment length on the grid Co-Authored-By: Claude Opus 5.5 --- quest/m1/time-addressing/consumers.md | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/quest/m1/time-addressing/consumers.md b/quest/m1/time-addressing/consumers.md index cb7aa33256e..28ce120597f 100644 --- a/quest/m1/time-addressing/consumers.md +++ b/quest/m1/time-addressing/consumers.md @@ -43,20 +43,30 @@ state again). too. A broadcast without one is not HLS-renderable. - **Segment N starts at the group containing pts N × interval**, the group a `Time` start at that pts resolves to, so an edge serves a segment from its - number alone. It runs up to the group containing the next line, which the - edge trims itself; ranges need no exclusive time end. The media sequence - number is N. + number alone. It runs up to where segment N + 1 starts, which the edge + trims itself; ranges need no exclusive time end. The media sequence number + is N. - **Whole groups only.** Never split a group: a segment starting mid-GOP is not independent. -- **A line inside a pause or gap** is an `EXT-X-GAP` segment. +- **A line inside a pause or gap** starts its segment at the first group + after it, so media resuming between lines is never dropped. If none starts + before the next line, the segment is `EXT-X-GAP`. - **A group containing more than one line** (it overran the interval) is listed at its first line, and each line it swallows is an `EXT-X-GAP` segment, so numbering never skips. Its `EXTINF` understates the media; document the deviation beside [fixed target duration](/quest/m1/archive/hls-target.md)'s overrun rule, whose target then comes from the interval. +- **Segments run under twice the interval** even without an overrun, since + the group containing a line can start up to one interval before it. Open + for the maintainer: a target of twice the interval (RFC 8216 compliant, + but players hold back twice as far) or the interval itself with longer + segments listed under the overrun rule. - DASH lists the same segments by `$Time$`. +Pin the segmentation with table-driven tests over pauses that end between +lines, unequal GOPs, and overruns. + Convert each consumer that numbers groups: hang and moq-mux timelines, HLS/DASH export, `seek` for GOP-aligned failover in moq-mux, HLS import's discontinuity packing, transcode's 1:1 numbering, archive keys and resume, From 2c6f706d408a071939566ef53255224a0d8a5407 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 00:02:31 -0700 Subject: [PATCH 3/5] quest: drop a private product name from the HLS grid plan Co-Authored-By: Claude Opus 5.5 --- quest/m1/time-addressing/consumers.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quest/m1/time-addressing/consumers.md b/quest/m1/time-addressing/consumers.md index 8ea7387ac00..0ec6eb0edec 100644 --- a/quest/m1/time-addressing/consumers.md +++ b/quest/m1/time-addressing/consumers.md @@ -25,7 +25,7 @@ Decided 2026-10-27 (maintainer): moq-mux publishes HLS records by default until this lands. Then the `.hls` record tracks, the per-rendition `hls` catalog field, and any A/V segment alignment go away. -Decided 2026-10-27 (maintainer, from moq.pro's HLS audit): how the export +Decided 2026-10-27 (maintainer, from an HLS audit): how the export cuts segments without records. HLS numbers segments by position, so every edge and every reload must derive the same consecutive numbers from timestamps alone, and a pts grid is the only stateless rule that does. From fa30a01030abb78afb0e24a3800ab8cc425a5c64 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 00:04:53 -0700 Subject: [PATCH 4/5] quest: scope the join interval promise to unpaused media Co-Authored-By: Claude Opus 5.5 --- quest/m1/time-addressing/consumers.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/quest/m1/time-addressing/consumers.md b/quest/m1/time-addressing/consumers.md index 0ec6eb0edec..a4b461bccd3 100644 --- a/quest/m1/time-addressing/consumers.md +++ b/quest/m1/time-addressing/consumers.md @@ -35,10 +35,11 @@ state again). - **A catalog `media` section** holds what applies to every audio and video track. `media.join.interval` promises that each one starts a group at least - that often, and is the export's grid, so every rendition shares segment - numbers. It is fixed for the broadcast instance, since the numbering - depends on it. The root `clock` moves to `media.clock` in the same break. Nested - so a later join mechanism that is not a fixed grid can sit beside it. + that often outside a pause, and is the export's grid, so every rendition + shares segment numbers. It is fixed for the broadcast instance, since the + numbering depends on it. The root `clock` moves to `media.clock` in the same + break. Nested so a later join mechanism that is not a fixed grid can sit + beside it. moq-mux declares the interval from its GOP hint, and `@moq/publish` does too. A broadcast without one is not HLS-renderable. - **Segment N starts at the group containing pts N × interval**, the group a From 82cef98a79f8ae282017a5f3cb250500b87af3b0 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 28 Oct 2026 00:08:22 -0700 Subject: [PATCH 5/5] quest: give every group one owning segment on the HLS grid Co-Authored-By: Claude Opus 5.5 --- quest/m1/time-addressing/consumers.md | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/quest/m1/time-addressing/consumers.md b/quest/m1/time-addressing/consumers.md index a4b461bccd3..f1df2aa5e47 100644 --- a/quest/m1/time-addressing/consumers.md +++ b/quest/m1/time-addressing/consumers.md @@ -42,20 +42,18 @@ state again). beside it. moq-mux declares the interval from its GOP hint, and `@moq/publish` does too. A broadcast without one is not HLS-renderable. -- **Segment N starts at the group containing pts N × interval**, the group a - `Time` start at that pts resolves to, so an edge serves a segment from its - number alone. It runs up to where segment N + 1 starts, which the edge - trims itself; ranges need no exclusive time end. The media sequence number - is N. +- **Each group belongs to one segment, numbered by grid line**: the first + line it contains, so an edge finds segment N from a `Time` start at pts + N × interval, the group that resolves to. A group containing no line + joins the segment of the group before it, or after a gap, the segment of + the line it starts after. The edge trims each segment to its own groups, + so ranges need no exclusive time end. The media sequence number is N. - **Whole groups only.** Never split a group: a segment starting mid-GOP is not independent. -- **A line inside a pause or gap** starts its segment at the first group - after it, so media resuming between lines is never dropped. If none starts - before the next line, the segment is `EXT-X-GAP`. -- **A group containing more than one line** (it overran the interval) is - listed at its first line, and each line it swallows is an `EXT-X-GAP` - segment, so numbering never skips. Its `EXTINF` understates the media; - document the deviation beside +- **A segment with no groups is `EXT-X-GAP`**, so numbering never skips + across a pause or an overrun. A group overrunning the interval is listed + at its first line, so its `EXTINF` understates the media; document the + deviation beside [fixed target duration](/quest/m1/archive/hls-target.md)'s overrun rule, whose target then comes from the interval. - **Segments run under twice the interval** even without an overrun, since @@ -66,7 +64,7 @@ state again). - DASH lists the same segments by `$Time$`. Pin the segmentation with table-driven tests over pauses that end between -lines, unequal GOPs, and overruns. +lines or past the next one, unequal GOPs, and overruns. Convert each consumer that numbers groups: hang and moq-mux timelines, HLS/DASH export, `seek` for GOP-aligned failover in moq-mux, HLS import's