Repository navigation
docs: keep the site on features and correct publisher restarts #5033
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+1,006
−3,425
Merged
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
73cc023
docs: keep the site on features and correct publisher restarts
kixelated 36e2765
docs: scope epoch claims to lite-07, add doc/AGENTS.md
kixelated ade212c
docs: second trim pass toward a feature entry point
kixelated 6c325f1
Merge origin/main into docs/entry-point
kixelated 8e46ec8
docs: audio groups end at any frame, one packet each by default
kixelated 489098f
docs(agents): point the moq-stats sync row at doc/concept/stats.md
kixelated 8ef0857
Merge origin/main into docs/entry-point
kixelated e2e556f
Merge remote-tracking branch 'origin/main' into merge-5033
kixelated 28b5768
docs: qualify restart takeover by clock and capture stamping by AEC a…
kixelated 23c99ee
Merge origin/main into docs/entry-point
kixelated dfb5514
docs: note --mtls-upstream needs --mtls-peer and which clock --delay …
kixelated 5887106
Merge origin/main into docs/entry-point
kixelated ecaebd2
docs: warn that low session limits starve the IETF request window, sc…
kixelated 58068e3
Merge origin/main into docs/entry-point
kixelated 83366d3
Merge origin/main into docs/entry-point
kixelated a0e3339
docs: warn against reusing an e2ee epoch, keep the Subscribe Options …
kixelated 0901d78
docs: no encrypted instance shares an epoch, replicas included
kixelated 197401b
Merge origin/main into docs/entry-point
kixelated ebe5c46
Merge origin/main into docs/entry-point
kixelated 52056a6
docs: route price is not static, cache capacity is a target
kixelated 7fa9954
docs: name the OBS encoding modes, mark gstreamer takeover opt-in, cl…
kixelated File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| `doc/` is the entry point to the repo: what exists, what it is for, and how to start. | ||
| It is not the reference. Rustdoc, TSDoc, `--help`, and `drafts/` are, and they sit next to the code so they stay accurate. | ||
|
|
||
| - Document features, and the behavior a user must know to choose or use one. | ||
| - Leave method and field lists, wire encodings, error codes, and per-version accounting to the API docs and drafts. Link to them. | ||
| - A code change touches `doc/` only when it changes what a reader would do. A fix, a new knob, or an edge case belongs in the API doc. | ||
| - Edit the paragraph that already covers the topic instead of appending a new one. When a page grows, cut something. | ||
| - Weight follows importance to a user, not how recently something changed. | ||
| - State a claim's conditions with it: opt-in or experimental versions, and the paths it does not cover. | ||
| - Describe only what ships. Plans live in quests. |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,129 +1,53 @@ | ||
| --- | ||
| title: HLS | ||
| description: Serve any broadcast as HLS, or import an HLS playlist | ||
| description: Serve a broadcast as HLS or DASH, or import an HLS playlist | ||
| --- | ||
|
|
||
| # HLS | ||
|
|
||
| `moq export hls` serves a MoQ broadcast as HLS over HTTP for players that | ||
| can't speak MoQ. `moq import hls` pulls a remote HLS master or media playlist | ||
| into a broadcast. | ||
| `moq export hls` serves a MoQ broadcast as HLS and DASH over HTTP for players | ||
| that can't speak MoQ. `moq import hls` pulls a remote HLS master or media | ||
| playlist into a broadcast. | ||
|
|
||
| ```bash | ||
| # Serve. Players open http://localhost:8089/my-stream.hang/master.m3u8 | ||
| # Serve. Players open http://localhost:8089/my-stream.hang/master.m3u8 (or manifest.mpd for DASH) | ||
| moq --connect https://relay.example.com/anon --broadcast my-stream.hang export hls --listen '[::]:8089' | ||
|
|
||
| # Import | ||
| moq --connect https://relay.example.com/anon --broadcast my-stream.hang import hls https://example.com/live/master.m3u8 | ||
| ``` | ||
|
|
||
| The CLI serves the one `--broadcast` it was given, at | ||
| `/{broadcast}/master.m3u8` and `/{broadcast}/manifest.mpd`. | ||
|
|
||
| ## How export works | ||
|
|
||
| Export never subscribes to media. It reads each rendition's | ||
| [timeline track](/concept/hang#catalog), a small index of that rendition's | ||
| groups, to build playlists, then fetches exactly the frames a requested segment | ||
| covers from the relay's cache and transmuxes them to CMAF on demand. So a | ||
| segment is servable for as long as the relay's | ||
| [cache](/bin/relay/config#cache) retains it, and idle renditions cost nothing. | ||
| Segment boundaries come from one reference rendition (the first video | ||
| rendition by name, or the first audio one without video) and are numbered by | ||
| its records. A video choice sticks while the rendition stays in the catalog, so | ||
| a newer rendition that sorts earlier does not rewind `EXT-X-MEDIA-SEQUENCE`; an | ||
| audio choice switches to video once a video rendition has a timeline. Every | ||
| other video rendition snaps each boundary to its nearest keyframe within about a | ||
| second, and a segment with none in range renders as `EXT-X-GAP`; audio takes every frame | ||
| inside the segment's span. A jump in content time renders as | ||
| `EXT-X-DISCONTINUITY`. Gaps are a fallback: a publisher wanting clean HLS | ||
| export should align video GOPs across renditions. A timeline that fails | ||
| (malformed, refused, or lost) is not retried, since the relay already rides out | ||
| transient source failures. The reference timeline failing ends every playlist | ||
| with `EXT-X-ENDLIST`, and any other rendition's timeline failing ends that | ||
| rendition's playlist at the last segment it covers. A broadcast | ||
| whose catalog advertises no timelines is skipped. One server exposes every | ||
| broadcast by path: | ||
| groups, to build playlists. A segment request fetches exactly the frames it | ||
| covers from the relay's cache and transmuxes them to CMAF. Media is never | ||
| transcoded, so the player must support the publisher's codecs. | ||
|
|
||
| ```text | ||
| /{broadcast}/master.m3u8 | ||
| /{broadcast}/manifest.mpd | ||
| /{broadcast}/{video|audio}/{rendition}/media.m3u8 | ||
| /{broadcast}/{video|audio}/{rendition}/init.{hash}.mp4 | ||
| /{broadcast}/{video|audio}/{rendition}/seg/{reference}.{segment}.m4s | ||
| /{broadcast}/{video|audio}/{rendition}/seg/{reference}.t{pts}.m4s | ||
| ``` | ||
| What that means for you: | ||
|
|
||
| Segment boundaries come from one reference rendition's timeline, and its records | ||
| number the segments. `{reference}` is a short hash of that rendition's kind and | ||
| name, so every edge that chose it derives the same URL. A reference that leaves | ||
| the catalog starts a new numbering under new URLs rather than reusing the old | ||
| ones for other content. | ||
| - **Idle renditions cost nothing**, and a segment is servable for as long as the relay's [cache](/bin/relay/config#cache) retains it. Keep `--window` (the playlist length, default 16 s) within that retention. | ||
| - **Joining costs the window, not the broadcast's age.** The exporter reads only what the timeline restates on join, and the window lists at most 256 segments. | ||
| - **Publish timelines.** A broadcast whose catalog advertises no timelines is skipped. | ||
| - **Align GOPs across video renditions.** Segment boundaries come from one reference rendition. Other video renditions snap to their nearest keyframe, and a segment with none nearby becomes an `EXT-X-GAP`. | ||
| - **Segment URLs are stable.** Every edge and every reload derive the same URLs, so a CDN in front of export caches them. | ||
|
|
||
| A [`moq-archive`](https://docs.rs/moq-archive) recording replayed through its | ||
| `Reader` is served the same way, with no second stored copy. Playlists come | ||
| from the replayed timelines alone, and a segment GETs only its rendition's | ||
| stored objects, so switching renditions never downloads both. An | ||
| inline-parameter-set codec with no catalog `description` is the exception: | ||
| the first playlist render GETs one keyframe group to build the init segment, | ||
| then caches it. Out-of-band configs need no media GET. A recording's playlists | ||
| list the same capped window as a live broadcast. A library embedder can set | ||
| `export::Config::history` to list past it instead: when the catalog's `archive` | ||
| entry names a `store` and no `replay` path, its spans are durable on this | ||
| broadcast, so only the recording's own retention trims the playlists and DASH | ||
| `timeShiftBufferDepth` is the listed span. The listing starts at the records | ||
| the timeline restates when the exporter joins (at most 256 from a `moq-mux` | ||
| publisher). The playlist ends with `EXT-X-ENDLIST` only once the reader's caller | ||
| declares the recording finished; the store holds no completion marker. | ||
|
|
||
| A viewer joining a days-old broadcast costs what the window lists, not the | ||
| broadcast's age: the exporter reads only the records its timeline restates on | ||
| join, never the whole history. The window lists at most 256 | ||
| segments however short they are, so every edge shows the same | ||
| `EXT-X-MEDIA-SEQUENCE`. Below that cap the window never evicts its newest | ||
| segment that starts on a video sync point, so a GOP longer than `--window` | ||
| stretches the window to that one GOP rather than leave nothing a player can | ||
| start at. A rendition with no media for a span lists that span as | ||
| a duration-preserving `EXT-X-GAP` and goes on listing after it. The master | ||
| playlist advertises a video rendition only once a listed segment starts at a | ||
| group start that is a sync point, so an early master may list audio alone, and | ||
| answers 404 while no rendition can start. Once advertised, a rendition stays in | ||
| the master for the rest of the publisher run, even if a GOP longer than 256 | ||
| records evicts its sync segment. | ||
| `Reader` is served the same way, with no second stored copy, and lists the | ||
| same window as a live broadcast. | ||
|
|
||
| The init URL carries a hash of its bytes, so a reconfigured rendition gets a | ||
| new one. An embedder of the library can also label the publisher's run with | ||
| `Broadcaster::set_generation`. Every segment URL then carries it | ||
| (`seg/{generation}.{reference}.{segment}.m4s`), since a restarted publisher reuses segment | ||
| numbers for different media. | ||
|
|
||
| `--window` sets the playlist duration (default 16 s) and caps segment | ||
| `Cache-Control: max-age` for every broadcast, | ||
| `--listen-tls-cert`/`--listen-tls-key` or `--listen-tls-generate` serve HTTPS, | ||
| and `--cors-origin` opens it to browsers. | ||
| H.264/H.265 and AAC/Opus renditions are served. Import handles classic HLS; | ||
| LL-HLS parts are not implemented yet. It publishes the playlist's own media | ||
| times, and the catalog clock maps the first imported segment to the time it | ||
| arrived, one mapping for every rendition. The library is | ||
| [`moq-hls`](https://docs.rs/moq-hls). | ||
|
|
||
| ## Recording segments | ||
|
|
||
| `moq_hls::export::segments::Segment::discontinuity` is the absolute timeline | ||
| sequence within one `Broadcaster`. Cursors created at different times report | ||
| the same sequence for the same timeline span, including after a rendition | ||
| rebinds. Skipping unavailable segments does not reset the sequence. | ||
| and `--cors-origin` opens it to browsers. See `moq export hls --help`. | ||
|
|
||
| HLS numbers a segment as `EXT-X-DISCONTINUITY-SEQUENCE` plus the | ||
| `EXT-X-DISCONTINUITY` tags before it (RFC 8216, section 6.2.1), and matching | ||
| content in every rendition must share that number. A recorder writes the first | ||
| retained segment's value as `EXT-X-DISCONTINUITY-SEQUENCE`, then writes | ||
| `current - previous` `EXT-X-DISCONTINUITY` tags before each later segment. The | ||
| difference can exceed one when the cursor skipped every segment of an epoch, and | ||
| one tag per change would then fall behind sibling renditions. Retain the | ||
| absolute value in the index alongside each segment. | ||
| ## Import | ||
|
|
||
| Recreating the broadcaster starts a new sequence namespace. Start a new | ||
| recording/playlist, or explicitly map the new broadcaster's sequences into a | ||
| recording-wide sequence with a discontinuity at the boundary. Do not compare | ||
| raw values across broadcaster instances or infer a restart from zero alone. | ||
| Import handles classic HLS; LL-HLS parts are not implemented. It publishes the | ||
| playlist's own media times, and anchors the catalog clock on the first imported | ||
| segment. | ||
|
|
||
| When updating moq.pro's moq-hls pin to this breaking release, update its recorder | ||
| and index together: store the returned sequence directly instead of accumulating | ||
| per-cursor break counts. Existing indexes containing counts need conversion | ||
| within their original recording namespace before combining them with new data. | ||
| The library, including the recorder cursors for mirroring a broadcast to | ||
| storage, is [`moq-hls`](https://docs.rs/moq-hls). |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.