Skip to content

A catalog describes what a track is: let encoders state it, and keep channel measurements out #7067

Closed

Description

@michaelstuurman

A catalog usually describes what a track is (codec, resolution, frame rate, bitrate, group structure), as set by whoever encodes it. It is not a live measurement feed. MSF-01 §5.3 fixes a track's attributes once declared ("MUST NOT be modified after being declared"), and §5.1 publishes a catalog "only when the availability of tracks changes". HLS describes a variant from its encoding as well: EXT-X-STREAM-INF (draft-pantos-hls-rfc8216bis-22 §4.4.6.2) takes BANDWIDTH, before encoding finishes, from "a representative period of similar content, encoded using the same settings".

moq-mux already leans this way: a supplied field is authoritative (estimate.rs), and measurement fills gaps for passthrough sources whose encoding is unknown (#2530, #4576). Through moq-c, an SDI encoder publishing H.264 720p/360p plus Opus for 32 s produced:

Setup Catalog versions
no hints 68
video bitrate/framerate hints 46
all hints in the linked PR 16

Of those 16, six are structural (start, a source restart, end). The other ten are:

  • delay estimates (×8, ~8 ms). The encoder can't know this up front, because it depends on runtime encode latency.
  • hls records added once each track's first group is written (×5, before the PR's group-duration hint).

The linked PR adds the facts an encoder knows: an audio bitrate hint, bitrate on JSON/binary tracks, and a group-duration hint that lists the HLS records in the first catalog. Everything is an appended has_*/value pair, so zeroed structs behave as today.

Questions

  1. Should jitter and delay leave the catalog (at least the MSF one)? Both describe delivery timing rather than the track. A receiver measures arrival jitter itself (@moq/watch's auto latency does), and a B-frame reorder depth is already in the codec configuration (SPS/VUI, and hang's reorder). MSF has no such fields, and rs/moq-msf marks them as non-standard extensions. If they stay, could a publisher opt out per track or per broadcast, so they are neither estimated nor supplied?
  2. Should the MSF catalog republish for estimate changes at all? Hang allows in-place updates; MSF-01 §5.3 doesn't. After feat(catalog)!: measure jitter and delay from timestamps, not the wall clock #6595, what difference between hang and MSF do you want here?
  3. Do HLS-bridge records belong in the published catalog, or in a side track, so a viewer that doesn't need them never sees them change?

Activity

  1. coderabbitai commented on Oct 28, 2026

    @coderabbitai
    Contributor
    🔗 Related PRs

    #6783 - feat(hang)!: opt-in per-rendition hls records replace the root archive [merged]
    #6527 - refactor(mux): m1 simplification audit [merged]
    #6937 - feat(mux)!: publish HLS records by default [merged]
    #6781 - feat(rate)!: native publishers disable simulcast rungs the grant cannot fund [merged]


    📝 Issue Planner

    Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.

    • Create Plan

    🧪 Issue enrichment is currently in open beta.

    You can configure auto-planning by selecting labels in the issue_enrichment configuration.

    To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

    issue_enrichment:
      auto_enrich:
        enabled: false

    💬 Have feedback or questions? Drop into our discord!

  2. added
    questBeing tracked/planned in a quest. See `quest/`
    on Oct 28, 2026
  3. kixelated commented on Oct 28, 2026

    @kixelated
    Collaborator

    Thanks for the numbers. Planned in #7124 as quest/m1/catalog-estimates.md:

    1. jitter and delay stay in hang. @moq/watch, moq-audio and the TS export use them as latency floors, and a receiver can't measure cross-track skew. Instead of an opt-out, hang republishes them only when a rise crosses the next multiple of the track's frame duration, which folds the ~8 ms delay creep into one update.
    2. MSF differs from hang. It skips identical republishes. With no hint, it waits until every track's first group has closed, then freezes the estimated fields (MSF-01 §5.3) and republishes only for track-list or identity changes.
    3. HLS records go away entirely with the planned time-addressed export. A declared media.join.interval replaces them, and feat(moq-c)!: supply bitrate and group duration hints #7068's group_duration_us hint then declares that interval.

    #7068 is reviewed on its own.

    (Written by Claude Opus 5.5)

  4. kixelated commented on Oct 28, 2026

    @kixelated
    Collaborator

    The plan from #7124 is in the merge queue. In short: jitter and delay stay in hang, but a rise republishes only when it crosses the next multiple of the frame duration. MSF skips duplicate catalogs. With no hint, MSF waits until each track's first group closes, then freezes its estimates. There is no opt-out.

    quest/m1/catalog-estimates.md tracks it. To donate tokens, run /quest-start quest/m1/catalog-estimates.

    (Written by Claude Opus 5.5)

  5. michaelstuurman commented on Oct 29, 2026

    @michaelstuurman
    ContributorAuthor

    Thanks for planning this. Quantizing fixes the churn. Before it lands, I'd like to propose a different home for the measured values, one MSF already has.

    What the publisher measures is worth keeping. jitter and delay come from the publisher's write order: the largest media span written together, a container's PTS−DTS, and how far a track's timestamps trail what another track already wrote. That is clean, with no network in it. A receiver can measure the same skew from arrivals once media flows live, but not while it joins: a joining FETCH or a relay cache delivers the head of a group in a burst. So a late joiner does need the publisher's figure. I agree with that part.

    Part of it belongs in the catalog. The reorder delay is the largest PTS−DTS offset: the SPS/VUI gives max_num_reorder_frames, CMAF carries composition offsets, and hang's reorder already declares it. An encode budget and the group duration are configuration. Those are facts of the encoding, ideally supplied by the encoder as #7068 does for bitrate, with a promised bound as MSF-01's buffers.max (§5.2.9).

    The runtime part doesn't. Warm-up, load and restarts vary. As a field that only rises, one start-up spike sets the floor for the whole broadcast. For example, with libmoq 0.6.17, video encode warm-up could peak at ~550 ms against a steady ~255 ms. delay then settled at ~423 ms, and every viewer buffered ~440 ms instead of ~290 ms. MSF freezing its estimates at first declaration would keep such a spike too.

    MSF's Media Timeline already carries it. MSF-01 §7 records [mediaPTS, [group, object], wallclock] in the first object of each group, for the tracks in its depends, and a catalog may hold several timelines. With one timeline per media track, a receiver reads the source delay as the wallclock difference between tracks at the same PTS, and the source jitter as the spread of wallclock − PTS. Both come from the publisher's clock alone, so no viewer clock sync is needed. A late joiner gets the current values from the latest group, they fall back after a spike, and the catalog never republishes for them. A safe bound is then the player's choice: the maximum over the timeline it has read, or the catalog's promised bound.

    This is how RTP splits it (RFC 3550): SDP describes the stream, RTCP Sender Reports map media time to an NTP wallclock (§6.4.1), and the receiver computes interarrival jitter itself (§6.4.1, A.8). It matters most for ingest, which MoQ will carry a lot of: an RTSP or SRT source arrives with network jitter the publisher can't know up front, and a gateway can fill the timeline from the source's Sender Reports.

    So the proposal:

    1. Catalog: configured values and promised bounds only, changing with the track list or a track's identity (MSF-01 §5.3). For MSF, no non-standard jitter/delay estimates: MSF-01 has neither.
    2. Timing: a Media Timeline per media track, with the wallclock taken when the object is handed to the transport. hang could carry the same, beside its root clock (which maps PTS zero once).
    3. Receiver: derives source delay and jitter from the timelines and adds what it measures on the network, as @moq/watch already does for arrival spread.

    Two points need the MSF draft, and I'll raise them there: which instant the timeline's wallclock marks ("encoded" isn't precise enough for delay), and keeping timelines per track rather than per render group (moq-wg/msf#203), since tracks aligned in PTS aren't aligned in when they're produced.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questBeing tracked/planned in a quest. See `quest/`

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions