Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
5374451
chore(quest): claim TS stream routing
kixelated Oct 22, 2026
28f27e9
fix(mux): give every TS PID a declared route
kixelated Oct 22, 2026
a4384f0
test(mux): cover every TS PID route
kixelated Oct 22, 2026
d04c93b
fix(mux): carry whole MPEG-2 pictures without starving audio
kixelated Oct 22, 2026
b907921
docs(mux): document TS routes and bench the PID accounting
kixelated Oct 22, 2026
11c9047
Merge origin/main
kixelated Oct 22, 2026
153a2d9
docs(mux): migrate note for TS stats rows
kixelated Oct 22, 2026
45e01a5
Merge origin/main
kixelated Oct 22, 2026
90d55c7
test(mux): a TS PID keeps one row through its PMT lifecycle
kixelated Oct 22, 2026
db75614
chore(quest): finish TS stream routes
kixelated Oct 22, 2026
fc80562
fix(mux): ignore a TS PMT on another program's PMT PID in Programs
kixelated Oct 22, 2026
9cb4901
Merge origin/main into stream-routes: rebuild PID accounting on the s…
kixelated Oct 25, 2026
efac015
fix(mux): group MPEG-1/2 video by sequence header
kixelated Oct 25, 2026
9c9fc1a
Merge remote-tracking branch 'origin/main' into stream-routes-iterate
kixelated Oct 25, 2026
9d66bd9
fix(mux): send the tables leading a placed PES by its placement
kixelated Oct 25, 2026
6f7e9b8
fix(mux): drop a partial section on a scrambled packet
kixelated Oct 25, 2026
e6848ba
Merge remote-tracking branch 'origin/main' into quest/m1/ts-reliabili…
kixelated Oct 25, 2026
28a947e
test(mux): clock the shared-budget test with audio
kixelated Oct 25, 2026
b4e3d5c
Merge remote-tracking branch 'origin/main' into HEAD
kixelated Oct 25, 2026
8c60315
fix(mux): find an MPEG-2 sequence header split across two PES
kixelated Oct 25, 2026
89ef969
fix(mux): settle a held MPEG-2 PES at a clock break
kixelated Oct 25, 2026
89e9976
Merge remote-tracking branch 'origin/main' into HEAD
kixelated Oct 25, 2026
8229354
Merge remote-tracking branch 'origin/main' into merge-6252
kixelated Oct 25, 2026
fae6c81
Merge remote-tracking branch 'origin/main' into merge-6252
kixelated Oct 26, 2026
aa1fc67
fix(mux): resume whole MPEG-1/2 video at a sequence header after a la…
kixelated Oct 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/ts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,15 @@ jobs:
HARNESS_RUNS: ${{ runner.temp }}/moq-test-runs
shell: bash -leo pipefail {0}

# MPEG-2 video the importer carries as whole PES: FFmpeg decodes the capture to the
# source's pictures, and the video passes the T-STD model.
- name: TS MPEG-2 video
if: ${{ !cancelled() }}
run: nix develop --command just test ts --mpeg2
env:
MOQ_TEST_RUNS: ${{ runner.temp }}/moq-test-runs
shell: bash -leo pipefail {0}

# A KLV stream joins the program mid-stream and leaves it: the export announces each
# layout at the next PMT version, and carries the stream only while a PMT lists it.
- name: TS PMT revision
Expand Down
11 changes: 8 additions & 3 deletions doc/bin/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,16 +94,21 @@ For a source already in MPEG-TS, `ffmpeg -c copy -f mpegts` re-muxes and drops P
or use [`import srt`](/bin/srt).

MPEG-TS import carries H.264/H.265 and AAC/MP2/AC-3/E-AC-3/Opus, and passes
SCTE-35 and every other stream it does not decode (KLV, subtitles, DVB AC-3)
through as tracks. Video AUs can span PES, and a PES can contain several. Supply
SCTE-35, DSM-CC sections, and every other stream it does not decode (MPEG-2
video, LATM AAC, KLV, subtitles, DVB AC-3) through as tracks. MPEG-2 video still
paces the program by its timestamps, and a subscriber joins it at a sequence
header. H.264/H.265 AUs can span PES, and a PES can contain several. Supply
a PTS for each AU or declare a proportional POC clock: AVC needs
`fixed_frame_rate_flag` and `temporal_poc_flag` in the TS timing descriptor plus
fixed-rate SPS VUI; HEVC needs VUI's `vui_poc_proportional_to_timing_flag`. Missing timing
ends the import instead of repeating a timestamp. Private and undecoded PES go through whole, at their place on the program clock
(PCR); one before the first PCR is dropped. A stream it cannot carry unaltered
is refused with a warning: any in a program with no PCR, and one whose PES carry
an ESCR, pack header or TREF, are scrambled, leave a non-video PES unbounded,
or have a malformed header.
or have a malformed header, as is `0x86` without a SCTE-35 registration, which
may be DTS audio. A PID no table claims, such as conditional access or ATSC
PSIP, is dropped with one warning and counted in the
[stats](/concept/stats#mpeg-ts-sections); `--passthrough` keeps them.
SCTE-35 cues and private PES share the media clock, or anchor on the first PCR
when the PMT declares only data. Audio-only and data-only programs round-trip
without video; a data-only program still requires PCR. A program declaring
Expand Down
12 changes: 11 additions & 1 deletion doc/concept/stats.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,17 @@ program clock since its last access unit. Its losses are `resyncs`, bytes
`discarded` scanning for audio sync, `unconfirmed` frames, `damaged` units,
`unplaced` units the program clock could not place, the times the stream was
`refused`, and its shares of `continuityCountError`, `transportError`, and
`ptsError`.
`ptsError`. A TS-scrambled packet on a carried stream counts as `refused`.

Every PID but the null PID either takes a route or is counted. A listed stream
the import cannot carry, such as `0x86` without a SCTE-35 registration, gets a
row with an empty `track` and one `refused`. Once every program's PMT is in, or
has been missing for 0.5 s of program clock, a PID that no PAT, PMT, or
captured SI table claims, such as the CAT, conditional access, or ATSC PSIP,
gets a `data` row with an empty `track` and `refused` counting its packets,
with one warning when it starts. An import of one `--program` of a
multi-program stream cannot tell another program's PIDs from these, so it
counts none of them.

The stream-wide counters count ETSI TR 101 290 checks at that standard's
fixed limits:
Expand Down
11 changes: 7 additions & 4 deletions drafts/draft-lcurley-moq-mpegts.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ type Verbatim = {
}
~~~

`streamType` is the PMT `stream_type` to re-announce: 0x86 for SCTE-35 {{scte35}}, 0x05 for private sections.
`streamType` is the PMT `stream_type` to re-announce: 0x86 for SCTE-35 {{scte35}}, 0x05 for private sections, 0x0A to 0x0D for ISO/IEC 13818-6 DSM-CC sections.

`framing` is required, and `"section"` is its only value: each frame is one complete section, header and CRC included, timestamped with its placement: the program clock at its first section byte, on the same timeline as a whole PES below.
A consumer MUST refuse a catalog whose `verbatim` record names any other `framing` or none, rather than guess; in particular it MUST NOT read the `"pes"` framing of draft -00, or a record without `framing`, as sections.
Expand All @@ -251,11 +251,12 @@ type PesPacket = {
}
~~~

`streamType` is the PMT `stream_type` to re-announce, as for `Verbatim`: 0x06 for private data such as asynchronous KLV, teletext, DVB subtitles and DVB AC-3, 0x15 for synchronous metadata.
`streamType` is the PMT `stream_type` to re-announce, as for `Verbatim`: 0x01 and 0x02 for MPEG-1 and MPEG-2 video, 0x11 for LATM AAC, 0x06 for private data such as asynchronous KLV, teletext, DVB subtitles and DVB AC-3, 0x15 for synchronous metadata.
Each frame MUST be exactly one PES packet, from its start code through its payload, with a `PES_packet_length` that matches the frame, or zero only where {{mpeg2}} lets a video stream leave it unbounded.
The `stream_id`, the PTS and DTS when present, and every other header field ride in the frame, with the PTS and DTS on the catalog clock; a PES without a PTS carries none.
The frame's timestamp is the packet's placement: the program clock at its first byte, on the same timeline.
Each frame MUST be a keyframe in its own group, as for `verbatim`.
Each frame MUST be a keyframe in its own group, as for `verbatim`, except on MPEG-1 and MPEG-2 video (0x01 and 0x02), whose pictures depend on earlier ones: there a PES in which a `sequence_header_code` (0x000001B3) begins, including one whose last bytes the next PES completes, MUST be a keyframe opening a group, and the PES up to the next one MUST join it.
A publisher drops the video PES before the first sequence header, and after a lost one until the next, rather than publish a picture without what it references.
A consumer MUST refuse a frame that is not one well-formed PES packet.
The record is a member beside `verbatim` rather than a `framing` value, so a consumer that predates it ignores the track instead of refusing the catalog.

Expand Down Expand Up @@ -370,7 +371,7 @@ This document defines no packaging value; {{msfts}} defines `mpeg2ts` for the pa
A consumer rebuilding a transport stream:

- MUST place each track on its recorded `pid`, and MUST assign an unused PID to a track with none.
- MUST put the PMT on `program.pmtPid` and build the PAT and PMT from `program`.
- MUST put the PMT on `program.pmtPid` and build the PAT and PMT from `program`. When it carries a NIT (the `si` entry for `table_id` 0x40 on PID 0x0010), the PAT MUST also list program 0 with that `network_PID`, so a receiver can find it.
- MUST choose a PCR PID and emit PCRs on it at the program clock's repetition interval. With no media stream it MUST use an unused PID with adaptation-only PCR packets and no PMT elementary-stream entry. It MUST NOT refuse a program for lacking video.
- MUST follow the catalog as it changes: when a track joins or leaves `tracks` or `programDescriptors` changes, it MUST emit a PMT at the next `version_number` (modulo 32) before any packet of a stream it adds, and MUST NOT emit a stream's packets after a PMT that no longer lists it. Each version it emits MUST differ from the one before, and no older version may follow a newer one. The PCR PID MUST NOT move until the stream it rides ends.
- MUST re-emit each track's `descriptors` as its ES-level descriptors, and `programDescriptors` as the PMT's `program_info`.
Expand Down Expand Up @@ -455,6 +456,8 @@ A broadcast demultiplexed from a DVB transport stream: video and audio described
- Section-framed `verbatim` timestamps use PCR-derived placement at the first section byte, as whole PES do, rather than a video presentation time.
- Added the `pesPacket` record: each frame one whole PES packet, timestamped with its placement. A consumer that predates it drops the track rather than refusing the catalog.
- `verbatim` carries sections only, breaking -00: `framing` is required and `"section"` its only value, and `streamId` is gone. A consumer refuses a -00 catalog with `"pes"` framing or a `verbatim` record without `framing`. A publisher carries an undecoded PES stream as `pesPacket`, and refuses one it cannot carry unaltered.
- MPEG-1/2 video and LATM AAC are named among `pesPacket` streams, and the DSM-CC types 0x0A to 0x0D among section-framed `verbatim` ones. A rebuilt PAT lists the network entry when the NIT is carried.
- MPEG-1/2 video `pesPacket` frames group from one sequence header to the next, rather than one PES per group.
- A flagged PCR jump on an `m2ts` track follows hang's discontinuity marker rather than a skipped group sequence.
- An `m2ts` track unwraps PCRs from its history, clamps extrapolated object times at a forward discontinuity, and cuts a fallback group after one second. `randomAccess` covers every published group; `muxRate` follows the measured source rate.
- A `tracks` entry is never changed in place: a stream the PMT revises continues under a new name, and one it drops leaves the section. `programDescriptors` follows the PMT; `program`'s `transportStreamId` and `programNumber` do not change.
Expand Down
9 changes: 4 additions & 5 deletions quest/m1/ts-reliability/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,9 +67,9 @@ refused means an error or a counted drop.
| Private and unknown PES types, LATM (0x11), metadata (0x15) | Preserved: `pesPacket`, PTS and DTS rebased | landed |
| PES clock fields (ESCR, pack header, TREF), PES scrambling | Refused, counted in `refused` | landed |
| Private sections (0x05), SCTE-35 (0x86 + `CUEI`) | Preserved: `verbatim` section, `pts_adjustment` rebased | landed |
| MPEG-1/2 video (0x01/0x02) | Preserved: `pesPacket`, keeping its clock role; today clock only, payload discarded | [routes](/quest/m1/ts-reliability/stream-routes.md) |
| DSM-CC types (0x0A to 0x0D), 0x86 without `CUEI` | Today PES-parsed into damage, or dropped uncounted | [routes](/quest/m1/ts-reliability/stream-routes.md) |
| CAT, EMM/ECM, ATSC PSIP, other SI PIDs, unlisted PIDs, TS scrambling | Outside the profile, raw lane; counted when the import reads every program. Today dropped uncounted | [routes](/quest/m1/ts-reliability/stream-routes.md) |
| MPEG-1/2 video (0x01/0x02) | Preserved: `pesPacket` grouped from one sequence header to the next, keeping its clock role | landed |
| DSM-CC types (0x0A to 0x0D), 0x86 without `CUEI` | DSM-CC preserved as a `verbatim` section; 0x86 without `CUEI` refused, counted in `refused` | landed |
| CAT, EMM/ECM, ATSC PSIP, other SI PIDs, unlisted PIDs, TS scrambling | Outside the profile, raw lane; counted in `refused`, unclaimed PIDs when the import reads every program | landed |
| DVB SI on 0x0010, 0x0011, 0x0012, 0x0014 | Preserved: snapshot track per PID and `table_id` | landed |
| Program identity, program and ES descriptors | Preserved: descriptors follow each PMT revision; an identity change refuses | landed |
| PMT add, remove, retype; PID reuse | Normalized without a new epoch: a changed or reused PID ends its track and continues under a fresh name, and the export rebuilds the PMT at the next `version_number` | landed |
Expand All @@ -79,7 +79,7 @@ refused means an error or a counted drop.
| Mux rate, null stuffing | Normalized: re-padded to `muxRate` while the source is CBR | landed |
| PSI and SI repetition | Normalized: PAT/PMT every 500 ms and at keyframes, SI at DVB maxima | landed |
| Media PES headers, TS packet layer | Normalized: rebuilt by export; the raw lane keeps them | landed |
| PAT network PID entry | Preserved: listed on 0x0010 when a NIT is carried; today dropped | [routes](/quest/m1/ts-reliability/stream-routes.md) |
| PAT network PID entry | Preserved: listed on 0x0010 when a NIT is carried | landed |

Decided in the 2026-10-22 root-cause audit: the rows above kept getting
patched one at a time because two foundations were missing. Decode time is
Expand All @@ -99,7 +99,6 @@ catalog linger (#5305).
## Required

- [PAT changes](/quest/m1/ts-reliability/pat-lifecycle.md) - a revised PAT never leaves a broadcast meaning another program
- [Stream routes](/quest/m1/ts-reliability/stream-routes.md) - every PID gets a declared route, and what is not carried is counted
- [TS workflow passes](/quest/m1/ts-reliability/validation-green.md) - the scheduled TS lanes pass on main again, with no lane loosened

## Related
Expand Down
4 changes: 3 additions & 1 deletion quest/m1/ts-reliability/pat-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ What the code does today:

- A PAT naming another PMT PID for the program adds a reassembler but never
drops the old one, and `program.pmtPid` keeps the first.
- `Programs` ignores a program a later PAT adds, without a warning.
- `Programs` ignores a program a later PAT adds, without a warning. Its
PIDs hold back the unclaimed-PID count for 0.5 s of program clock, then
count as unclaimed in `ts::stats`, since no program reads them.
- Under `--program all`, the shared parser hands a later PAT that removes one
program to every program's `Import`; the removed program's `handle_pat`
errors, `Programs::decode` propagates it, and the whole multiplex ingest
Expand Down
108 changes: 0 additions & 108 deletions quest/m1/ts-reliability/stream-routes.md

This file was deleted.

Loading
Loading