Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 2 additions & 7 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,8 @@ axum = { version = "0.8", default-features = false, features = [
axum-server = "0.8"
base64 = "0.23"
bitflags = "2"
# MSB-first bit reading for codec configs; the version h264-reader builds on.
bitstream-io = "4.10"
block2 = "0.6.2"
bytes = "1"
clap = { version = "4", features = ["derive"] }
Expand Down
1 change: 0 additions & 1 deletion quest/m2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,6 @@ lite-07; bitrate enforcement is not part of this release.
- [Auth session caps cost O(1)](/quest/m2/auth-session-index.md) - per-token and per-remote counts and a time-ordered expiry queue replace the scans, after #7211
- [The relay refuses a cluster peer it cannot dial](/quest/m2/relay-undialable-peers.md) - moq-tokio reports the schemes it can dial, and the relay refuses an undialable peer at startup instead of retrying forever
- [Drop moq-auth's iroh transport](/quest/m2/auth-drop-iroh.md) - the next breaking moq-auth release drops `Transport::Iroh` and the JS `"iroh"` value
- [H.264 parsing with maintained crates](/quest/m2/h264-reader.md) - moq-mux parses SPS VUI with h264-reader and shares one bit reader across codec config parsers
- [moq-transport finish interop](/quest/m2/ietf-finish.md) - the interop `finish` case runs on every even moq-transport draft instead of being skipped
- [JS joining FETCH](/quest/m2/ietf-js-joining-fetch.md) - the JS publisher answers a joining FETCH that arrives after a quick finish, mirroring the Rust bound
- [ashpd uses maintained D-Bus crates](/quest/m2/ashpd-maintained.md) - wait for a stable zbus 6 port, then remove the portal dependency exceptions
Expand Down
4 changes: 0 additions & 4 deletions quest/m2/aac-sbr-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,3 @@ Follow-up of [moq-dev/moq#7398](https://github.com/moq-dev/moq/pull/7398), which
Decided 2026-10-30 (maintainer, follow-up planning after the /quest-afk run). Add the field to moq-mux's public config and delete moq-audio's own parse; test an implicit and an explicit SBR config.

Public API: an additive field on `moq_mux::codec::aac::Config`. Wire: none.

## Required

- [Parse H.264 and codec config bits with maintained crates](/quest/m2/h264-reader.md) - its PR, #7398, rewrites both readers
7 changes: 3 additions & 4 deletions quest/m2/audit-deps.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,10 +144,9 @@ locks have none. Most duplicates are ecosystem transitions owned upstream:

### Hand-rolled code

Planned: [H.264 parsing](/quest/m2/h264-reader.md); moq-mux's avcC and
hvcC records go through mp4-atom, and relay metrics now render through
prometheus-client. Reviewed and kept,
each small or cold: about eleven `(d * 2).min(max)` backoff loops (moq-tokio
moq-mux's avcC and hvcC records go through mp4-atom, its H.264 SPS through
h264-reader, and relay metrics now render through prometheus-client. Reviewed
and kept, each small or cold: about eleven `(d * 2).min(max)` backoff loops (moq-tokio
already has a `Backoff` that moq-relay's cluster loop could reuse), per-byte
hex formatting in moq-auth and moq-hls, the STUN token bucket, the JS
deep-equal pair, and duplicated WebVTT stamps and `jwt=` lookups.
Expand Down
33 changes: 0 additions & 33 deletions quest/m2/h264-reader.md

This file was deleted.

5 changes: 3 additions & 2 deletions quest/m2/intra-refresh/h264-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@ follows the hang draft, which rounds up
([#5212](https://github.com/moq-dev/moq/pull/5212)).

- `rs/moq-mux/src/codec/h264/split.rs`: `sei_has_recovery_point` returns the
`recovery_frame_cnt` the `h264-parser` crate already exposes on
`SeiPayload::RecoveryPoint`, and the split frame carries it. A count of zero
`recovery_frame_cnt` that `h264-reader`'s
`nal::sei::recovery_point::RecoveryPoint::read` parses from the message, and
the split frame carries it. A count of zero
stays a plain keyframe. Recovery points with `broken_link_flag` set remain
out of scope, as before.
- The length-prefixed path bypasses the splitter: `rs/moq-mux/src/import/track.rs`
Expand Down
1 change: 0 additions & 1 deletion quest/m2/sps-fuzz-h264-reader.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,4 @@ Public API: none. Wire: none.

## Required

- [Parse H.264 with maintained crates](/quest/m2/h264-reader.md) - its PR, #7398, moves SPS parsing to h264-reader
- [Media parsers have fuzz targets](/quest/m2/media-fuzz.md) - its PR, #7313, adds the SPS target and the fuzz-findings epic
1 change: 1 addition & 0 deletions rs/moq-audio/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ pulseaudio = ["cpal?/pulseaudio"]
aec = ["capture", "playback"]

[dependencies]
bitstream-io = { workspace = true }
bytes = { workspace = true }
# Pure-Rust cross-platform audio I/O. Optional: only the `capture` and
# `playback` features need it. Encode stays on unsafe-libopus, so a build
Expand Down
38 changes: 10 additions & 28 deletions rs/moq-audio/src/aac.rs
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ fn declares_sbr(description: &[u8]) -> bool {

// The sync extension itself. Padding can't be mistaken for it: the pattern
// has to match, name SBR or PS, and then set the flag.
if bits.left() < 16 || bits.read(11)? != SYNC_EXTENSION {
if bits.read(11)? != SYNC_EXTENSION {
return Some(false);
}
if !matches!(object_type(bits)?, OBJECT_TYPE_SBR | OBJECT_TYPE_PS) {
Expand All @@ -151,7 +151,7 @@ fn declares_sbr(description: &[u8]) -> bool {
Some(bits.read(1)? == 1)
}

scan(&mut Bits::new(description)).unwrap_or(false)
scan(&mut Bits::endian(description, bitstream_io::BigEndian)).unwrap_or(false)
}

/// An audioObjectType, which escapes to a second field once it runs out of room.
Expand All @@ -163,34 +163,16 @@ fn object_type(bits: &mut Bits) -> Option<u8> {
}

/// A big-endian bit cursor, which is how an AudioSpecificConfig is packed.
struct Bits<'a> {
data: &'a [u8],
pos: usize,
}

impl<'a> Bits<'a> {
fn new(data: &'a [u8]) -> Self {
Self { data, pos: 0 }
}

fn left(&self) -> usize {
self.data.len() * 8 - self.pos
}
type Bits<'a> = bitstream_io::BitReader<&'a [u8], bitstream_io::BigEndian>;

/// The next `count` bits, or `None` when the config is shorter than that.
fn read(&mut self, count: usize) -> Option<u32> {
if count > 32 || self.left() < count {
return None;
}

let mut value = 0;
for _ in 0..count {
let bit = (self.data[self.pos / 8] >> (7 - self.pos % 8)) & 1;
value = (value << 1) | u32::from(bit);
self.pos += 1;
}
/// Reads the next `count` bits, or `None` when the config is shorter than that.
trait ReadBits {
fn read(&mut self, count: u32) -> Option<u32>;
}

Some(value)
impl ReadBits for Bits<'_> {
fn read(&mut self, count: u32) -> Option<u32> {
bitstream_io::BitRead::read_var(self, count).ok()
}
}

Expand Down
2 changes: 1 addition & 1 deletion rs/moq-mux/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ categories = ["multimedia", "network-programming", "web-programming"]
[dependencies]
anyhow = { workspace = true }
base64 = { workspace = true }
bitstream-io = { workspace = true }
bytes = { workspace = true }
crc = "3"
ebml-iterable = { workspace = true }
h264-parser = { version = "0.4.2" }
h264-reader = "0.9.0"
hang = { workspace = true }
kio = { workspace = true, features = ["runtime"] }
Expand Down
33 changes: 12 additions & 21 deletions rs/moq-mux/src/codec/aac/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ impl Config {
/// Read an AudioSpecificConfig's leading fields: the audioObjectType, sample rate, and
/// channelConfiguration.
fn read_header<T: Buf>(reader: &mut BitReader<T>) -> Result<(u8, u32, u8)> {
if reader.buf.remaining() < 2 {
if reader.remaining().is_none_or(|remaining| remaining < 2) {
return Err(Error::ConfigTooShort);
}
let object_type = read_object_type(reader)?;
Expand Down Expand Up @@ -370,51 +370,42 @@ const SAMPLE_RATES: [u32; 13] = [
96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350,
];

/// MSB-first bit reader that pulls bytes from a [`Buf`] on demand.
/// MSB-first bit reader that pulls bytes from a [`Buf`] on demand, so the buffer advances only
/// past the bytes it reads.
///
/// AudioSpecificConfig is bit-packed: an explicit 24-bit sample rate pushes the
/// following channelConfiguration off byte boundaries, so the fields can't be
/// read a whole byte at a time.
struct BitReader<'a, T: Buf> {
buf: &'a mut T,
current: u8,
bits_left: u8,
bits: bitstream_io::BitReader<bytes::buf::Reader<&'a mut T>, bitstream_io::BigEndian>,
/// Every bit read is copied here when set, with alignment redone on the writer's own bytes.
record: Option<BitWriter>,
}

impl<'a, T: Buf> BitReader<'a, T> {
fn new(buf: &'a mut T) -> Self {
Self {
buf,
current: 0,
bits_left: 0,
bits: bitstream_io::BitReader::new(buf.reader()),
record: None,
}
}

/// Bytes left in the buffer, `None` partway through one.
fn remaining(&mut self) -> Option<usize> {
Some(self.bits.reader()?.get_ref().remaining())
}

/// Skip to the next byte boundary.
fn align(&mut self) {
self.bits_left = 0;
bitstream_io::BitRead::byte_align(&mut self.bits);
if let Some(record) = &mut self.record {
record.align();
}
}

/// Read `n` bits (n <= 32) MSB-first, returning `short` if the buffer runs dry.
fn read(&mut self, n: u8, short: Error) -> Result<u32> {
let mut value = 0u32;
for _ in 0..n {
if self.bits_left == 0 {
if !self.buf.has_remaining() {
return Err(short);
}
self.current = self.buf.get_u8();
self.bits_left = 8;
}
self.bits_left -= 1;
value = (value << 1) | u32::from((self.current >> self.bits_left) & 1);
}
let value = bitstream_io::BitRead::read_var(&mut self.bits, n.into()).map_err(|_| short)?;
if let Some(record) = &mut self.record {
record.write(n, value);
}
Expand Down
70 changes: 64 additions & 6 deletions rs/moq-mux/src/codec/annexb.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,63 @@ use std::collections::BTreeMap;

use bytes::{Buf, Bytes, BytesMut};

/// A bit reader over an H.264 or H.265 parameter set or SEI NAL unit's RBSP: the bytes after its
/// `header`-byte NAL header, with emulation prevention removed as they are read.
///
/// These NAL units end in `rbsp_trailing_bits`, so a trailing zero byte is Annex B's
/// `trailing_zero_8bits` left behind by the splitter, and is dropped rather than refused as an
/// emulated start code.
pub(crate) fn rbsp(nal: &[u8], header: usize) -> h264_reader::rbsp::BitReader<h264_reader::rbsp::ByteReader<&[u8]>> {
h264_reader::rbsp::BitReader::new(rbsp_bytes_reader(nal, header))
}

/// The RBSP bytes of an H.264 or H.265 parameter set or SEI NAL unit, as [`rbsp`] reads them.
/// `None` when the payload holds a byte sequence emulation prevention forbids.
pub(crate) fn rbsp_bytes(nal: &[u8], header: usize) -> Option<Vec<u8>> {
use std::io::Read;

let mut out = Vec::with_capacity(nal.len());
rbsp_bytes_reader(nal, header).read_to_end(&mut out).ok()?;
Some(out)
}

/// The `(payloadType, payload)` of each message in an H.264 or H.265 SEI RBSP, which share the
/// `sei_message` syntax. A malformed message ends the iteration. Unlike h264-reader's
/// `SeiReader`, a payload of any size is skipped without a copy.
pub(crate) fn sei_messages(rbsp: &[u8]) -> impl Iterator<Item = (u32, &[u8])> {
let mut buf = rbsp;
std::iter::from_fn(move || {
// Each message needs at least a type and size byte; the last byte is the RBSP stop bit.
if buf.len() <= 2 {
return None;
}
let kind = sei_value(&mut buf)?;
let size = sei_value(&mut buf)?;
let (payload, rest) = buf.split_at_checked(size as usize)?;
buf = rest;
Some((kind, payload))
})
}

/// An SEI `payloadType` or `payloadSize`: a run of 0xFF bytes, each adding 255, then the remainder.
fn sei_value(buf: &mut &[u8]) -> Option<u32> {
let mut value = 0u32;
loop {
let (&byte, rest) = buf.split_first()?;
*buf = rest;
value = value.checked_add(byte.into())?;
if byte != 0xFF {
return Some(value);
}
}
}

fn rbsp_bytes_reader(nal: &[u8], header: usize) -> h264_reader::rbsp::ByteReader<&[u8]> {
let payload = nal.get(header..).unwrap_or_default();
let end = payload.iter().rposition(|&b| b != 0).map_or(0, |last| last + 1);
h264_reader::rbsp::ByteReader::without_skip(&payload[..end])
}

/// The four-byte Annex B start code preceding a NAL unit.
pub const START_CODE: Bytes = Bytes::from_static(&[0, 0, 0, 1]);

Expand Down Expand Up @@ -189,21 +246,22 @@ impl InBandCodec {
/// Insert parameter set `nal` (header included) under the id it defines, replacing any
/// earlier set with that id.
pub(crate) fn insert_parameter_set(self, sets: &mut ParameterSets, nal: &Bytes) -> Result<()> {
use h264_parser::{bitreader::BitReader, eg::read_ue};
use h264_reader::rbsp::BitRead;

let nal_type = self.nal_type(nal);
let header = match self {
Self::H264 => 1,
Self::H265 => 2,
};
let rbsp = h264_parser::nal::ebsp_to_rbsp(nal.get(header..).unwrap_or_default());
let r = &mut BitReader::new(&rbsp);
let r = &mut rbsp(nal, header);
let id = match (self, nal_type) {
// seq_parameter_set_id follows profile_idc, the constraint flags, and level_idc.
(Self::H264, 7) => r.skip_bits(24).and_then(|()| read_ue(r)),
(Self::H264, 7) => r
.skip(24, "profile_idc..level_idc")
.and_then(|()| r.read_ue("seq_parameter_set_id")),
// pic_parameter_set_id opens a PPS in both codecs.
(Self::H264, 8) | (Self::H265, 34) => read_ue(r),
(Self::H265, 32) => r.read_bits(4), // vps_video_parameter_set_id
(Self::H264, 8) | (Self::H265, 34) => r.read_ue("pic_parameter_set_id"),
(Self::H265, 32) => r.read::<4, u32>("vps_video_parameter_set_id"),
(Self::H265, 33) => crate::codec::h265::sps_id(r),
_ => return Err(Error::ParameterSetId(nal_type)),
};
Expand Down
Loading