diff --git a/quest/m0/README.md b/quest/m0/README.md index 55f21f35783..be7e4f2bad6 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -22,5 +22,8 @@ edge, which embeds moq-relay; it moved there on 2026-09-28. - [Nightly passes on main](/quest/m0/nightly/README.md) - the scheduled Nightly passes end to end, so red means a new break - [Release shutdown backport](/quest/m0/ffi-shutdown-backport.md) - `release` publishes a non-breaking moq-ffi with an async `drain()` and qmux sessions on the QUIC stream limit +- [Backport IETF close codes](/quest/m0/backport-ietf-close-codes.md) - `@moq/net` on `release` closes moq-transport sessions with the right error for three hostile cases +- [Backport @moq/flate bounds](/quest/m0/backport-flate-budget.md) - `@moq/flate` on `release` stops inflating past its cap and stops pinning pako scratch +- [Backport MKV, FLV, and VP9 import fixes](/quest/m0/backport-mux-import.md) - moq-mux on `release` refuses malformed imports instead of wrapping, panicking, or dropping tracks - [CI cost and latency](/quest/m0/ci/README.md) - cloud CI runs only in a batched merge gate as one `just ci` pass per batch, expensive checks run nightly and gate `release`, caches move to R2, and runners are chosen last - [More tests under load](/quest/m0/test-flakes-2/README.md) - load-only and queue failures, one quest per flake, fixed at the cause, since each one fails a whole batch diff --git a/quest/m0/backport-flate-budget.md b/quest/m0/backport-flate-budget.md new file mode 100644 index 00000000000..693f50ff293 --- /dev/null +++ b/quest/m0/backport-flate-budget.md @@ -0,0 +1,23 @@ +# [S] Backport the @moq/flate inflate bounds to release + +## Goal + +`@moq/flate` on `release` stops inflating a hostile slice once it passes its +cap and stops pinning pako's scratch buffer, as +[moq-dev/moq#6900](https://github.com/moq-dev/moq/pull/6900) fixed on main, and +publishes a patch release (npm has 0.1.0 to 0.1.3). + +## Plan + +Decided 2026-10-27 (maintainer): one PR against `release` per backport, in m0. + +`release`'s `js/flate/src/index.ts` is the older per-frame version (a 64 MiB +`maxFrameSize`, not main's 32 MiB group budget), so port the two fixes onto its +shape rather than main's budget: + +- `onData` only returns past the cap, so pako inflates the rest of the slice. + Stop inflation at the cap and fail the frame. +- `concat()` passes a single chunk through, pinning pako's scratch buffer. + Copy it out. + +Public API: none. Wire: none. diff --git a/quest/m0/backport-ietf-close-codes.md b/quest/m0/backport-ietf-close-codes.md new file mode 100644 index 00000000000..2926db29294 --- /dev/null +++ b/quest/m0/backport-ietf-close-codes.md @@ -0,0 +1,26 @@ +# [S] Backport the IETF close-code fixes to release + +## Goal + +`@moq/net` on `release` closes a moq-transport session with the right error for +the three cases [moq-dev/moq#6899](https://github.com/moq-dev/moq/pull/6899) +fixed on main, and publishes a patch release. + +## Plan + +Decided 2026-10-27 (maintainer): one PR against `release` per backport, in m0, +since these are hostile-input fixes in shipped packages. + +The bugs on `release` (as of 6ce67943f7): + +- `js/net/src/ietf/connection.ts:265`: a REQUEST_UPDATE as the initial message + aborts only the stream instead of closing the session with a protocol violation. +- `connection.ts:314-317`: an unsolicited PUBLISH_NAMESPACE logs and closes + cleanly instead of with an error. +- `subscriber.ts:807`: a `DuplicateTrackAlias` closes the session without its + error code. + +The announce-slot and SUBSCRIBE_TRACKS parts of #6899 are main-only and do not +apply. Port the regression tests that cover the three cases. + +Public API: none. Wire: close codes only, matching main. diff --git a/quest/m0/backport-mux-import.md b/quest/m0/backport-mux-import.md new file mode 100644 index 00000000000..bc7f4ffbe32 --- /dev/null +++ b/quest/m0/backport-mux-import.md @@ -0,0 +1,27 @@ +# [S] Backport the MKV, FLV, and VP9 import fixes to release + +## Goal + +moq-mux on `release` refuses the malformed MKV, FLV, and VP9 input that +[moq-dev/moq#6908](https://github.com/moq-dev/moq/pull/6908) made fail loud on +main, instead of wrapping timestamps, panicking, or silently dropping tracks, +and publishes a patch release. + +## Plan + +Decided 2026-10-27 (maintainer): one PR against `release` per backport, in m0. + +The bugs on `release` (as of 6ce67943f7): + +- `vp9/mod.rs:52,136-137`: `u16` width and height wrap at 65536. +- `flv/import.rs:220,364`: unsupported codecs are warned about and dropped. +- `mkv/import.rs:200`: `HierarchyProblems` allowed. +- `mkv/import.rs:246-247`: a TimestampScale of 0 accepted. +- `mkv/import.rs:298`: unsupported tracks skipped with a log line. +- `mkv/import.rs:314`: a duplicate TrackNumber is not refused. +- `mkv/import.rs:417`: the cluster timestamp plus relative timestamp overflows. + +Port each fix with its test. A dropped track that becomes an error is a +behavior change for `release` users; call it out in the changelog. + +Public API: none. Wire: none. diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index b9b75e152bf..1d4182c2255 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -137,4 +137,3 @@ The historical samples locate candidates; they are not controlled benchmarks. - [First shared FFI release](/quest/m0/ci/nightly-ffi-reuse.md) - condition: the backport merges and the next `moq-ffi-v*` tag runs `Release FFI`; verify the wrappers - [Fewer BTreeMap instantiations](/quest/m0/ci/btree-keys.md) - moq-net shares BTreeMap key and value types, kept only if its CPU time drops - [Halve the slowest tests' CPU](/quest/m0/ci/slow-tests.md) - the slowest 1% of tests take under half their 289 CPU-seconds, with coverage kept -- [Measure moving unit tests out](/quest/m0/ci/test-layout.md) - deprioritized: keep inline unit tests unless a profile shows moq-net's second library build is worth a migration diff --git a/quest/m0/ci/test-layout.md b/quest/m0/ci/test-layout.md deleted file mode 100644 index 30748956f56..00000000000 --- a/quest/m0/ci/test-layout.md +++ /dev/null @@ -1,36 +0,0 @@ -# [L] Measure moving unit tests out of hub crates - -## Goal - -Decide from measured CI CPU time whether hub crates, starting with moq-net, -should move unit tests into external harnesses (`[lib] test = false`) so the -library stops compiling a second time as the unit-test target. Until that -evidence exists, moq keeps its inline unit tests. - -## Plan - -Decided with the maintainer on 2026-10-25: measure before restructuring. - -Deprioritized 2026-10-26: a first pass found no evidence of savings and a -large migration cost, so keep the inline layout. Load-bearing facts, at main -5516a406f4: - -- About half of moq-net's source is test code, and its library harness has - about 2.7k tests. moq-mux has a similar share. -- Roughly 1k of those tests name private items. Moving them needs a - test-support surface that does not exist. The `fuzz` feature is not one: it - is public and implies `async`, which would break the no-default-feature - boundary. -- A four-case prototype worked mechanically but measured nothing: the shared - host was loaded and cache state differed between runs. -- Interop, bare-FIN, the feed/fetch benchmark, and `sh/rs/net-features.sh` - select library tests, so a migration must carry them too. - -Revisit only if a profile, after the thin-generics and BTreeMap work, shows the -unit-test target's library build is still a large share of a moq-net edit's -CPU. A credible comparison then needs a settled test-support design (or -private-state tests kept inline), a prototype that preserves every compiled test -across feature sets, and identical edit-plus-tests runs with mbx warm on a quiet -host. - -Public API: none, unless the test-support surface leaks; say so if it does. diff --git a/quest/m0/nightly/README.md b/quest/m0/nightly/README.md index b1ae1052197..a153222320e 100644 --- a/quest/m0/nightly/README.md +++ b/quest/m0/nightly/README.md @@ -33,3 +33,4 @@ Public API: none. Wire: none. - [moq-net compiles in Fuzz and send-depth](/quest/m0/nightly/send-overflow.md) - the ietf publisher future stops overflowing the Send check - [Probing beside Cubic](/quest/m0/nightly/competing-cubic.md) - the `competing_cubic` probe test stays within its deadline budget +- [cargo deny passes](/quest/m0/nightly/cargo-deny.md) - Audit stops failing on the yanked crypto-bigint and moq-studio's egui licenses diff --git a/quest/m0/nightly/cargo-deny.md b/quest/m0/nightly/cargo-deny.md new file mode 100644 index 00000000000..fe32a2278e0 --- /dev/null +++ b/quest/m0/nightly/cargo-deny.md @@ -0,0 +1,30 @@ +# [S] cargo deny passes on main + +## Goal + +`just rs audit` (`cargo deny check`) passes on main, so Nightly's Audit job +goes green. + +## Plan + +Audit has failed on every scheduled run since at least 2026-10-23 (for +example run [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673)): + +- Yanked `crypto-bigint` 0.7.8, through elliptic-curve 0.14 and p256/p384 + into moq-auth, moq-token, and their dependents. `cargo update -p + crypto-bigint`, then confirm nothing pins the yanked version. +- Licenses from `rs/moq-studio`'s egui stack + ([moq-dev/moq#6797](https://github.com/moq-dev/moq/pull/6797)): + `clipboard-win` and `error-code` (BSL-1.0), and `epaint_default_fonts` + (OFL-1.1, Ubuntu-font-1.0). + +Decided 2026-10-27 (maintainer): add BSL-1.0 to the global `allow` list (a +permissive, OSI-approved license), and give OFL-1.1 and Ubuntu-font-1.0 +per-crate `exceptions` for `epaint_default_fonts` only, so font licenses do not +spread to code. Studio keeps its clipboard and default fonts. + +The `RUSTSEC-2023-0071` (`rsa`) ignore stays: `rsa` still enters through +moq-auth's dev-only compatibility fixtures (published moq-auth 0.2.1 and +moq-token 0.7). Drop it when those fixtures move past `rsa`. + +Public API: none. Wire: none. diff --git a/quest/m0/test-flakes-2/README.md b/quest/m0/test-flakes-2/README.md index 7fa84ab1c0f..8d26c58301d 100644 --- a/quest/m0/test-flakes-2/README.md +++ b/quest/m0/test-flakes-2/README.md @@ -54,3 +54,4 @@ Public API: none. Wire: none. - [IETF22 fallback](/quest/m0/test-flakes-2/ietf22-fallback.md) - an IETF22 browser viewer follows a fallback within the media lane's bound, keeping its session and its audio - [Stuck-window eviction test on macOS](/quest/m0/test-flakes-2/stuck-window-evicts.md) - the evicted writer reliably sees its stop on macOS, fixed at the race - [Swapped Rust to JS Opus](/quest/m0/test-flakes-2/swapped-js-opus.md) - the `rust-swapped -> js` interop cell stops timing out under load +- [JS host-task yields follow the clock](/quest/m0/test-flakes-2/js-host-yields.md) - `Budget` and `nextTask` yield through the injected clock, so `Manual` tests stop racing them diff --git a/quest/m0/test-flakes-2/ffi-audio-gap.md b/quest/m0/test-flakes-2/ffi-audio-gap.md index 1b90e9df505..dc19831cc6f 100644 --- a/quest/m0/test-flakes-2/ffi-audio-gap.md +++ b/quest/m0/test-flakes-2/ffi-audio-gap.md @@ -3,8 +3,8 @@ ## Goal The interop tone check (`test/interop/tone.py`, "timestamp gap") stops -failing for FFI publishers: `python -> c` and the `cpp ->` cells seen in -[moq-dev/moq#6464](https://github.com/moq-dev/moq/pull/6464) and +failing for FFI publishers: `python -> c`, `go -> c`, and the `cpp ->` +cells seen in [moq-dev/moq#6464](https://github.com/moq-dev/moq/pull/6464) and [moq-dev/moq#6648](https://github.com/moq-dev/moq/pull/6648). ## Plan diff --git a/quest/m0/test-flakes-2/js-host-yields.md b/quest/m0/test-flakes-2/js-host-yields.md new file mode 100644 index 00000000000..e32768abcc0 --- /dev/null +++ b/quest/m0/test-flakes-2/js-host-yields.md @@ -0,0 +1,34 @@ +# [M] JS host-task yields follow the injected clock + +## Goal + +`Budget` and `nextTask` (`js/net/src/util/yield.ts`) yield through the +injected clock, so a `Manual` clock (`js/clock/src/manual.ts`) drives those +yields like its timers. Tests that advance a `Manual` clock pass whether or +not a `Budget` yields, and production keeps real host-task yields. + +## Plan + +Found while landing [moq-dev/moq#6905](https://github.com/moq-dev/moq/pull/6905), +whose new test raced the clock and was fixed per test with `clock.drive(...)`. +`Manual.advance` cannot see a host-task yield, so a test that advances past a +yielding loop observes state before the loop resumes. + +Decided 2026-10-27 (maintainer): the yields go through the clock, not +`Manual.advance` draining host tasks, which slows every advance and still +cannot tell when a yield chain has finished. + +Reproduce by forcing `Budget.poll` to always yield. These tests on main then +fail and are the acceptance set: + +- pooled probe readers share demand +- handle announcements sync with shared origin +- lite connection probes only while estimate observed +- listing past announces cap is reset +- integration: ietf blind handle picks up a publisher that arrives late (seen + once under a full `just check`) + +Once the clock owns the yields, drop per-test workarounds such as #6905's +`clock.drive(...)` where plain advances now suffice. + +Public API: `@moq/clock` gains whatever yield hook the clock needs. Wire: none. diff --git a/quest/m1/README.md b/quest/m1/README.md index 504e48c0079..71f33ce310a 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -165,6 +165,8 @@ surface and zero-copy texture import, land before the cut. - [Bindings decode media as media](/quest/m1/obs-data-decode.md) - moq-ffi decodes a media rendition with its kind, so OBS stops losing every other group - [Relay demand gaps](/quest/m1/relay-demand-gaps.md) - pre-07 backlog, IETF warm-relay newcomer, and double upstream pulls are fixed or ruled out - [fMP4 boundary sync](/quest/m1/fmp4-boundary-sync.md) - a declared fMP4 boundary opens a passthrough group only on a sync sample +- [Segment cuts hold bounded state](/quest/m1/segmenter-cuts.md) - the HLS segmenter keeps one pending cut, so `styp` input without sync samples cannot grow it +- [Next Object on a cold Rust copy](/quest/m1/next-object-cold-copy.md) - a join after an upstream group report starts at the first frame received, never skipping - [One moq-ffi build definition](/quest/m1/ffi-build-flags.md) - every binding takes moq-ffi's default features from one place - [A reported drop means the same everywhere](/quest/m1/resume-drop-final.md) - Rust and JS agree whether a group reported dropped can still be served - [moq-clock moves into kio::clock](/quest/m1/kio-clock.md) - moq-clock and its test macro become `kio::clock` and `#[kio::test]` before the cut publishes them diff --git a/quest/m1/api-audit/README.md b/quest/m1/api-audit/README.md index 6f0a1ee43f0..795b3c11bde 100644 --- a/quest/m1/api-audit/README.md +++ b/quest/m1/api-audit/README.md @@ -79,6 +79,8 @@ Rejected in planning: - [One lifecycle vocabulary in @moq/e2ee](/quest/m1/api-audit/lifecycle.md) - @moq/e2ee ends tracks and groups with finish and abort, as @moq/net does - [Demand is wanted, everywhere](/quest/m1/api-audit/wanted.md) - every `used`/`unused` pair becomes `wanted()` and `wanted_changed()`, and unused ones go - [One meaning for Credential](/quest/m1/api-audit/dial-credential.md) - the internal dial type gets its own name in JS and moq-tokio, and the `Presented` alias goes +- [One strict path rule](/quest/m1/api-audit/strict-paths.md) - `Relative` refuses empty segments, the error is `path::Error`, and auth mints only strict paths +- [Unix sockets accept through the allowlist](/quest/m1/api-audit/unix-accept.md) - the public pre-allowlist `unix::Listener::accept` becomes crate-private ## Related diff --git a/quest/m1/api-audit/strict-paths.md b/quest/m1/api-audit/strict-paths.md new file mode 100644 index 00000000000..7d05cc2f198 --- /dev/null +++ b/quest/m1/api-audit/strict-paths.md @@ -0,0 +1,31 @@ +# [S] One strict path rule, minted and parsed + +## Goal + +`path::Relative` refuses empty segments as `Path::new` does, the path error is +`path::Error`, and moq-auth and @moq/auth mint only roots and paths that parse +strictly, so a token can never grant a path no peer can spell. + +## Plan + +Follows [moq-dev/moq#6648](https://github.com/moq-dev/moq/pull/6648), which +made `Path::new` strict and introduced `path::Invalid { TooLong, EmptySegment }`. + +Decided 2026-10-27 (maintainer): + +- `path::Relative` refuses a leading, trailing, or doubled `/`, so both path + types share one rule. +- Rename `path::Invalid` to `path::Error`, same variants: the module's error, + named the way `encode::Config` is. +- #6648's other choices stay: relay grant-field path errors are `Forbidden`, + and `Path::from_static` panics on an invalid literal. +- moq-auth and @moq/auth refuse to mint a root or path that would not parse + strictly, in Rust and JS alike. + +Public API: breaking for moq-net (`path::Invalid` becomes `path::Error`, +`Relative::new` fails on empty segments), moq-auth, and @moq/auth. Mirror the +names in JS and the bindings. Wire: none for moq-lite, but the hang catalog +carries `Relative` in JSON (`broadcast`, `echo.path`), so a catalog spelling one +with a leading, trailing, or doubled `/` stops parsing. Our publishers never +emit those forms. @moq/net's `Path.normalizeRelative` and the hang draft's +`broadcast` field adopt the same rule. diff --git a/quest/m1/api-audit/unix-accept.md b/quest/m1/api-audit/unix-accept.md new file mode 100644 index 00000000000..453ba92a227 --- /dev/null +++ b/quest/m1/api-audit/unix-accept.md @@ -0,0 +1,24 @@ +# [S] Unix sockets accept only through the server's allowlist + +## Goal + +No public moq-tokio API runs the qmux handshake on a Unix socket before the +peer allowlist is checked. Embedders accept Unix peers through the server's +`unix` config, which checks `allow` against the peer credentials first. + +## Plan + +Found by [moq-dev/moq#6907](https://github.com/moq-dev/moq/pull/6907): the +public `unix::Listener::accept` (`rs/moq-tokio/src/unix.rs`) calls +`accept_pending(None)`, so it parses qmux from every local user and returns +`PeerCred` for the caller to check afterwards. The server path +(`spawn_unix_loop` in `server.rs`) passes `config.unix.allow` and refuses +before the handshake. + +Decided 2026-10-27 (maintainer): make the accept path crate-private rather +than move the allowlist onto the Listener. Nothing outside moq-tokio calls it, +so this deletes API with no consumer. Keep public only what the server config +needs, and log a refused peer once (it is logged twice today). + +Public API: breaking for moq-tokio; `unix::Listener::accept` (and anything +public only to serve it) goes. Wire: none. diff --git a/quest/m1/next-object-cold-copy.md b/quest/m1/next-object-cold-copy.md new file mode 100644 index 00000000000..02b0e6ccfb8 --- /dev/null +++ b/quest/m1/next-object-cold-copy.md @@ -0,0 +1,26 @@ +# [S] A Next Object join on a cold Rust copy starts at object 0 + +## Goal + +A downstream Next Object join that reaches a Rust relay after the upstream +reported a group, but before any of that group's frames arrived, receives the +group from its first frame received, never skipping frames. + +## Plan + +Found while landing [moq-dev/moq#6815](https://github.com/moq-dev/moq/pull/6815) +and not yet reproduced. `report_object` keeps the exact upstream Object ID +`k`, while readers count frames from 0, so the join may start at frame `k+1` +and skip the first `k+1` frames the relay then receives. That loses data, +unlike #6815's accepted precision trade-offs. JS avoids it since #6815's fix +for synthetic group-head reports (a separate `reportedGroup` hook). + +Reproduce it first with a Rust test that reports `{g, k}` upstream, then joins +with Next Object before any frame of `g` arrives. If it reproduces, keep the +reported Object ID out of the reader's frame index, as JS does. + +Public API: none. Wire: none. + +## Related + +- [Interop covers late join and FETCH](/quest/m1/interop-coverage/late-join.md) - the cross-implementation late-join case diff --git a/quest/m1/segmenter-cuts.md b/quest/m1/segmenter-cuts.md new file mode 100644 index 00000000000..49818433fd3 --- /dev/null +++ b/quest/m1/segmenter-cuts.md @@ -0,0 +1,39 @@ +# [S] Segment cuts hold bounded state + +## Goal + +The moq-mux HLS segmenter's pending cuts stay bounded, so an input that +declares segments faster than it opens groups cannot grow its state. + +## Plan + +Found while landing [moq-dev/moq#6783](https://github.com/moq-dev/moq/pull/6783). +`Segmenter.cuts` (`rs/moq-mux/src/hls.rs`) is a `VecDeque` that +`cut()` appends to without bound, and only a new group start drains it. Cuts +come from a declared fMP4 segment, including a `styp` box in network input +(for example `moq-cli` HLS ingest of a remote origin with `--hls`). An origin +that sends `styp` segments with rising pts and no sync sample never starts a +group, so the queue grows by one entry per segment. + +No compaction is equivalent to the queue. A group start drops the cuts at or +before its start, but the only lower bound on the next group start is the +current one: `container::Producer::write` lets a keyframe dip below the previous +group's content, and fMP4 validates rising DTS, not PTS. So with cuts at 10 and +20 and frames reported at both, a group at 15 still needs the 10 boundary, and +neither a lone `Option` nor collapsing against the latest frame keeps +it. Both callers in `container/fmp4/import.rs` declare a fragment's cut at its +earliest pts just before reporting that fragment's frames. + +Open question for the maintainer: which bounded policy drops a boundary when a +source declares more cuts than groups, given the ceiling already caps segment +length (for example, keep the earliest and latest pending cuts), or whether a +reorder window can supply a tighter lower bound. Add a test of the 10, 20, 15 +case. The existing `explicit_cuts_override_the_grid` test stays as a +regression guard: it declares both cuts before any group, so it needs both +boundaries. + +Public API: none. Wire: none. + +## Related + +- [fMP4 boundary sync](/quest/m1/fmp4-boundary-sync.md) - the same non-sync declared boundary, on the group side diff --git a/quest/m3/ci-host.md b/quest/m3/ci-host.md index 1c4f1244a6b..75db78808bf 100644 --- a/quest/m3/ci-host.md +++ b/quest/m3/ci-host.md @@ -32,8 +32,7 @@ Decided 2026-10-26 (maintainer): runner is deregistered. Rollback is re-registering it. - `nix/runner` (the persistent-VM module, rejected 2026-10-25) is deleted, with its `runner-vm` recipes and flake checks. It can land ahead as its own - PR. Then check whether `quest/m0/ci/check-without-userns.md` is still - needed, and update `quest/m2/gpu-ci.md`, which assumes the VM module. + PR. Then update `quest/m2/gpu-ci.md`, which assumes the VM module. - Independent of `quest land`: the host only changes which machine answers `vars.MOQ_QUEUE_RUNNER`. - If entries are slower than home, step up the host (more cores, or an OVH