From 197d85d4253dd53021da101ccfb2aa0d88acdee8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 24 Oct 2026 17:02:57 -0700 Subject: [PATCH 1/2] quest: plan follow-ups from the 10-24 quest-complete run Adds a Nightly-green epic under m0 (six children), five load flakes, two CI papercuts, the JS readvertisement hold, and moq-c's request refusal, and parks the claim-request quest behind announce takeover. Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 1 + quest/m0/ci/README.md | 2 + quest/m0/ci/mbx-low-disk.md | 25 +++++++++++ quest/m0/ci/select-launcher-path.md | 16 ++++++++ quest/m0/nightly/README.md | 41 +++++++++++++++++++ quest/m0/nightly/advisories.md | 17 ++++++++ quest/m0/nightly/drill-credit.md | 23 +++++++++++ quest/m0/nightly/feature-shapes.md | 24 +++++++++++ quest/m0/nightly/qmux-fallback-close.md | 21 ++++++++++ quest/m0/nightly/shallow-base.md | 15 +++++++ quest/m0/nightly/ts-schedule.md | 21 ++++++++++ quest/m1/README.md | 1 + quest/m1/api-audit/claim-request-type.md | 14 +++++++ quest/m1/c/README.md | 1 + quest/m1/c/request-refuse.md | 18 ++++++++ quest/m1/ietf-js-readvertise.md | 22 ++++++++++ quest/m1/test-flakes-2/README.md | 5 +++ quest/m1/test-flakes-2/dart-leak-rss.md | 17 ++++++++ quest/m1/test-flakes-2/gst-pinned-replay.md | 15 +++++++ .../m1/test-flakes-2/macos-receive-credit.md | 18 ++++++++ .../m1/test-flakes-2/mux-missing-sequence.md | 15 +++++++ quest/m1/test-flakes-2/rtsp-stall-timers.md | 17 ++++++++ 22 files changed, 349 insertions(+) create mode 100644 quest/m0/ci/mbx-low-disk.md create mode 100644 quest/m0/ci/select-launcher-path.md create mode 100644 quest/m0/nightly/README.md create mode 100644 quest/m0/nightly/advisories.md create mode 100644 quest/m0/nightly/drill-credit.md create mode 100644 quest/m0/nightly/feature-shapes.md create mode 100644 quest/m0/nightly/qmux-fallback-close.md create mode 100644 quest/m0/nightly/shallow-base.md create mode 100644 quest/m0/nightly/ts-schedule.md create mode 100644 quest/m1/c/request-refuse.md create mode 100644 quest/m1/ietf-js-readvertise.md create mode 100644 quest/m1/test-flakes-2/dart-leak-rss.md create mode 100644 quest/m1/test-flakes-2/gst-pinned-replay.md create mode 100644 quest/m1/test-flakes-2/macos-receive-credit.md create mode 100644 quest/m1/test-flakes-2/mux-missing-sequence.md create mode 100644 quest/m1/test-flakes-2/rtsp-stall-timers.md diff --git a/quest/m0/README.md b/quest/m0/README.md index d6a4a485771..e79e34f1aca 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -26,6 +26,7 @@ Each lands its fixes in its own PR and plans only what needs a decision. ## Required - [MALFORMED_AUTH_TOKEN session code](/quest/m0/malformed-auth-token-code.md) - a 0x16 close reports a malformed token instead of `unknown code=22`, and moq-net sends it for an unparseable token +- [Nightly passes on main](/quest/m0/nightly/README.md) - the scheduled Nightly passes end to end, so red means a new break - [Release shutdown backport](/quest/m0/ffi-shutdown-backport.md) - `release` publishes a non-breaking moq-ffi with an async `drain()` and qmux sessions on the QUIC stream limit - [Adopt quest resolve](/quest/m0/quest-resolve.md) - local merges resolve concurrent quest link-list edits once kixelated/quest ships its merge driver - [CI cost and latency](/quest/m0/ci/README.md) - cloud CI runs only in the merge queue as one `just ci` pass, expensive checks run nightly and gate `release`, caches move to R2, and runners are chosen last diff --git a/quest/m0/ci/README.md b/quest/m0/ci/README.md index 5a17251d99f..29dc8e476dc 100644 --- a/quest/m0/ci/README.md +++ b/quest/m0/ci/README.md @@ -90,11 +90,13 @@ The historical samples locate candidates; they are not controlled benchmarks. - [Require the release gate](/quest/m0/ci/release-gates.md) - condition: the next cut carries `release-gate.yml`; then release PRs require it - [Share local check capacity](/quest/m0/ci/local-gate.md) - gate expensive phases and bound internal fanout +- [select.sh launcher path](/quest/m0/ci/select-launcher-path.md) - `sh/rs/select.sh` runs from a subdirectory by resolving its launcher before `cd` - [Build queue entries concurrently](/quest/m0/ci/queue-concurrency.md) - the queue builds two or more entries at once instead of one - [Share Python check and test preparation](/quest/m0/ci/python-setup.md) - prepare editable bindings once per combined CI pass while preserving standalone commands - [Parallelize OBS source checks](/quest/m0/ci/obs-parallel.md) - check independent translation units concurrently when OBS is on the critical path - [Measure nightly compiler levers](/quest/m0/ci/nightly-levers.md) - the parallel frontend and Cranelift, adopted only if the numbers beat their risks - [Bump to the manifest fix](/quest/m0/ci/mbx-bump.md) - condition: an mbx release contains jdx/mr-boxington#798; then bump the Rust cache's pin - [Verify the R2 Rust cache](/quest/m0/ci/mbx-r2-verify.md) - measure the hosted Nightly jobs' hits, restore time, and expiry once the bump lands +- [mbx on a full disk](/quest/m0/ci/mbx-low-disk.md) - gc never evicts a live build's trees, and the Dart check names the real cause - [Verify the Nix cache](/quest/m0/ci/nix-cache-verify.md) - confirm CI and releases use `nix.moq.dev` and retire Cachix - [First shared FFI release](/quest/m0/ci/nightly-ffi-reuse.md) - condition: the first `moq-ffi-v*` tag that runs `Release FFI`, and a `Shipping sizes` run; verify the wrappers and size keys diff --git a/quest/m0/ci/mbx-low-disk.md b/quest/m0/ci/mbx-low-disk.md new file mode 100644 index 00000000000..d236396d2d3 --- /dev/null +++ b/quest/m0/ci/mbx-low-disk.md @@ -0,0 +1,25 @@ +# [S] mbx keeps a live build's trees on a full disk + +## Goal + +A CI job never rebuilds moq-ffi mid-run because mbx garbage collection +evicted the generated source trees its build is using, and when a Dart build +hook does rebuild, its error names the cause. + +## Plan + +Seen three times in about 34 hours (merge-group runs of +[moq-dev/moq#6449](https://github.com/moq-dev/moq/pull/6449), +[moq-dev/moq#6256](https://github.com/moq-dev/moq/pull/6256), and +[moq-dev/moq#6436](https://github.com/moq-dev/moq/pull/6436)): the runner +disk dropped below mbx's 50 GiB free-space minimum, gc evicted trees the +job had restored, and the Dart build hook rebuilt moq-ffi. Then +`sh/dart/check.sh` reported "update hook_vars in sh/dart/generate.sh", which +sends the reader to the wrong file. + +- Either mbx gc skips trees a live build holds, or the runner keeps enough + free disk that gc never runs mid-job; pick by measurement. +- The check's message says the hook rebuilt and why it may have (stale + `hook_vars` or an evicted tree). + +Public API: none. Wire: none. diff --git a/quest/m0/ci/select-launcher-path.md b/quest/m0/ci/select-launcher-path.md new file mode 100644 index 00000000000..b1d826232ef --- /dev/null +++ b/quest/m0/ci/select-launcher-path.md @@ -0,0 +1,16 @@ +# [XS] select.sh resolves its launcher before cd + +## Goal + +`cd sh/rs && bash select.sh lint --all` runs the selected checks instead of +failing with `bash: ./../local.sh: No such file or directory`. + +## Plan + +[moq-dev/moq#6256](https://github.com/moq-dev/moq/pull/6256) re-execs +`sh/rs/select.sh` through `sh/local.sh` after changing to the repository +root, so a relative `$0` no longer resolves +([review thread](https://github.com/moq-dev/moq/pull/6256#discussion_r4366860705)). +Resolve both paths before the `cd`, or name them relative to the root. + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/README.md b/quest/m0/nightly/README.md new file mode 100644 index 00000000000..39fd3c3db0b --- /dev/null +++ b/quest/m0/nightly/README.md @@ -0,0 +1,41 @@ +# Nightly passes on main + +## Goal + +The scheduled Nightly on `main` passes end to end, so a red run means a new +break instead of noise. Nightly is the backstop for what the PR gate skips on +purpose (benches, examples, feature shapes; see +[moq-dev/moq#5531](https://github.com/moq-dev/moq/pull/5531)), and it has +failed every night since at least 2026-10-19, which let a bench and two +example breaks reach `main` unnoticed. + +## Plan + +Decided 2026-10-24 (maintainer), from a triage of run +[40090959108](https://github.com/moq-dev/moq/actions/runs/40090959108): one +child per distinct cause, each landing on its own. The PR gate keeps +skipping benches and examples; a green Nightly catches them within a day. + +This README's own work, after the children: + +- Nightly's multi-step jobs (Features first) keep going past a failed step + and fail at the end, so one break never hides the steps after it. +- Confirm a scheduled run passes. Failures already owned elsewhere stay + there: the audio near-zero p95 row belongs to + [audio jitter target](/quest/m1/audio-jitter-target/README.md), the + macOS `receive_credit` timeout to + [test flakes](/quest/m1/test-flakes-2/README.md), and the paired bench + threshold to [bench CI](/quest/m1/bench-ci.md). The `cli-export` + bufferbloat failure is expected to clear with + [moq-dev/moq#6656](https://github.com/moq-dev/moq/pull/6656). + +Public API: none. Wire: none. + +## Required + +- [Claimed fetches wait for credit](/quest/m0/nightly/drill-credit.md) - the impaired burst drills stop losing groups to "no stream credit" +- [Feature shapes in the gate](/quest/m0/nightly/feature-shapes.md) - feature-gated dead code fails the PR that adds it, not the next Nightly +- [qmux fallback closes cleanly](/quest/m0/nightly/qmux-fallback-close.md) - the WebSocket fallback startup benchmark stops failing "not cleanly closed" +- [ts_schedule regression](/quest/m0/nightly/ts-schedule.md) - the moq-mux `ts_schedule` benches return to their old cost +- [Nightly's base in a shallow checkout](/quest/m0/nightly/shallow-base.md) - `sh/gh/nightly.test.py` passes in Nightly's depth-1 checkout +- [Advisory bumps](/quest/m0/nightly/advisories.md) - both audits pass on today's advisories diff --git a/quest/m0/nightly/advisories.md b/quest/m0/nightly/advisories.md new file mode 100644 index 00000000000..44e3aae34c5 --- /dev/null +++ b/quest/m0/nightly/advisories.md @@ -0,0 +1,17 @@ +# [XS] Advisory bumps + +## Goal + +Nightly's Audit and JS audit jobs pass on the advisories open on 2026-10-24. + +## Plan + +- RUSTSEC-2026-0387: `crypto-bigint` 0.7.5 to at least 0.7.6. +- GHSA-xjj6-f33q-929c (high), GHSA-4qv8-2p9j-2685, GHSA-f72v-6w45-q472: + `undici` 7.29.1 to at least 7.30.1, reached through + `@moq/demo-carla > wrangler > miniflare`. + +New advisories keep arriving, so the audits fail again on their own; the +standing review belongs to the [dependency audit](/quest/m2/audit-deps.md). + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/drill-credit.md b/quest/m0/nightly/drill-credit.md new file mode 100644 index 00000000000..ba34f8d8084 --- /dev/null +++ b/quest/m0/nightly/drill-credit.md @@ -0,0 +1,23 @@ +# [M] Claimed fetches wait for credit + +## Goal + +`bursts_cross_a_cluster`, `bursts_cross_a_transport_peer`, and +`bursts_cross_a_flapping_peer` (`::impaired`, `rs/moq-relay/tests/drills.rs`) +deliver every group again, and drill-sensitivity's +`ingress-burst-never-collected` baseline passes unmutated. + +## Plan + +Broken by [moq-dev/moq#6409](https://github.com/moq-dev/moq/pull/6409) +("refuse native requests without stream credit"), reproduced on `main`: the +drills panic at `drills.rs:1305` with groups lost as +`Failed("no stream credit")`. A relay's claimed FETCH under a burst is now +refused when the peer's stream credit is exhausted, where it used to wait. + +Find which hop refuses and decide by behavior, not by the test: a request a +relay makes on a subscriber's behalf should queue for credit (backpressure) +rather than fail, unless the refusal is the bound #6409 meant to add against +a hostile peer. Keep that bound; never loosen the drill's assertion. + +Public API: none expected. Wire: none. diff --git a/quest/m0/nightly/feature-shapes.md b/quest/m0/nightly/feature-shapes.md new file mode 100644 index 00000000000..62acbcebd97 --- /dev/null +++ b/quest/m0/nightly/feature-shapes.md @@ -0,0 +1,24 @@ +# [S] Feature shapes in the gate + +## Goal + +Code that is dead under one of a crate's feature shapes fails the PR that +adds it, instead of breaking Nightly's Features job the next morning. + +## Plan + +Decided 2026-10-24 (maintainer). The Features job ("Check feature and +profile variants") has failed every night on a different feature-gated item: +`moq-net/src/model/bandwidth.rs` (10-22), `moq-net/src/transport.rs` (10-23), +and now `Connection::open_request` +(`rs/moq-tokio/src/quic/driver/connection.rs:378`), which +[moq-dev/moq#6409](https://github.com/moq-dev/moq/pull/6409) added without +`#[cfg(feature = "net")]`; only `quic/web/session.rs` uses it. Reproduce: +`cargo clippy -p moq-tokio --all-targets --no-default-features --features driver,aws-lc-rs -- -D warnings`. + +- Fix `open_request`'s gating. +- Add the feature and profile variant check to the PR gate, scoped to the + crates a branch changes the way `just check` selects them. Measure its + cost in the PR. Nightly keeps the full matrix. + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/qmux-fallback-close.md b/quest/m0/nightly/qmux-fallback-close.md new file mode 100644 index 00000000000..1715c37d3f2 --- /dev/null +++ b/quest/m0/nightly/qmux-fallback-close.md @@ -0,0 +1,21 @@ +# [S] qmux fallback closes cleanly + +## Goal + +Nightly's "WebSocket fallback startup benchmark" (`just js qmux-bench`, in +the Audio quality job) passes again instead of failing with +`WebSocketError: WebSocket was not cleanly closed`. + +## Plan + +New on 2026-10-24 and reproduced locally on `main`; it passed on 10-23. +Candidates in that window are +[moq-dev/moq#6409](https://github.com/moq-dev/moq/pull/6409) (qmux refuses +exhausted credit) and +[moq-dev/moq#6458](https://github.com/moq-dev/moq/pull/6458) (qmux abort and +close without the peer's help); the later qmux fixes +[moq-dev/moq#6644](https://github.com/moq-dev/moq/pull/6644) and +[moq-dev/moq#6687](https://github.com/moq-dev/moq/pull/6687) do not fix it. +Bisect, then fix the close at its cause. + +Public API: none. Wire: none expected. diff --git a/quest/m0/nightly/shallow-base.md b/quest/m0/nightly/shallow-base.md new file mode 100644 index 00000000000..636fb21b259 --- /dev/null +++ b/quest/m0/nightly/shallow-base.md @@ -0,0 +1,15 @@ +# [XS] Nightly's base in a shallow checkout + +## Goal + +`test_paired_base_is_the_previous_scheduled_nightly` +(`sh/gh/nightly.test.py:46`) passes in Nightly's Tests matrix. + +## Plan + +The test runs `git rev-parse HEAD~1`, but the Tests matrix in `nightly.yml` +checks out with the default `fetch-depth: 1`, so it exits 128. Added by +[moq-dev/moq#6511](https://github.com/moq-dev/moq/pull/6511). Make the test +build its own history instead of reading the checkout's. + +Public API: none. Wire: none. diff --git a/quest/m0/nightly/ts-schedule.md b/quest/m0/nightly/ts-schedule.md new file mode 100644 index 00000000000..37eb7aeec30 --- /dev/null +++ b/quest/m0/nightly/ts-schedule.md @@ -0,0 +1,21 @@ +# [S] ts_schedule regression + +## Goal + +`rust/shared/ts_schedule/{1/1, 1/16, 4/1}` (`rs/moq-mux/benches/ts_schedule.rs`) +return to their cost before the regression, and Nightly's paired gate on +the shared bench targets passes. + +## Plan + +Nightly's first paired run (0513c07369 against 4381af6f0) measured them +400 to 460% slower, too large for noise. The window holds +[moq-dev/moq#6417](https://github.com/moq-dev/moq/pull/6417), +[moq-dev/moq#6533](https://github.com/moq-dev/moq/pull/6533), and +[moq-dev/moq#6527](https://github.com/moq-dev/moq/pull/6527). Bisect, then +fix the cost at its source, or show the bench now measures more work and +rebaseline with that reason. The smaller `server_admission` (+20%) and +`uring-session_lite/no-gso` (+11%) deltas belong to the threshold +calibration in [bench CI](/quest/m1/bench-ci.md). + +Public API: none. Wire: none. diff --git a/quest/m1/README.md b/quest/m1/README.md index b355b87a40a..b0b6dfe5b9b 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -152,6 +152,7 @@ Binding durations wait on Duration support in the generated C. - [API Extractor checks](/quest/m1/api-extractor-checks.md) - API Extractor fails a PR on a forgotten export, an unresolved declaration, or an undocumented export, replacing the hand-rolled `.d.ts` checkers - [Generated JS API reference](/quest/m1/js-api-reference.md) - the doc site renders an API reference for every `@moq/*` package from API Extractor's doc model - [IETF lazy discovery](/quest/m1/ietf-lazy-discovery.md) - JS moq-transport sessions never hold overlapping SUBSCRIBE_NAMESPACE requests at a peer that refuses them +- [JS readvertisement hold](/quest/m1/ietf-js-readvertise.md) - a JS IETF session handles a namespace readvertisement only after its withdrawal, as Rust does - [Draft 14-16 session errors](/quest/m1/ietf-adapter-session-errors.md) - the moq-transport adapter keeps session-level error causes, so a terminal rejection is not redialed - [Interop coverage](/quest/m1/interop-coverage/README.md) - the riskiest moq-transport features moq-net exposes get cross-implementation tests over the even drafts 14 to 22 - [AMF0 exact reads](/quest/m1/rtmp-amf0-exact-reads.md) - a truncated RTMP string fails with `UnexpectedEof` instead of parsing NUL padding diff --git a/quest/m1/api-audit/claim-request-type.md b/quest/m1/api-audit/claim-request-type.md index 07112d9ecaf..02ed19653e8 100644 --- a/quest/m1/api-audit/claim-request-type.md +++ b/quest/m1/api-audit/claim-request-type.md @@ -30,4 +30,18 @@ Kept from [moq-dev/moq#5678](https://github.com/moq-dev/moq/pull/5678): prepare and publish, one epoch per answer, and the cold-start reset until [announce takeover](/quest/m1/announce-takeover.md) lands. +Parked 2026-10-24 (maintainer) until announce takeover lands: #6496's +conformance tests subscribe through a `dynamic()` claim and get `Unroutable` +from the cold-start reset (the worker's answer arrives under a fresh epoch), +not from a missing route; `dynamic()` already builds the same route as +`dynamic_route` minus `publishes: false`. Takeover removes the reset at its +source. On resuming, main has new out-of-crate callers to move: `wide_hop.rs`, +moq-hls `export/rendition.rs`, moq-cli `play/media.rs`, moq-mux +`benches/export.rs`, and moq-gst `source/imp.rs`, which used the removed +`Publisher::standalone` and needs a redesign. + Public API: moq-net breaking (three hidden items go private). Wire: none. + +## Required + +- [Announcement takeover](/quest/m1/announce-takeover.md) - removes the cold-start reset the claim's first subscribe hits diff --git a/quest/m1/c/README.md b/quest/m1/c/README.md index 67938fd2b9b..7c245e04c0b 100644 --- a/quest/m1/c/README.md +++ b/quest/m1/c/README.md @@ -70,3 +70,4 @@ requests internally and keeps its ABI. - [C consumers](/quest/m1/c/consumers.md) - the C interop client and `doc/lib/c` samples move onto the generated API - [Async failure contract](/quest/m1/c/async-failure-contract.md) - a generated async call that fails early fires no callback, and one that started always delivers its terminal callback - [Durations in generated C](/quest/m1/c/duration.md) - the C backend renders UniFFI's `Duration` instead of refusing it +- [Refuse only waiting readers](/quest/m1/c/request-refuse.md) - freeing or aborting a group request refuses its readers instead of dropping the group for everyone diff --git a/quest/m1/c/request-refuse.md b/quest/m1/c/request-refuse.md new file mode 100644 index 00000000000..f625abb9561 --- /dev/null +++ b/quest/m1/c/request-refuse.md @@ -0,0 +1,18 @@ +# [S] moq-c refuses only the waiting readers + +## Goal + +`moq_group_request_free` and `moq_group_request_abort` refuse the group to +the readers waiting on that request, as Rust's `Fill::refuse_groups` does, +instead of marking it dropped for every later reader. + +## Plan + +[moq-dev/moq#6171](https://github.com/moq-dev/moq/pull/6171) added +`Fill::refuse_groups` and moved Rust's fill paths onto it; moq-c's +`group_request_free` (`rs/moq-c/src/publish.rs:893`) still calls +`drop_groups`. The C signatures stay; only the behavior changes, so note it +in the PR's Impact. If the generated C API replaces the hand-written group +request calls first, delete this quest. + +Public API: moq-c behavior change, no signature change. Wire: none. diff --git a/quest/m1/ietf-js-readvertise.md b/quest/m1/ietf-js-readvertise.md new file mode 100644 index 00000000000..57154400ab0 --- /dev/null +++ b/quest/m1/ietf-js-readvertise.md @@ -0,0 +1,22 @@ +# [S] JS holds a readvertisement until its withdrawal is handled + +## Goal + +On drafts 14 to 16, a JS moq-transport session handles a PUBLISH_NAMESPACE +that follows a PUBLISH_NAMESPACE_DONE for the same namespace only after the +withdrawal's handler has finished, as Rust does. + +## Plan + +[moq-dev/moq#6530](https://github.com/moq-dev/moq/pull/6530) fixed Rust +(`Readvertised` in `rs/moq-net/src/ietf/adapter.rs`, test +`a_readvertisement_waits_for_the_withdrawal_to_be_handled`); without it +`epochs::ietf14` failed every run. `js/net/src/ietf/adapter.ts` treats +PUBLISH_NAMESPACE_DONE as a plain CloseStream, so a readvertisement can +overtake it. Mirror the hold and its test. + +Not planned (2026-10-24, maintainer): the same race per track, where +PUBLISH_DONE routes as CloseStream and a new PUBLISH as NewRequest. It was +found by reading the code and never reproduced. + +Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/README.md b/quest/m1/test-flakes-2/README.md index d1a4566ad24..f702a38b10b 100644 --- a/quest/m1/test-flakes-2/README.md +++ b/quest/m1/test-flakes-2/README.md @@ -44,3 +44,8 @@ Public API: none. Wire: none. - [Tone check grace](/quest/m1/test-flakes-2/tone-grace.md) - the interop tone check charges its grace only for audible playback, so a refilling ring under CPU starvation is not a failure - [IETF22 fallback](/quest/m1/test-flakes-2/ietf22-fallback.md) - an IETF22 browser viewer follows a fallback within the media lane's bound, keeping its session and its audio - [Go reservation grant](/quest/m1/test-flakes-2/go-reservation-grant.md) - the Go reservation test waits for its grant +- [macOS receive credit](/quest/m1/test-flakes-2/macos-receive-credit.md) - `partial_control_requests_give_way` passes on the macOS lane, after checking for a shared platform cause +- [moq-mux missing sequence](/quest/m1/test-flakes-2/mux-missing-sequence.md) - `missing_sequence_skips_on_live_track` runs on a paused clock +- [moq-rtsp stall timers](/quest/m1/test-flakes-2/rtsp-stall-timers.md) - the 200 ms stall and keyframe tests run on a mocked clock +- [moq-gst pinned replay](/quest/m1/test-flakes-2/gst-pinned-replay.md) - the pinned-instance finish test waits on events instead of sleep loops +- [Dart leak RSS](/quest/m1/test-flakes-2/dart-leak-rss.md) - the leak tests measure what moq owns, not process RSS diff --git a/quest/m1/test-flakes-2/dart-leak-rss.md b/quest/m1/test-flakes-2/dart-leak-rss.md new file mode 100644 index 00000000000..ac69f84c449 --- /dev/null +++ b/quest/m1/test-flakes-2/dart-leak-rss.md @@ -0,0 +1,17 @@ +# [S] Dart leak tests measure moq's memory + +## Goal + +The three RSS checks in `dart/moq_ffi/test/leak_test.dart` (lines 15, 36, +66) fail only when moq leaks, never from Dart heap growth or allocator +retention under load. + +## Plan + +Each compares `ProcessInfo.currentRss` before and after 2000 calls of 64 KiB +and asserts growth under 32 MiB, with no forced GC; `dart_test.yaml` +already runs them with `concurrency: 1`. Measure what moq itself owns, such +as a native allocation counter or live-handle count exposed for tests, +instead of whole-process RSS. + +Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/gst-pinned-replay.md b/quest/m1/test-flakes-2/gst-pinned-replay.md new file mode 100644 index 00000000000..c7154f7b978 --- /dev/null +++ b/quest/m1/test-flakes-2/gst-pinned-replay.md @@ -0,0 +1,15 @@ +# [S] The moq-gst pinned-instance test waits on events + +## Goal + +`a_pinned_instance_winning_its_path_back_does_not_replay_its_finish` +(`rs/moq-gst/src/source/imp.rs:3960`) never fails under load. + +## Plan + +It drives a real GStreamer element against a real-time origin and polls +with `std::thread::sleep(50ms)` loops capped at 200 iterations, racing the +pipeline threads. Wait on the announce and pad events themselves, and drive +the origin's timers with a `moq_clock::Manual` clock. + +Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/macos-receive-credit.md b/quest/m1/test-flakes-2/macos-receive-credit.md new file mode 100644 index 00000000000..555ce5ed8d2 --- /dev/null +++ b/quest/m1/test-flakes-2/macos-receive-credit.md @@ -0,0 +1,18 @@ +# [S] macOS receive-credit timeout + +## Goal + +`receive_credit::partial_control_requests_give_way` +(`rs/moq-tokio/tests/integration/receive_credit.rs:502`) passes on Nightly's +macOS lane. + +## Plan + +It panics "stop timed out" on macOS (real sockets, 10 s timeout) while all +nine `receive_credit` tests pass on Linux. macOS failed a different test on +each of 10-22, 10-23, and 10-24 (10-23: +`quic::driver::tests::stuck_window_evicts_the_lower_priority`), so first +check for a shared platform cause, such as socket buffer sizes or timer +resolution, before fixing tests one by one. + +Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/mux-missing-sequence.md b/quest/m1/test-flakes-2/mux-missing-sequence.md new file mode 100644 index 00000000000..13bc004850e --- /dev/null +++ b/quest/m1/test-flakes-2/mux-missing-sequence.md @@ -0,0 +1,15 @@ +# [XS] The moq-mux missing-sequence test on a mocked clock + +## Goal + +`missing_sequence_skips_on_live_track` +(`rs/moq-mux/src/container/consumer.rs:2537`) never fails under load. + +## Plan + +It wraps the read in a real `tokio::time::timeout(1s)` while the skip waits +on the consumer's `hole_timer` (`moq_clock::sleep_until`, 100 ms +`max_delay`) on wall time. Pause the clock the way its sibling +`gap_keeps_late_arriving_group_within_max_delay` does. + +Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/rtsp-stall-timers.md b/quest/m1/test-flakes-2/rtsp-stall-timers.md new file mode 100644 index 00000000000..e64f05da276 --- /dev/null +++ b/quest/m1/test-flakes-2/rtsp-stall-timers.md @@ -0,0 +1,17 @@ +# [S] moq-rtsp stall tests on a mocked clock + +## Goal + +`a_silent_session_stalls` and `rtcp_alone_stalls` +(`rs/moq-rtsp/src/client.rs:938`, `:954`) and the two `keyframe_timeout` +tests (`:979`, `:1096`) never fail under load. + +## Plan + +They set 200 ms timeouts against a real-socket fake server on wall time. +`client.timeout` also bounds every RTSP request step (`within()` at `:266`), +so a loaded runner can stall during setup instead of at the point under +test. Drive the timeouts with a `moq_clock::Manual` clock, or move the fake +server off real sockets. + +Public API: none. Wire: none. From 86e09760ade85e336a06488169f77765e6e965ab Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 24 Oct 2026 17:16:14 -0700 Subject: [PATCH 2/2] quest: tighten four follow-up plans from review The mbx check names how it tells the causes apart, the Dart leak tests measure native live bytes rather than handles, the macOS reference points at the test, and the RTSP stall tests require a mocked clock. Co-Authored-By: Claude Opus 5.5 --- quest/m0/ci/mbx-low-disk.md | 5 +++-- quest/m1/test-flakes-2/dart-leak-rss.md | 7 ++++--- quest/m1/test-flakes-2/macos-receive-credit.md | 4 ++-- quest/m1/test-flakes-2/rtsp-stall-timers.md | 4 ++-- 4 files changed, 11 insertions(+), 9 deletions(-) diff --git a/quest/m0/ci/mbx-low-disk.md b/quest/m0/ci/mbx-low-disk.md index d236396d2d3..f99cde59ca4 100644 --- a/quest/m0/ci/mbx-low-disk.md +++ b/quest/m0/ci/mbx-low-disk.md @@ -19,7 +19,8 @@ sends the reader to the wrong file. - Either mbx gc skips trees a live build holds, or the runner keeps enough free disk that gc never runs mid-job; pick by measurement. -- The check's message says the hook rebuilt and why it may have (stale - `hook_vars` or an evicted tree). +- The check tells the two causes apart before naming one: it blames + `hook_vars` only when they differ from `sh/dart/generate.sh`, and otherwise + reports that the restored tree was missing. Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/dart-leak-rss.md b/quest/m1/test-flakes-2/dart-leak-rss.md index ac69f84c449..efdc1de2256 100644 --- a/quest/m1/test-flakes-2/dart-leak-rss.md +++ b/quest/m1/test-flakes-2/dart-leak-rss.md @@ -10,8 +10,9 @@ retention under load. Each compares `ProcessInfo.currentRss` before and after 2000 calls of 64 KiB and asserts growth under 32 MiB, with no forced GC; `dart_test.yaml` -already runs them with `concurrency: 1`. Measure what moq itself owns, such -as a native allocation counter or live-handle count exposed for tests, -instead of whole-process RSS. +already runs them with `concurrency: 1`. Measure the live bytes moq's native +allocator holds, exposed for tests, instead of whole-process RSS. A +live-handle count is not enough: per-call buffers such as `demand.name()`'s +`String` or a failed `resolve`'s error reason can leak without a handle. Public API: none. Wire: none. diff --git a/quest/m1/test-flakes-2/macos-receive-credit.md b/quest/m1/test-flakes-2/macos-receive-credit.md index 555ce5ed8d2..a869b100d7d 100644 --- a/quest/m1/test-flakes-2/macos-receive-credit.md +++ b/quest/m1/test-flakes-2/macos-receive-credit.md @@ -3,12 +3,12 @@ ## Goal `receive_credit::partial_control_requests_give_way` -(`rs/moq-tokio/tests/integration/receive_credit.rs:502`) passes on Nightly's +(`rs/moq-tokio/tests/integration/receive_credit.rs:537`) passes on Nightly's macOS lane. ## Plan -It panics "stop timed out" on macOS (real sockets, 10 s timeout) while all +It panics "stop timed out" (`:502`) on macOS (real sockets, 10 s timeout) while all nine `receive_credit` tests pass on Linux. macOS failed a different test on each of 10-22, 10-23, and 10-24 (10-23: `quic::driver::tests::stuck_window_evicts_the_lower_priority`), so first diff --git a/quest/m1/test-flakes-2/rtsp-stall-timers.md b/quest/m1/test-flakes-2/rtsp-stall-timers.md index e64f05da276..5454a90095c 100644 --- a/quest/m1/test-flakes-2/rtsp-stall-timers.md +++ b/quest/m1/test-flakes-2/rtsp-stall-timers.md @@ -11,7 +11,7 @@ tests (`:979`, `:1096`) never fail under load. They set 200 ms timeouts against a real-socket fake server on wall time. `client.timeout` also bounds every RTSP request step (`within()` at `:266`), so a loaded runner can stall during setup instead of at the point under -test. Drive the timeouts with a `moq_clock::Manual` clock, or move the fake -server off real sockets. +test. Drive the timeouts with a `moq_clock::Manual` clock; moving the fake +server off real sockets alone would leave them on wall time. Public API: none. Wire: none.