Repository navigation
quest: plan follow-ups from the 10-24 quest-complete run #6691
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
197d85d
quest: plan follow-ups from the 10-24 quest-complete run
kixelated 86e0976
quest: tighten four follow-up plans from review
kixelated eedb9f1
Merge remote-tracking branch 'origin/main' into HEAD
kixelated c125846
Merge remote-tracking branch 'origin/main' into HEAD
kixelated File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # [S] mbx keeps a live build's trees on a full disk | ||
|
|
||
| ## Goal | ||
|
|
||
| A CI job never rebuilds moq-ffi mid-run because mbx garbage collection | ||
| evicted the generated source trees its build is using, and when a Dart build | ||
| hook does rebuild, its error names the cause. | ||
|
|
||
| ## Plan | ||
|
|
||
| Seen three times in about 34 hours (merge-group runs of | ||
| [moq-dev/moq#6449](https://github.com/moq-dev/moq/pull/6449), | ||
| [moq-dev/moq#6256](https://github.com/moq-dev/moq/pull/6256), and | ||
| [moq-dev/moq#6436](https://github.com/moq-dev/moq/pull/6436)): the runner | ||
| disk dropped below mbx's 50 GiB free-space minimum, gc evicted trees the | ||
| job had restored, and the Dart build hook rebuilt moq-ffi. Then | ||
| `sh/dart/check.sh` reported "update hook_vars in sh/dart/generate.sh", which | ||
| sends the reader to the wrong file. | ||
|
|
||
| - Either mbx gc skips trees a live build holds, or the runner keeps enough | ||
| free disk that gc never runs mid-job; pick by measurement. | ||
| - The check tells the two causes apart before naming one: it blames | ||
| `hook_vars` only when they differ from `sh/dart/generate.sh`, and otherwise | ||
| reports that the restored tree was missing. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # [XS] select.sh resolves its launcher before cd | ||
|
|
||
| ## Goal | ||
|
|
||
| `cd sh/rs && bash select.sh lint --all` runs the selected checks instead of | ||
| failing with `bash: ./../local.sh: No such file or directory`. | ||
|
|
||
| ## Plan | ||
|
|
||
| [moq-dev/moq#6256](https://github.com/moq-dev/moq/pull/6256) re-execs | ||
| `sh/rs/select.sh` through `sh/local.sh` after changing to the repository | ||
| root, so a relative `$0` no longer resolves | ||
| ([review thread](https://github.com/moq-dev/moq/pull/6256#discussion_r4366860705)). | ||
| Resolve both paths before the `cd`, or name them relative to the root. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # Nightly passes on main | ||
|
|
||
| ## Goal | ||
|
|
||
| The scheduled Nightly on `main` passes end to end, so a red run means a new | ||
| break instead of noise. Nightly is the backstop for what the PR gate skips on | ||
| purpose (benches, examples, feature shapes; see | ||
| [moq-dev/moq#5531](https://github.com/moq-dev/moq/pull/5531)), and it has | ||
| failed every night since at least 2026-10-19, which let a bench and two | ||
| example breaks reach `main` unnoticed. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-10-24 (maintainer), from a triage of run | ||
| [40090959108](https://github.com/moq-dev/moq/actions/runs/40090959108): one | ||
| child per distinct cause, each landing on its own. The PR gate keeps | ||
| skipping benches and examples; a green Nightly catches them within a day. | ||
|
|
||
| This README's own work, after the children: | ||
|
|
||
| - Nightly's multi-step jobs (Features first) keep going past a failed step | ||
| and fail at the end, so one break never hides the steps after it. | ||
| - Confirm a scheduled run passes. Failures already owned elsewhere stay | ||
| there: the audio near-zero p95 row belongs to | ||
| [audio jitter target](/quest/m1/audio-jitter-target/README.md), the | ||
| macOS `receive_credit` timeout to | ||
| [test flakes](/quest/m1/test-flakes-2/README.md), and the paired bench | ||
| threshold to [bench CI](/quest/m1/bench-ci.md). The `cli-export` | ||
| bufferbloat failure is expected to clear with | ||
| [moq-dev/moq#6656](https://github.com/moq-dev/moq/pull/6656). | ||
|
|
||
| Public API: none. Wire: none. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Claimed fetches wait for credit](/quest/m0/nightly/drill-credit.md) - the impaired burst drills stop losing groups to "no stream credit" | ||
| - [Feature shapes in the gate](/quest/m0/nightly/feature-shapes.md) - feature-gated dead code fails the PR that adds it, not the next Nightly | ||
| - [qmux fallback closes cleanly](/quest/m0/nightly/qmux-fallback-close.md) - the WebSocket fallback startup benchmark stops failing "not cleanly closed" | ||
| - [ts_schedule regression](/quest/m0/nightly/ts-schedule.md) - the moq-mux `ts_schedule` benches return to their old cost | ||
| - [Nightly's base in a shallow checkout](/quest/m0/nightly/shallow-base.md) - `sh/gh/nightly.test.py` passes in Nightly's depth-1 checkout | ||
| - [Advisory bumps](/quest/m0/nightly/advisories.md) - both audits pass on today's advisories |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # [XS] Advisory bumps | ||
|
|
||
| ## Goal | ||
|
|
||
| Nightly's Audit and JS audit jobs pass on the advisories open on 2026-10-24. | ||
|
|
||
| ## Plan | ||
|
|
||
| - RUSTSEC-2026-0387: `crypto-bigint` 0.7.5 to at least 0.7.6. | ||
| - GHSA-xjj6-f33q-929c (high), GHSA-4qv8-2p9j-2685, GHSA-f72v-6w45-q472: | ||
| `undici` 7.29.1 to at least 7.30.1, reached through | ||
| `@moq/demo-carla > wrangler > miniflare`. | ||
|
|
||
| New advisories keep arriving, so the audits fail again on their own; the | ||
| standing review belongs to the [dependency audit](/quest/m2/audit-deps.md). | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # [M] Claimed fetches wait for credit | ||
|
|
||
| ## Goal | ||
|
|
||
| `bursts_cross_a_cluster`, `bursts_cross_a_transport_peer`, and | ||
| `bursts_cross_a_flapping_peer` (`::impaired`, `rs/moq-relay/tests/drills.rs`) | ||
| deliver every group again, and drill-sensitivity's | ||
| `ingress-burst-never-collected` baseline passes unmutated. | ||
|
|
||
| ## Plan | ||
|
|
||
| Broken by [moq-dev/moq#6409](https://github.com/moq-dev/moq/pull/6409) | ||
| ("refuse native requests without stream credit"), reproduced on `main`: the | ||
| drills panic at `drills.rs:1305` with groups lost as | ||
| `Failed("no stream credit")`. A relay's claimed FETCH under a burst is now | ||
| refused when the peer's stream credit is exhausted, where it used to wait. | ||
|
|
||
| Find which hop refuses and decide by behavior, not by the test: a request a | ||
| relay makes on a subscriber's behalf should queue for credit (backpressure) | ||
| rather than fail, unless the refusal is the bound #6409 meant to add against | ||
| a hostile peer. Keep that bound; never loosen the drill's assertion. | ||
|
|
||
| Public API: none expected. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| # [S] Feature shapes in the gate | ||
|
|
||
| ## Goal | ||
|
|
||
| Code that is dead under one of a crate's feature shapes fails the PR that | ||
| adds it, instead of breaking Nightly's Features job the next morning. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided 2026-10-24 (maintainer). The Features job ("Check feature and | ||
| profile variants") has failed every night on a different feature-gated item: | ||
| `moq-net/src/model/bandwidth.rs` (10-22), `moq-net/src/transport.rs` (10-23), | ||
| and now `Connection::open_request` | ||
| (`rs/moq-tokio/src/quic/driver/connection.rs:378`), which | ||
| [moq-dev/moq#6409](https://github.com/moq-dev/moq/pull/6409) added without | ||
| `#[cfg(feature = "net")]`; only `quic/web/session.rs` uses it. Reproduce: | ||
| `cargo clippy -p moq-tokio --all-targets --no-default-features --features driver,aws-lc-rs -- -D warnings`. | ||
|
|
||
| - Fix `open_request`'s gating. | ||
| - Add the feature and profile variant check to the PR gate, scoped to the | ||
| crates a branch changes the way `just check` selects them. Measure its | ||
| cost in the PR. Nightly keeps the full matrix. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # [S] qmux fallback closes cleanly | ||
|
|
||
| ## Goal | ||
|
|
||
| Nightly's "WebSocket fallback startup benchmark" (`just js qmux-bench`, in | ||
| the Audio quality job) passes again instead of failing with | ||
| `WebSocketError: WebSocket was not cleanly closed`. | ||
|
|
||
| ## Plan | ||
|
|
||
| New on 2026-10-24 and reproduced locally on `main`; it passed on 10-23. | ||
| Candidates in that window are | ||
| [moq-dev/moq#6409](https://github.com/moq-dev/moq/pull/6409) (qmux refuses | ||
| exhausted credit) and | ||
| [moq-dev/moq#6458](https://github.com/moq-dev/moq/pull/6458) (qmux abort and | ||
| close without the peer's help); the later qmux fixes | ||
| [moq-dev/moq#6644](https://github.com/moq-dev/moq/pull/6644) and | ||
| [moq-dev/moq#6687](https://github.com/moq-dev/moq/pull/6687) do not fix it. | ||
| Bisect, then fix the close at its cause. | ||
|
|
||
| Public API: none. Wire: none expected. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # [XS] Nightly's base in a shallow checkout | ||
|
|
||
| ## Goal | ||
|
|
||
| `test_paired_base_is_the_previous_scheduled_nightly` | ||
| (`sh/gh/nightly.test.py:46`) passes in Nightly's Tests matrix. | ||
|
|
||
| ## Plan | ||
|
|
||
| The test runs `git rev-parse HEAD~1`, but the Tests matrix in `nightly.yml` | ||
| checks out with the default `fetch-depth: 1`, so it exits 128. Added by | ||
| [moq-dev/moq#6511](https://github.com/moq-dev/moq/pull/6511). Make the test | ||
| build its own history instead of reading the checkout's. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| # [S] ts_schedule regression | ||
|
|
||
| ## Goal | ||
|
|
||
| `rust/shared/ts_schedule/{1/1, 1/16, 4/1}` (`rs/moq-mux/benches/ts_schedule.rs`) | ||
| return to their cost before the regression, and Nightly's paired gate on | ||
| the shared bench targets passes. | ||
|
|
||
| ## Plan | ||
|
|
||
| Nightly's first paired run (0513c07369 against 4381af6f0) measured them | ||
| 400 to 460% slower, too large for noise. The window holds | ||
| [moq-dev/moq#6417](https://github.com/moq-dev/moq/pull/6417), | ||
| [moq-dev/moq#6533](https://github.com/moq-dev/moq/pull/6533), and | ||
| [moq-dev/moq#6527](https://github.com/moq-dev/moq/pull/6527). Bisect, then | ||
| fix the cost at its source, or show the bench now measures more work and | ||
| rebaseline with that reason. The smaller `server_admission` (+20%) and | ||
| `uring-session_lite/no-gso` (+11%) deltas belong to the threshold | ||
| calibration in [bench CI](/quest/m1/bench-ci.md). | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # [S] moq-c refuses only the waiting readers | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq_group_request_free` and `moq_group_request_abort` refuse the group to | ||
| the readers waiting on that request, as Rust's `Fill::refuse_groups` does, | ||
| instead of marking it dropped for every later reader. | ||
|
|
||
| ## Plan | ||
|
|
||
| [moq-dev/moq#6171](https://github.com/moq-dev/moq/pull/6171) added | ||
| `Fill::refuse_groups` and moved Rust's fill paths onto it; moq-c's | ||
| `group_request_free` (`rs/moq-c/src/publish.rs:893`) still calls | ||
| `drop_groups`. The C signatures stay; only the behavior changes, so note it | ||
| in the PR's Impact. If the generated C API replaces the hand-written group | ||
| request calls first, delete this quest. | ||
|
|
||
| Public API: moq-c behavior change, no signature change. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| # [S] JS holds a readvertisement until its withdrawal is handled | ||
|
|
||
| ## Goal | ||
|
|
||
| On drafts 14 to 16, a JS moq-transport session handles a PUBLISH_NAMESPACE | ||
| that follows a PUBLISH_NAMESPACE_DONE for the same namespace only after the | ||
| withdrawal's handler has finished, as Rust does. | ||
|
|
||
| ## Plan | ||
|
|
||
| [moq-dev/moq#6530](https://github.com/moq-dev/moq/pull/6530) fixed Rust | ||
| (`Readvertised` in `rs/moq-net/src/ietf/adapter.rs`, test | ||
| `a_readvertisement_waits_for_the_withdrawal_to_be_handled`); without it | ||
| `epochs::ietf14` failed every run. `js/net/src/ietf/adapter.ts` treats | ||
| PUBLISH_NAMESPACE_DONE as a plain CloseStream, so a readvertisement can | ||
| overtake it. Mirror the hold and its test. | ||
|
|
||
| Not planned (2026-10-24, maintainer): the same race per track, where | ||
| PUBLISH_DONE routes as CloseStream and a new PUBLISH as NewRequest. It was | ||
| found by reading the code and never reproduced. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # [S] Dart leak tests measure moq's memory | ||
|
|
||
| ## Goal | ||
|
|
||
| The three RSS checks in `dart/moq_ffi/test/leak_test.dart` (lines 15, 36, | ||
| 66) fail only when moq leaks, never from Dart heap growth or allocator | ||
| retention under load. | ||
|
|
||
| ## Plan | ||
|
|
||
| Each compares `ProcessInfo.currentRss` before and after 2000 calls of 64 KiB | ||
| and asserts growth under 32 MiB, with no forced GC; `dart_test.yaml` | ||
| already runs them with `concurrency: 1`. Measure the live bytes moq's native | ||
| allocator holds, exposed for tests, instead of whole-process RSS. A | ||
| live-handle count is not enough: per-call buffers such as `demand.name()`'s | ||
| `String` or a failed `resolve`'s error reason can leak without a handle. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # [S] The moq-gst pinned-instance test waits on events | ||
|
|
||
| ## Goal | ||
|
|
||
| `a_pinned_instance_winning_its_path_back_does_not_replay_its_finish` | ||
| (`rs/moq-gst/src/source/imp.rs:3960`) never fails under load. | ||
|
|
||
| ## Plan | ||
|
|
||
| It drives a real GStreamer element against a real-time origin and polls | ||
| with `std::thread::sleep(50ms)` loops capped at 200 iterations, racing the | ||
| pipeline threads. Wait on the announce and pad events themselves, and drive | ||
| the origin's timers with a `moq_clock::Manual` clock. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # [S] macOS receive-credit timeout | ||
|
|
||
| ## Goal | ||
|
|
||
| `receive_credit::partial_control_requests_give_way` | ||
| (`rs/moq-tokio/tests/integration/receive_credit.rs:537`) passes on Nightly's | ||
| macOS lane. | ||
|
|
||
| ## Plan | ||
|
|
||
| It panics "stop timed out" (`:502`) on macOS (real sockets, 10 s timeout) while all | ||
| nine `receive_credit` tests pass on Linux. macOS failed a different test on | ||
| each of 10-22, 10-23, and 10-24 (10-23: | ||
| `quic::driver::tests::stuck_window_evicts_the_lower_priority`), so first | ||
| check for a shared platform cause, such as socket buffer sizes or timer | ||
| resolution, before fixing tests one by one. | ||
|
|
||
| Public API: none. Wire: none. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # [XS] The moq-mux missing-sequence test on a mocked clock | ||
|
|
||
| ## Goal | ||
|
|
||
| `missing_sequence_skips_on_live_track` | ||
| (`rs/moq-mux/src/container/consumer.rs:2537`) never fails under load. | ||
|
|
||
| ## Plan | ||
|
|
||
| It wraps the read in a real `tokio::time::timeout(1s)` while the skip waits | ||
| on the consumer's `hole_timer` (`moq_clock::sleep_until`, 100 ms | ||
| `max_delay`) on wall time. Pause the clock the way its sibling | ||
| `gap_keeps_late_arriving_group_within_max_delay` does. | ||
|
|
||
| Public API: none. Wire: none. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.