From c9986ded1958606283885884ca120db2c2ca3c79 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 20:52:37 -0700 Subject: [PATCH 1/5] quest(m0): plan the Interop failures on main The cross-client matrix fails with unroutable whenever WebSocket wins the connect race: #4180's upgrade re-announces every route under a new generation, and #5294 retires the epochless handles resolved before it. Released wire-compat aborts on planned breaks the last release made stale, and its released JS lanes run a client that uses a current-only API. Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 8 ++ quest/m0/upgrade-unroutable.md | 105 +++++++++++++++++++++++++++ quest/m0/wire-compat-green.md | 46 ++++++++++++ quest/m1/transport-upgrade/README.md | 1 + 4 files changed, 160 insertions(+) create mode 100644 quest/m0/upgrade-unroutable.md create mode 100644 quest/m0/wire-compat-green.md diff --git a/quest/m0/README.md b/quest/m0/README.md index 0747a5de22c..626b5c9a192 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -55,12 +55,20 @@ Audio playout: the jitter target is default in both languages (#4162); the 2026-10-08 audit, since only a manual browser proof and a native trace replay remain and no release waits on them. +Interop on main (2026-10-13): the cross-client matrix fails with `unroutable` +whenever WebSocket wins the connect race, since #5294 retired epochless +handles and #4180's upgrade re-announces every route. The released +wire-compat step aborts on planned breaks the last release made stale. +Both rank with TS duration fidelity: a red Interop hides real regressions. + ## Required - [Self-hosted CI](/quest/m0/self-hosted-ci.md) - same-repo Check and Test run on a self-hosted NixOS runner with a main-written local cache, behind a `CI_RUNNER` kill switch - [CI host](/quest/m0/ci-host.md) - the maintainer brings up the spare desktop as the `moq-ci` and `moq-gpu` runner host - [Draft-22 media on 0.17](/quest/m0/release-22/README.md) - a 0.17.x with the LOCATION_FILTER and FIRST_OBJECT fixes and moq-noq 1.3.5, before Seattle, then draft-20+ FETCH - [TS duration fidelity](/quest/m0/ts-duration-fidelity.md) - Interop's TS compliance captures the whole round-tripped stream again, fixing a regression that turns Interop red on main +- [Upgrade keeps broadcasts](/quest/m0/upgrade-unroutable.md) - a WebSocket-to-QUIC upgrade no longer retires the client's epochless handles, which fails Interop's cross-client rounds with `unroutable` +- [Wire compat on main](/quest/m0/wire-compat-green.md) - the nightly released-wire step runs again: planned breaks re-measured against the new releases, and a released-API client for the released JS lanes - [web-transport releases the qmux fixes](/quest/m0/qmux-credit-upstream.md) - waiting on moq-dev/web-transport#412 and #413 to merge and ship, which qmux credit bumps to - [qmux credit](/quest/m0/qmux-credit.md) - qmux returns connection credit for dropped and stopped streams and delivers its close frame, on both lines - [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - every first-party publisher that can restart mints a fresh route epoch, the newest wins a path, and only routes with the same epoch resume a subscription diff --git a/quest/m0/upgrade-unroutable.md b/quest/m0/upgrade-unroutable.md new file mode 100644 index 00000000000..74b0008b2ab --- /dev/null +++ b/quest/m0/upgrade-unroutable.md @@ -0,0 +1,105 @@ +# [M] A WebSocket-to-QUIC upgrade keeps a client's broadcasts usable + +## Goal + +A client whose session comes up over the WebSocket fallback and then +upgrades to QUIC keeps using the broadcasts it already resolved: a new track, +TRACK_INFO, subscribe, or FETCH through a handle taken before the upgrade +succeeds, on default (epochless, lite-06) sessions as well as lite-07. Interop's +full matrix on `main` stops failing with `unroutable`, and its idle-out +failures go away with it. + +Non-goals: stitching two different publisher instances. A real restart +(a new epoch, or an epochless route replaced by a different publisher) still +retires old handles, as #5294 decided. + +## Plan + +Seen 2026-10-12 and 2026-10-13: Interop fails on the nightly schedule run of +`main` (run 38284317921) and on every PR that touches `test/interop` +(#5381's run 38337824317, among others). Only the cross-client rounds fail; +every `tail` case passes. The same cells pass on one run and fail on the next: + +- Rust (`moq export fmp4`), Go, and C++ subscribers exit with `unroutable` + (`catalog consumer: unroutable`, `container consumer: unroutable`), from + any publisher. +- The C subscriber reports `session status: 1`, then `2`, then `timed out + waiting for data`. +- The round then fails with `relay connection N (subscriber) idled out + instead of closing`: a client that failed exited without closing its + session. + +Cause, from the logs. In the failing js -> rust cell, the Rust subscriber +logs `connected version=moq-lite-07-wip transport=websocket`, then +`upgraded from WebSocket ... transport=webtransport` 30 ms later, then +`track info failed ... track=catalog.json err=unroutable`. On the CI runner +the debug-build QUIC dial (it fetches the certificate fingerprint over HTTP +first) often loses the race to the WebSocket head start, so the +upgrade from #4180 (2026-10-06) runs right after connect. Since #5294 +(f0fcbefd6, merged 2026-10-11 19:23 UTC), an epochless route's requests +carry a `RequestValidity` (`rs/moq-net/src/model/broadcast.rs`). It reports +`Unroutable` once the best route from the same `via` is a different +`Instance`, and an epochless instance is the route entry's generation +(`RouteEntry::instance` in `model/origin.rs`). The upgraded session +re-announces the same paths from the same peer under new generations. Every +handle resolved before the upgrade is then treated as retired, and its next +request fails. Epochless is the default: lite-06 carries no epoch, and the +browser and binding publishers don't set one. + +Reproduced while planning, on `main` at 079d7d8d4. Add an `audio` track +to `websocket_upgrades_to_quic` and subscribe to it through `cam` after the +upgrade: the subscribe ends with `Unroutable`. With +`RequestValidity::check` forced to pass, the same subscribe succeeds. The +local `just test interop` matrix passes, because QUIC wins there. + +Onset matches. The first Interop run with `unroutable` started at 20:09 UTC on +2026-10-11. The last red runs before it failed for other reasons (`media`, +`ts`). Runs before #5294 have no `unroutable`. It still passes sometimes, +whenever QUIC wins the race. + +Interop's PR trigger only fires on harness paths, so the nightly and +harness-only PRs like #5381 are the only places this shows up. Users hit it +too: `moq export` on a default connection exits whenever WebSocket wins +briefly. + +Open decision (maintainer): what should a self-initiated upgrade mean for +epochless routes? The +[transport upgrade](/quest/m1/transport-upgrade/README.md) plan says +epochless routes "signal Restart and need a fresh follower resolution". Read +literally, every default client must re-resolve after any upgrade. + +1. (recommended) The upgrade is a transport swap, not a new instance. The + upgraded session's re-announcement of a path continues the old session's + route identity (an update in place, the way `Announced::attached` re-prices), + unless the peer itself sends a `Restart` or End+Start. Content still never + crosses a real restart: if the relay's own route changed, it announces + that on both sessions. This is the smallest change and keeps the upgrade + invisible, which is why it exists. +2. Upgrades are restarts, and every first-party consumer (`moq export`, + `moq subscribe`, moq-ffi/moq-c consumers, the interop clients) follows + `Restart` and re-requests after `Unroutable`. Consistent with the letter + of the epoch line, but every application pays for a transport detail. +3. Don't upgrade sessions whose routes are epochless. Not decidable per + session (epochs are per route), and default clients would stay on + WebSocket. + +Whichever is chosen, land the reproduction above as the regression, on an +epochless route and on an epoch route. Check whether the +relay hop has the same problem when a publisher upgrades: the relay's lite +subscriber builds the same validity for the publisher's session. Then run +`just test interop --all` until it passes (the race can't be forced locally; +a fast machine usually lets QUIC win, so the unit test carries the proof). + +Afterwards, if a failing client still leaves its connection to idle out, +fix that client's error path to close its session (the C client's `fail()` +and the C++ client on a consumer error both look suspect), as a separate +commit. + +Public API: none expected for option 1. Wire: none. + +## Related + +- [Transport upgrade](/quest/m1/transport-upgrade/README.md) - its "epochless routes signal Restart" decision is the one this quest revisits +- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - #5294's retired-request rule, which must keep holding for real restarts +- [#5294](https://github.com/moq-dev/moq/pull/5294) - refuse new requests from retired sources, the commit that exposed this +- [#4180](https://github.com/moq-dev/moq/pull/4180) - the WebSocket-to-QUIC upgrade diff --git a/quest/m0/wire-compat-green.md b/quest/m0/wire-compat-green.md new file mode 100644 index 00000000000..8ece45a47f7 --- /dev/null +++ b/quest/m0/wire-compat-green.md @@ -0,0 +1,46 @@ +# [S] Released wire compatibility runs again on main + +## Goal + +Interop's `Released wire compatibility` step (`just test wire-compat`) runs +its whole matrix against the newest releases on `main`'s nightly. Every cell +passes or is skipped by a planned break the maintainer re-accepted against +those exact releases. + +## Plan + +Two independent causes, both on the scheduled Interop run of 2026-10-12 +(38284317921) and earlier: + +- Stale planned breaks. A release went out with moq-cli 0.14.3, moq-relay + 0.17.3, hang 0.21.12, and @moq/net 0.4.3. Every entry in + `test/interop/compat/planned-breaks.json` pins the previous releases, so + `resolve.ts` aborts with `stale planned break: moq-lite-07-wip, moq-cli + changed from 0.14.2 to 0.14.3` before any cell runs. That is the tripwire + working as designed. Re-measure each entry against the new releases: delete + an entry whose break is gone (the `catalog-archive` reshape and the released + relay's FETCH hang may have shipped fixed), and refresh the pinned versions + and reason of one that persists. Get the maintainer's acceptance in the PR, + as the existing reasons record. +- The released JS lanes run this checkout's client. `compat.sh` copies + `test/interop/clients/js-native/subscribe.ts` into the released `@moq/*` + environment. #5099 (2026-10-10) changed that script to call + `catalog.latest()`, which released `@moq/hang` 0.5.2 lacks. Every + `publisher=released fetch` cell has failed with `catalog.latest is not a + function` since the 2026-10-11 nightly. The released environment needs a + client that compiles against the released API. Recommended: give the compat + lanes their own observer in `test/interop/compat/client.ts`, which is + already the client meant for both environments. Stop sharing the interop + client, whose API follows the checkout. The alternative is to stage the + client from the released package's git tag, but that ties the compat run + to old harness flags. + +Run `just test wire-compat` locally (it needs `gh` registry access) before +and after. Don't dispatch the nightly on a branch. + +Public API: none. Wire: none. + +## Related + +- [#5099](https://github.com/moq-dev/moq/pull/5099) - the API change the shared client picked up +- [#4728](https://github.com/moq-dev/moq/pull/4728) - the released wire-compat harness and its planned-break tripwire diff --git a/quest/m1/transport-upgrade/README.md b/quest/m1/transport-upgrade/README.md index e7a6be3f9eb..d785e53f1c7 100644 --- a/quest/m1/transport-upgrade/README.md +++ b/quest/m1/transport-upgrade/README.md @@ -82,3 +82,4 @@ and no cached groups splice across instances. Keep JS's matrix consistent. ## Related - [Closed fallback](https://github.com/moq-dev/moq/pull/5284) - a WebSocket session that closes right after connecting falls back to the pending QUIC dial instead of redialing +- [Upgrade keeps broadcasts](/quest/m0/upgrade-unroutable.md) - revisits whether an upgrade is a Restart for epochless routes, after it broke Interop on main From c2a67d132b58a06a703dfb82590a932eaa320cac Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 21:01:33 -0700 Subject: [PATCH 2/5] quest(m0): record the maintainer's Interop decisions An epochless upgrade is an explicit Restart that first-party consumers follow, never a retry after Unroutable. Planning found the upgrade retires old handles without announcing that Restart. The released JS compat lanes get their own observer, and both quests stay in m0. Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 5 +- quest/m0/upgrade-unroutable.md | 143 ++++++++++++++++----------- quest/m0/wire-compat-green.md | 13 ++- quest/m1/transport-upgrade/README.md | 2 +- 4 files changed, 94 insertions(+), 69 deletions(-) diff --git a/quest/m0/README.md b/quest/m0/README.md index 626b5c9a192..3825cad7f44 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -57,7 +57,8 @@ remain and no release waits on them. Interop on main (2026-10-13): the cross-client matrix fails with `unroutable` whenever WebSocket wins the connect race, since #5294 retired epochless -handles and #4180's upgrade re-announces every route. The released +handles and #4180's upgrade re-announces every route without a `Restart`. +Both quests stay in m0 (maintainer, 2026-10-13). The released wire-compat step aborts on planned breaks the last release made stale. Both rank with TS duration fidelity: a red Interop hides real regressions. @@ -67,7 +68,7 @@ Both rank with TS duration fidelity: a red Interop hides real regressions. - [CI host](/quest/m0/ci-host.md) - the maintainer brings up the spare desktop as the `moq-ci` and `moq-gpu` runner host - [Draft-22 media on 0.17](/quest/m0/release-22/README.md) - a 0.17.x with the LOCATION_FILTER and FIRST_OBJECT fixes and moq-noq 1.3.5, before Seattle, then draft-20+ FETCH - [TS duration fidelity](/quest/m0/ts-duration-fidelity.md) - Interop's TS compliance captures the whole round-tripped stream again, fixing a regression that turns Interop red on main -- [Upgrade keeps broadcasts](/quest/m0/upgrade-unroutable.md) - a WebSocket-to-QUIC upgrade no longer retires the client's epochless handles, which fails Interop's cross-client rounds with `unroutable` +- [Upgrade restarts epochless broadcasts](/quest/m0/upgrade-unroutable.md) - a WebSocket-to-QUIC upgrade announces `Restart` for the epochless handles it retires and first-party consumers follow it, fixing Interop's `unroutable` rounds - [Wire compat on main](/quest/m0/wire-compat-green.md) - the nightly released-wire step runs again: planned breaks re-measured against the new releases, and a released-API client for the released JS lanes - [web-transport releases the qmux fixes](/quest/m0/qmux-credit-upstream.md) - waiting on moq-dev/web-transport#412 and #413 to merge and ship, which qmux credit bumps to - [qmux credit](/quest/m0/qmux-credit.md) - qmux returns connection credit for dropped and stopped streams and delivers its close frame, on both lines diff --git a/quest/m0/upgrade-unroutable.md b/quest/m0/upgrade-unroutable.md index 74b0008b2ab..6797512bbcb 100644 --- a/quest/m0/upgrade-unroutable.md +++ b/quest/m0/upgrade-unroutable.md @@ -1,17 +1,16 @@ -# [M] A WebSocket-to-QUIC upgrade keeps a client's broadcasts usable +# [L] A WebSocket-to-QUIC upgrade restarts epochless broadcasts explicitly ## Goal -A client whose session comes up over the WebSocket fallback and then -upgrades to QUIC keeps using the broadcasts it already resolved: a new track, -TRACK_INFO, subscribe, or FETCH through a handle taken before the upgrade -succeeds, on default (epochless, lite-06) sessions as well as lite-07. Interop's -full matrix on `main` stops failing with `unroutable`, and its idle-out -failures go away with it. +When a client's session upgrades from the WebSocket fallback to QUIC, every +epochless broadcast it had resolved gets an explicit `Restart` announce +event, and every first-party consumer follows it by resolving the path +again. Interop's full matrix on `main` stops failing with `unroutable`, +and its idle-out failures go away with it. -Non-goals: stitching two different publisher instances. A real restart -(a new epoch, or an epochless route replaced by a different publisher) still -retires old handles, as #5294 decided. +Non-goals: carrying an epochless route's identity across the upgrade, and +treating `Unroutable` as a cue to retry. Epoch routes are unchanged: a +matching epoch already continues across the upgrade. ## Plan @@ -46,60 +45,86 @@ handle resolved before the upgrade is then treated as retired, and its next request fails. Epochless is the default: lite-06 carries no epoch, and the browser and binding publishers don't set one. -Reproduced while planning, on `main` at 079d7d8d4. Add an `audio` track -to `websocket_upgrades_to_quic` and subscribe to it through `cam` after the -upgrade: the subscribe ends with `Unroutable`. With -`RequestValidity::check` forced to pass, the same subscribe succeeds. The -local `just test interop` matrix passes, because QUIC wins there. - Onset matches. The first Interop run with `unroutable` started at 20:09 UTC on 2026-10-11. The last red runs before it failed for other reasons (`media`, `ts`). Runs before #5294 have no `unroutable`. It still passes sometimes, -whenever QUIC wins the race. - -Interop's PR trigger only fires on harness paths, so the nightly and -harness-only PRs like #5381 are the only places this shows up. Users hit it -too: `moq export` on a default connection exits whenever WebSocket wins -briefly. - -Open decision (maintainer): what should a self-initiated upgrade mean for -epochless routes? The -[transport upgrade](/quest/m1/transport-upgrade/README.md) plan says -epochless routes "signal Restart and need a fresh follower resolution". Read -literally, every default client must re-resolve after any upgrade. - -1. (recommended) The upgrade is a transport swap, not a new instance. The - upgraded session's re-announcement of a path continues the old session's - route identity (an update in place, the way `Announced::attached` re-prices), - unless the peer itself sends a `Restart` or End+Start. Content still never - crosses a real restart: if the relay's own route changed, it announces - that on both sessions. This is the smallest change and keeps the upgrade - invisible, which is why it exists. -2. Upgrades are restarts, and every first-party consumer (`moq export`, - `moq subscribe`, moq-ffi/moq-c consumers, the interop clients) follows - `Restart` and re-requests after `Unroutable`. Consistent with the letter - of the epoch line, but every application pays for a transport detail. -3. Don't upgrade sessions whose routes are epochless. Not decidable per - session (epochs are per route), and default clients would stay on - WebSocket. - -Whichever is chosen, land the reproduction above as the regression, on an -epochless route and on an epoch route. Check whether the -relay hop has the same problem when a publisher upgrades: the relay's lite -subscriber builds the same validity for the publisher's session. Then run -`just test interop --all` until it passes (the race can't be forced locally; -a fast machine usually lets QUIC win, so the unit test carries the proof). - -Afterwards, if a failing client still leaves its connection to idle out, -fix that client's error path to close its session (the C client's `fail()` -and the C++ client on a consumer error both look suspect), as a separate -commit. - -Public API: none expected for option 1. Wire: none. +whenever QUIC wins the race. Users hit it too: `moq export` on a default +connection exits whenever WebSocket wins briefly. + +Reproduced while planning, on `main` at 079d7d8d4, by extending +`websocket_upgrades_to_quic` (`rs/moq-tokio/src/connection.rs`) with an +epochless `audio` track that nothing subscribes until after the upgrade: + +- A subscribe through the pre-upgrade `cam` handle ends with `Unroutable`. + With `RequestValidity::check` forced to pass, it succeeds. +- An announce consumer opened on the subscriber origin before the upgrade + sees only the initial `Start`. No `Restart` arrives after the upgrade, + even though the old handle is already retired. +- A fresh `routed_broadcast("cam")` after the upgrade subscribes `audio` + fine. + +So retirement works as #5294 intends. What's missing is the signal: the +handles retire silently, and nothing tells a holder to follow. + +Decided by the maintainer 2026-10-13 on #5388: an upgrade on an epochless +route is a Restart. "No re-requests after Unroutable please, explicitly +Restart only." This matches the +[transport upgrade](/quest/m1/transport-upgrade/README.md) plan ("epochless +routes instead signal Restart and need a fresh follower resolution"). The +other options, carrying the route identity across the upgrade and not +upgrading epochless sessions, were rejected. + +1. The upgrade signals Restart. Whenever the upgraded session's route + retires a handle resolved through the old session's epochless route, the + origin delivers `AnnounceEvent::Restart` for that prefix to every announce + consumer, before or atomically with the retirement. A holder must never + see `Unroutable` from a retired handle without a `Restart` it could have + followed. Find why the existing epochless-replacement Restart (the + `PendingUpdate::Restart` path in `model/origin.rs`) does not fire here. + The likely reason is that the replacement comes from the same `via`, so + it reads as an update in place. Fix it in the origin, not in moq-tokio, so + a relay hop gets the same signal when a publisher upgrades. The relay's + lite subscriber builds the same validity for a publisher's session; check + that hop too. Subscriptions already open stay sticky on the old session + until it drains, as today. +2. First-party consumers follow it. Each holds the announce stream for the + path it resolved, and on `Restart` drops its handles and resolves afresh. + On `Unroutable` it fails; it never retries a request. Audit and fix: + - Rust: `moq export` and `moq subscribe` (the interop subscriber), plus + any other moq-cli command that resolves a broadcast and keeps reading. + `moq play` already follows `Restart` (see + [Broadcast epochs](/quest/m0/broadcast-epoch/README.md)); confirm it. + - Bindings: moq-ffi's catalog and container consumers, as Go, C++, and + Python reach them, and moq-c's (`moq_origin_announced_broadcast`, + `moq_consume_catalog`). Prefer following once in moq-ffi and moq-c over + duplicating it in every language wrapper. If following needs new API, + report it per the Cross-Package Sync table. + - JS: `js/net` doesn't upgrade yet (the + [JavaScript half](/quest/m1/transport-upgrade/js.md) of the upgrade + line), but a Rust relay's Restart reaches JS consumers. Confirm + `@moq/watch` and the js-native interop subscriber follow it. + - The interop clients under `test/interop/clients`, where they resolve a + broadcast themselves. +3. Regression: land the reproduction above in `websocket_upgrades_to_quic` + or a sibling. It asserts that the announce consumer sees `Restart` for + `cam`, that following it (a fresh `routed_broadcast`) serves the new + track, and that the old handle still ends with `Unroutable` and is never + re-requested. Add the same case on an epoch route, where the old handle + keeps working and no `Restart` is sent. Then run `just test interop --all`. + The race can't be forced locally (a fast machine lets QUIC win), so the + unit test carries the proof. + +If a failing client still leaves its connection to idle out, fix that +client's error path to close its session, as a separate commit. The C +client's `fail()` and the C++ client on a consumer error both look suspect. + +Public API: an announce `Restart` on upgrade is new observable behavior. +Consumer-follow API in moq-ffi and moq-c only if the audit needs it; report +it in the PR. Wire: none. ## Related -- [Transport upgrade](/quest/m1/transport-upgrade/README.md) - its "epochless routes signal Restart" decision is the one this quest revisits -- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - #5294's retired-request rule, which must keep holding for real restarts +- [Transport upgrade](/quest/m1/transport-upgrade/README.md) - its "epochless routes signal Restart" decision, which this quest implements for the Rust upgrade +- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - #5294's retired-request rule and the players that already follow `Restart` - [#5294](https://github.com/moq-dev/moq/pull/5294) - refuse new requests from retired sources, the commit that exposed this - [#4180](https://github.com/moq-dev/moq/pull/4180) - the WebSocket-to-QUIC upgrade diff --git a/quest/m0/wire-compat-green.md b/quest/m0/wire-compat-green.md index 8ece45a47f7..84352a7da76 100644 --- a/quest/m0/wire-compat-green.md +++ b/quest/m0/wire-compat-green.md @@ -27,13 +27,12 @@ Two independent causes, both on the scheduled Interop run of 2026-10-12 environment. #5099 (2026-10-10) changed that script to call `catalog.latest()`, which released `@moq/hang` 0.5.2 lacks. Every `publisher=released fetch` cell has failed with `catalog.latest is not a - function` since the 2026-10-11 nightly. The released environment needs a - client that compiles against the released API. Recommended: give the compat - lanes their own observer in `test/interop/compat/client.ts`, which is - already the client meant for both environments. Stop sharing the interop - client, whose API follows the checkout. The alternative is to stage the - client from the released package's git tag, but that ties the compat run - to old harness flags. + function` since the 2026-10-11 nightly. Decided by the maintainer + 2026-10-13 on #5388: the compat lanes run their own observer in + `test/interop/compat/client.ts`, using only API the released packages + have, and stop sharing the interop client, whose API follows the checkout. + Staging the client from the released package's tag was rejected, since it + ties the compat run to old harness flags. Run `just test wire-compat` locally (it needs `gh` registry access) before and after. Don't dispatch the nightly on a branch. diff --git a/quest/m1/transport-upgrade/README.md b/quest/m1/transport-upgrade/README.md index d785e53f1c7..0d5fc8bc213 100644 --- a/quest/m1/transport-upgrade/README.md +++ b/quest/m1/transport-upgrade/README.md @@ -82,4 +82,4 @@ and no cached groups splice across instances. Keep JS's matrix consistent. ## Related - [Closed fallback](https://github.com/moq-dev/moq/pull/5284) - a WebSocket session that closes right after connecting falls back to the pending QUIC dial instead of redialing -- [Upgrade keeps broadcasts](/quest/m0/upgrade-unroutable.md) - revisits whether an upgrade is a Restart for epochless routes, after it broke Interop on main +- [Upgrade restarts epochless broadcasts](/quest/m0/upgrade-unroutable.md) - the Rust upgrade's missing epochless `Restart`, and the consumers that follow it From a4fce52a2bc2ed39904fe6ab73adff9fbe27b0eb Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 21:02:29 -0700 Subject: [PATCH 3/5] quest(m0): name @moq/json as the package that lacks latest() Co-Authored-By: Claude Opus 5.5 --- quest/m0/wire-compat-green.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/quest/m0/wire-compat-green.md b/quest/m0/wire-compat-green.md index 84352a7da76..5e1f28cc47e 100644 --- a/quest/m0/wire-compat-green.md +++ b/quest/m0/wire-compat-green.md @@ -25,7 +25,8 @@ Two independent causes, both on the scheduled Interop run of 2026-10-12 - The released JS lanes run this checkout's client. `compat.sh` copies `test/interop/clients/js-native/subscribe.ts` into the released `@moq/*` environment. #5099 (2026-10-10) changed that script to call - `catalog.latest()`, which released `@moq/hang` 0.5.2 lacks. Every + `catalog.latest()` on a `Json.Snapshot.Consumer`, a method released + `@moq/json` 0.4.2 lacks (`js/json/src/snapshot/consumer.ts`). Every `publisher=released fetch` cell has failed with `catalog.latest is not a function` since the 2026-10-11 nightly. Decided by the maintainer 2026-10-13 on #5388: the compat lanes run their own observer in From fafcf8e9f447498fe859acd3e3f8bef7bb778465 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 21:07:59 -0700 Subject: [PATCH 4/5] quest(m0): the upgrade's Restart is held, not missing An epochless Restart waits out the origin's update_hold while the old handles retire at once, so a request in that window gets Unroutable before any Restart could be followed. Align retirement with the Restart instead of changing same-via handling. Co-Authored-By: Claude Opus 5.5 --- quest/m0/README.md | 12 +++---- quest/m0/upgrade-unroutable.md | 51 ++++++++++++++++------------ quest/m1/transport-upgrade/README.md | 2 +- 3 files changed, 37 insertions(+), 28 deletions(-) diff --git a/quest/m0/README.md b/quest/m0/README.md index 3825cad7f44..a1453262e52 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -56,11 +56,11 @@ Audio playout: the jitter target is default in both languages (#4162); the remain and no release waits on them. Interop on main (2026-10-13): the cross-client matrix fails with `unroutable` -whenever WebSocket wins the connect race, since #5294 retired epochless -handles and #4180's upgrade re-announces every route without a `Restart`. -Both quests stay in m0 (maintainer, 2026-10-13). The released -wire-compat step aborts on planned breaks the last release made stale. -Both rank with TS duration fidelity: a red Interop hides real regressions. +whenever WebSocket wins the connect race: since #5294, #4180's upgrade +retires epochless handles before its held `Restart` reaches anyone. The +released wire-compat step aborts on planned breaks the last release made +stale. Both stay in m0 (maintainer, 2026-10-13) and rank with TS duration +fidelity: a red Interop hides real regressions. ## Required @@ -68,7 +68,7 @@ Both rank with TS duration fidelity: a red Interop hides real regressions. - [CI host](/quest/m0/ci-host.md) - the maintainer brings up the spare desktop as the `moq-ci` and `moq-gpu` runner host - [Draft-22 media on 0.17](/quest/m0/release-22/README.md) - a 0.17.x with the LOCATION_FILTER and FIRST_OBJECT fixes and moq-noq 1.3.5, before Seattle, then draft-20+ FETCH - [TS duration fidelity](/quest/m0/ts-duration-fidelity.md) - Interop's TS compliance captures the whole round-tripped stream again, fixing a regression that turns Interop red on main -- [Upgrade restarts epochless broadcasts](/quest/m0/upgrade-unroutable.md) - a WebSocket-to-QUIC upgrade announces `Restart` for the epochless handles it retires and first-party consumers follow it, fixing Interop's `unroutable` rounds +- [Upgrade restarts epochless broadcasts](/quest/m0/upgrade-unroutable.md) - a WebSocket-to-QUIC upgrade retires epochless handles only once their `Restart` is delivered, and first-party consumers follow it, fixing Interop's `unroutable` rounds - [Wire compat on main](/quest/m0/wire-compat-green.md) - the nightly released-wire step runs again: planned breaks re-measured against the new releases, and a released-API client for the released JS lanes - [web-transport releases the qmux fixes](/quest/m0/qmux-credit-upstream.md) - waiting on moq-dev/web-transport#412 and #413 to merge and ship, which qmux credit bumps to - [qmux credit](/quest/m0/qmux-credit.md) - qmux returns connection credit for dropped and stopped streams and delivers its close frame, on both lines diff --git a/quest/m0/upgrade-unroutable.md b/quest/m0/upgrade-unroutable.md index 6797512bbcb..cb60111ec40 100644 --- a/quest/m0/upgrade-unroutable.md +++ b/quest/m0/upgrade-unroutable.md @@ -58,13 +58,19 @@ epochless `audio` track that nothing subscribes until after the upgrade: - A subscribe through the pre-upgrade `cam` handle ends with `Unroutable`. With `RequestValidity::check` forced to pass, it succeeds. - An announce consumer opened on the subscriber origin before the upgrade - sees only the initial `Start`. No `Restart` arrives after the upgrade, - even though the old handle is already retired. + does get `Restart` for `cam`, but only after the origin's `update_hold` + (300 ms by default; `TableCursor::update` holds epochless restarts, and + `identical_reannounce_restarts_without_an_epoch` covers the event). A + first check within about 300 ms saw nothing; a 2 s wait saw the `Restart`. + Codex caught this on #5388's review. - A fresh `routed_broadcast("cam")` after the upgrade subscribes `audio` fine. -So retirement works as #5294 intends. What's missing is the signal: the -handles retire silently, and nothing tells a holder to follow. +So the signal exists, but it arrives late: `RequestValidity` retires the old +handle as soon as the new route wins, while its `Restart` waits out the hold. +A consumer that asks for a track inside that window, which every interop +subscriber does right after connecting, gets `Unroutable` before any +`Restart` could reach it. And no interop consumer follows `Restart` anyway. Decided by the maintainer 2026-10-13 on #5388: an upgrade on an epochless route is a Restart. "No re-requests after Unroutable please, explicitly @@ -74,19 +80,20 @@ routes instead signal Restart and need a fresh follower resolution"). The other options, carrying the route identity across the upgrade and not upgrading epochless sessions, were rejected. -1. The upgrade signals Restart. Whenever the upgraded session's route - retires a handle resolved through the old session's epochless route, the - origin delivers `AnnounceEvent::Restart` for that prefix to every announce - consumer, before or atomically with the retirement. A holder must never - see `Unroutable` from a retired handle without a `Restart` it could have - followed. Find why the existing epochless-replacement Restart (the - `PendingUpdate::Restart` path in `model/origin.rs`) does not fire here. - The likely reason is that the replacement comes from the same `via`, so - it reads as an update in place. Fix it in the origin, not in moq-tokio, so - a relay hop gets the same signal when a publisher upgrades. The relay's - lite subscriber builds the same validity for a publisher's session; check - that hop too. Subscriptions already open stay sticky on the old session - until it drains, as today. +1. Retirement and its Restart agree. A holder must never see `Unroutable` + from a retired handle before the `Restart` it could have followed has been + delivered. The origin already emits the `Restart`; align the two in the + origin, not in moq-tokio, so a relay hop behaves the same when a publisher + upgrades. The relay's lite subscriber builds the same validity for a + publisher's session; check that hop too. Recommended: a handle's + `RequestValidity` retires when its `Restart` is published, so the old + route keeps serving new requests through the hold. The old session is + still draining, so it can. The alternative is to skip the hold for a + replacement that retires handles, but the hold exists to absorb path + hunting after a withdrawal, so check what that would cost first. Don't + change same-`via` handling: an identical epochless re-announcement already + restarts. Subscriptions already open stay sticky on the old session until + it drains, as today. 2. First-party consumers follow it. Each holds the announce stream for the path it resolved, and on `Restart` drops its handles and resolves afresh. On `Unroutable` it fails; it never retries a request. Audit and fix: @@ -106,10 +113,12 @@ upgrading epochless sessions, were rejected. - The interop clients under `test/interop/clients`, where they resolve a broadcast themselves. 3. Regression: land the reproduction above in `websocket_upgrades_to_quic` - or a sibling. It asserts that the announce consumer sees `Restart` for - `cam`, that following it (a fresh `routed_broadcast`) serves the new - track, and that the old handle still ends with `Unroutable` and is never - re-requested. Add the same case on an epoch route, where the old handle + or a sibling, with the clock under control rather than a wall-clock wait + for the hold. It asserts that a request through the old handle made + right after the upgrade, before the `Restart` is delivered, does not fail + with `Unroutable`. It also asserts that the announce consumer then sees + `Restart` for `cam`, and that following it (a fresh `routed_broadcast`) + serves the new track. Add the same case on an epoch route, where the old handle keeps working and no `Restart` is sent. Then run `just test interop --all`. The race can't be forced locally (a fast machine lets QUIC win), so the unit test carries the proof. diff --git a/quest/m1/transport-upgrade/README.md b/quest/m1/transport-upgrade/README.md index 0d5fc8bc213..8865ba953c7 100644 --- a/quest/m1/transport-upgrade/README.md +++ b/quest/m1/transport-upgrade/README.md @@ -82,4 +82,4 @@ and no cached groups splice across instances. Keep JS's matrix consistent. ## Related - [Closed fallback](https://github.com/moq-dev/moq/pull/5284) - a WebSocket session that closes right after connecting falls back to the pending QUIC dial instead of redialing -- [Upgrade restarts epochless broadcasts](/quest/m0/upgrade-unroutable.md) - the Rust upgrade's missing epochless `Restart`, and the consumers that follow it +- [Upgrade restarts epochless broadcasts](/quest/m0/upgrade-unroutable.md) - the Rust upgrade retires epochless handles before their `Restart` arrives, and the consumers that follow it From 37d555d550c823bdea59d7a0437ef49aeb9a806d Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 21:13:01 -0700 Subject: [PATCH 5/5] quest(m0): retire on a deliverable Restart; the event already exists Co-Authored-By: Claude Opus 5.5 --- quest/m0/upgrade-unroutable.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/quest/m0/upgrade-unroutable.md b/quest/m0/upgrade-unroutable.md index cb60111ec40..89a3f2ec830 100644 --- a/quest/m0/upgrade-unroutable.md +++ b/quest/m0/upgrade-unroutable.md @@ -86,8 +86,12 @@ upgrading epochless sessions, were rejected. origin, not in moq-tokio, so a relay hop behaves the same when a publisher upgrades. The relay's lite subscriber builds the same validity for a publisher's session; check that hop too. Recommended: a handle's - `RequestValidity` retires when its `Restart` is published, so the old - route keeps serving new requests through the hold. The old session is + `RequestValidity` retires only once its held `Restart` becomes + deliverable, so the old route keeps serving new requests through the + hold. Queueing is not enough: today `OriginState::sync_route` queues the + `PendingUpdate::Restart` (via `TableCursor::update`) before + `routes.poke_below` invalidates the `Selection`, and the hold is only + applied later in `AnnounceConsumer::poll_next`. The old session is still draining, so it can. The alternative is to skip the hold for a replacement that retires handles, but the hold exists to absorb path hunting after a withdrawal, so check what that would cost first. Don't @@ -127,9 +131,10 @@ If a failing client still leaves its connection to idle out, fix that client's error path to close its session, as a separate commit. The C client's `fail()` and the C++ client on a consumer error both look suspect. -Public API: an announce `Restart` on upgrade is new observable behavior. -Consumer-follow API in moq-ffi and moq-c only if the audit needs it; report -it in the PR. Wire: none. +Public API: the upgrade's `Restart` already exists. What changes is when +an old handle retires relative to it, and that first-party consumers follow +it. Consumer-follow API in moq-ffi and moq-c only if the audit needs it; +report it in the PR. Wire: none. ## Related