From 7462c05a2b4708c4c175d132b78c7f0bef0f9026 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 11:17:15 -0700 Subject: [PATCH 1/2] quest: plan connection-blocked writer fixes from the QUIC audit Co-Authored-By: Claude Opus 5.5 --- quest/m1/quic/README.md | 4 ++- quest/m1/quic/connection-blocked.md | 41 +++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) create mode 100644 quest/m1/quic/connection-blocked.md diff --git a/quest/m1/quic/README.md b/quest/m1/quic/README.md index dc3e48e07bb..474050188b0 100644 --- a/quest/m1/quic/README.md +++ b/quest/m1/quic/README.md @@ -8,7 +8,8 @@ MoQ never negotiates. One core serves the tokio backend, the thread-per-core `moq-uring` backend, and qmux; iroh stays on upstream noq. BBR correctness, reliable stream resets (#5690), hierarchical send groups (#5711, #5770), one `Limits` type for relay peers (#5871), and endpoint sharding (#5688) are on -`main`. What remains is bounded incomplete-message memory; per-broadcast +`main`. What remains is bounded incomplete-message memory and clean exits +for connection-blocked writers; per-broadcast fairness gives the crate-private send domains (#5872) their consumer in [Broadcast fairness](/quest/m2/broadcast-fairness.md). qmux already runs on the shared stream state machine, outside this epic. @@ -71,6 +72,7 @@ cache limits remain separate; bitrate pacing composes with this memory cap. ## Required - [Bound message assembly](/quest/m1/quic/receive-memory.md) - complete headers and frames return credit, and receiver priority cancellation restores progress +- [Connection-blocked writers end cleanly](/quest/m1/quic/connection-blocked.md) - an ended stream leaves the blocked-writer list, and STOP_SENDING surfaces under exhausted connection credit ## Related diff --git a/quest/m1/quic/connection-blocked.md b/quest/m1/quic/connection-blocked.md new file mode 100644 index 00000000000..d5b11c44602 --- /dev/null +++ b/quest/m1/quic/connection-blocked.md @@ -0,0 +1,41 @@ +# [S] Connection-blocked writers end cleanly + +## Goal + +In `moq-quic`, a send stream waiting on connection credit leaves the +blocked-writer list when it ends, and a write to a stopped stream returns +`Stopped` even while connection credit is exhausted. The fix is made once in the +shared stream engine, so tokio, io_uring, and qmux all get it. + +Today a peer that grants streams but never MAX_DATA makes the relay keep an +entry for every stream it opens and cancels, with no bound. A writer parked on +`Blocked` also sleeps through a STOP_SENDING that has already arrived, until +unrelated credit shows up or the connection closes. + +## Plan + +Decided in the 2026-10-22 audit plan: one quest for both, because both bugs +sit on the same connection-blocked path in +`rs/moq-quic/src/connection/streams/`. + +- Leak: `Streams::connection_blocked` is a `Vec` that `poll` drains only + once `write_limit() > 0`, and `stream_freed` leaves it alone. Tie + membership to stream lifetime by making it an insertion-ordered set (such as + an `IndexSet`) and removing the entry when a send stream is freed or reset. + Decided over deleting the list and scanning live streams when credit + returns: a removal stays O(1), and the set never holds more entries than + there are live streams. +- Hidden stop: `SendStream::write_source` returns `Blocked` on + `limit == 0` before `Send::write` can look at `stop_reason`. Check for + terminal stream state before connection-level flow control. +- Regression tests go in the existing `state.rs` unit tests. One test + cancels many streams under zero connection credit and asserts that the + blocked set stays empty. The other sends STOP_SENDING on one stream while + another holds the connection window, and asserts that the write returns + `Stopped`. + +Declined from the same audit: qmux's `closed()` resolving once the FIN is +serialized, before the batch reaches the transport. qmux has no ACKs, so `Ok` +only ever meant "handed off locally". A FIN already in the kernel's +TCP/WebSocket buffer is lost the same way if the transport dies, and record +order still keeps finish, closed, close correct. From 96cb37fe1005f84f1bdb435164bb20e7afba4202 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 22 Oct 2026 11:23:19 -0700 Subject: [PATCH 2/2] quest: the leak test must block each stream before freeing it Co-Authored-By: Claude Opus 5.5 --- quest/m1/quic/connection-blocked.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/quest/m1/quic/connection-blocked.md b/quest/m1/quic/connection-blocked.md index d5b11c44602..89ef0f709dd 100644 --- a/quest/m1/quic/connection-blocked.md +++ b/quest/m1/quic/connection-blocked.md @@ -29,8 +29,9 @@ sit on the same connection-blocked path in `limit == 0` before `Send::write` can look at `stop_reason`. Check for terminal stream state before connection-level flow control. - Regression tests go in the existing `state.rs` unit tests. One test - cancels many streams under zero connection credit and asserts that the - blocked set stays empty. The other sends STOP_SENDING on one stream while + writes to many streams under zero connection credit, each write returning + `Blocked`, then resets and frees those streams and asserts that the blocked + set is empty. The other sends STOP_SENDING on one stream while another holds the connection window, and asserts that the write returns `Stopped`.