Repository navigation
feat(quic)!: schedule datagrams in their send group - #5770
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…batim Unmodified copies, not yet compiled, so a reviewer can diff the adaptation that follows against upstream: - quinn-rs/quinn 7616e6b2782722f3ce4a1b181ef08817d7f545e4 quinn/src -> rs/moq-tokio/src/quic/driver (lib.rs -> mod.rs), the commit moq-quic forked from. - moq-dev/noq e27c0ce00741d81c8620b98e345d4f2c88e0768c web-transport-moq/src -> rs/moq-tokio/src/quic/web (lib.rs -> mod.rs). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop every moq-noq dependency. moq-tokio's QUIC backend now drives moq-quic through quinn's async layer (`quic::driver`) with web-transport-moq folded in as `quic::web`; moq-uring's sans-IO driver moves from moq-noq-proto to moq-quic. Upstream noq stays only behind moq-tokio's `iroh` feature. - The `noq` cargo feature is `quic` everywhere it was forwarded. - `moq_tokio::noq::Error` is `moq_tokio::quic::Error`; `Error::Noq` is `Error::Quic`. - moq-quic takes `moq_clock::Instant` and raises a once-per-connection Draining endpoint event, which backs the driver's `wait_all_draining`. - qlog writes one trace per endpoint (per dial on moq-uring's connect side), since moq-quic holds one sink per transport config. - Session stats report the congestion controller's bandwidth estimate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The capsule reader maps a lost stream to UnexpectedEnd, so every ordinary session close warned "failed to read capsule". A read error while the connection is closed is now debug; one on a live connection still warns. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Relay peak RSS on the fanout workloads matches moq-noq on origin/main, which was the last open part of the Goal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Carry #5638's steering changes onto the renamed moq-quic drivers: the shard CID comes from moq_sock::shard::cid, and moq-uring stops greasing the QUIC bit on sharded endpoints. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Endpoint::close now refuses attempts still queued for accept, and wait_all_draining waits for accepts partway through Incoming::accept, so a server shutdown cannot release the socket before those peers hear back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace quinn's single pending-stream heap with a hierarchical schedule: - Streams outside every group go first, in quinn's order (priority, then round robin), so control and HTTP/3 streams are never queued behind media. - Groups belong to a scheduling domain (one default). Inside a domain the highest backlogged priority is served strictly, and equal priorities share bytes by deficit round robin, charged in wire bytes. Domains share bytes the same way, so opening more groups buys a domain nothing. - A stream's priority widens to i64 and orders it only within its group. Debt carries across a group going idle; unused credit does not, and a blocked stream holds no turn. Connection::send_groups opens, moves and closes groups and domains as owned tokens. moq-tokio's and moq-uring's drivers wrap a group in a shared SendGroup handle that closes on drop, and their streams gain set_group. Retransmits re-queue through the same schedule, so a lost range keeps its group's share. Tests drive the real send path with differently sized streams: byte fairness within a bounded window, preemption, per-group order, priority updates, sparse sequences, blocked streams, retransmits, domains, resets and leaving a group. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The poll transport gains send groups: - Session::send_group(priority) opens one, or returns None on a transport without them. - SendStream::Group names the transport's group (Ungrouped, uninhabited, for flat transports), SendStream::set_group joins one, and SendStream::set_priority widens to i64. - SendGroup::set_priority moves a whole group. The lite and IETF publishers open one group per subscription (and per standalone FETCH; a joining FETCH shares its subscription's) at the subscriber priority. Group streams join it ranked by sequence in the subscription's group order, with a group's first stream ahead of the rest of it. SUBSCRIBE_UPDATE moves the group once instead of reranking every stream, and an order flip re-stamps the open streams. On a transport without groups the session-wide PriorityQueue still ranks every stream, as before. moq-tokio's and moq-uring's adapters reach backend groups through new Grouped and GroupedSend extension traits: native QUIC and WebTransport open real groups; iroh, qmux, ffi and wasm stay flat. The moq-tokio adapter defers a join made while a write or FIN watch holds the stream, like a priority change. A benchmark compares the groups with the PriorityQueue and two scalar packings on congested audio and video workloads, and sweeps subscriptions against in-flight streams. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hierarchical send groups landed in moq-quic and every subscription uses one, so drop the quest and the links that waited on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quest/m1/quic/scheduler
The server socket retried every failed recv, so a persistent error spun the endpoint task while tokio kept readiness asserted. Only WouldBlock retries now, matching the driver's tokio runtime. Also cap moq_quic at info like noq was, fix the driver doc example, and point the moq-uring README at the just recipe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the async publishers name it, so a model-only build flagged it as dead code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…quest/m1/quic/scheduler
Drop GroupedSend::set_order: the adapters use a single set_priority(i64), documented everywhere as the priority within the stream's send group, or session-wide when ungrouped. Grouped and GroupedSend leave the public API. pub(crate) is refused (E0446) because the public adapters' `type Group` projects through GroupedSend, so both are `pub` in a private module and never re-exported: other crates can neither name nor implement them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # rs/moq-quic/README.md # rs/moq-tokio/src/quic/driver/connection.rs # rs/moq-tokio/src/quic/driver/mod.rs # rs/moq-tokio/src/quic/driver/send_stream.rs # rs/moq-tokio/src/quic/web/send.rs # rs/moq-tokio/src/quic/web/session.rs # rs/moq-uring/src/quic/driver/stream.rs # rs/moq-uring/src/quic/mod.rs
# Conflicts: # quest/m1/quic/scheduler.md # quest/m2/quic-deadline.md
Brings in receive credit (#5675). Its ConnectionTag now also tags the send group and domain tokens, so moq-quic refuses a token from another connection as its docs promised, instead of indexing whichever of its own groups shares the slot. The drivers' pointer checks became redundant and are gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…duler # Conflicts: # quest/m3/cut-through/README.md
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moq-quic SendGroups::{open, set_priority, priority}, the tokio and uring
driver handles and send_group, and moq-net's poll::Session::send_group
and poll::SendGroup::set_priority now take i64. Group and stream
priorities share one width, so no layer converts between them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Browser WebTransport from Wasm has no send groups, and Lite datagrams bypass the stream scheduler, so neither gets the byte-fair split. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Datagram priority will move datagrams into the send-group hierarchy; its quest now owns updating the Priority row when that lands. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… quest/m1/quic/datagram-priority
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 25 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d93eb5e9f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A SUBSCRIBE_UPDATE refreshed the steer only when a group was served next, so datagrams before then kept the old order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 33fc34e350
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
write_frames judged emptiness by its own frames, so a top-ranked datagram that did not fit behind an ACK or other control frame stepped aside and let lower-ranked data take the packet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#5770 schedules datagrams in their send group, so ungrouped datagrams now go before equal-priority stream data, and one that does not fit ends the packet instead of letting streams fill it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Quest:
m1/quic/datagram-priority. Follows #5711.Datagrams used to go first in every packet, ahead of every stream. A low-priority datagram flood could starve a high-priority stream. Now each moq-quic send group queues its own datagrams, and they are scheduled with the group's streams.
What changed
order(the same value a stream uses for priority). On a tie, a datagram goes before a stream. Its frame bytes are charged to the group's and the domain's DRR credit. Ungrouped datagrams go in the ungrouped queue, ahead of every group, like ungrouped streams. The datagrams-first loop inpopulate_packetis gone;write_frameswrites streams and datagrams in one pass.datagram_send_buffer_sizeis one connection-wide budget counted in frame bytes. When a datagram doesn't fit, the scheduler evicts from the domain holding the most datagram bytes (oldest on a tie). Within that domain it takes the lowest-priority group's oldest datagram, and the new datagram is dropped if it would be the victim.SendDatagramError::Blocked,Event::DatagramsUnblocked, thedropflag, andsend_datagram_waitare removed.moq_tokio::quic::driver::Connection::send_datagram(data, group, order), the matchingquic::web::Sessionmethods, and the moq-uring equivalents.transport::poll::Session::poll_send_datagrambecomes a synchronoussend_datagram(payload, group: Option<&Group>, order: i64). The lite and IETF publishers send each datagram in its subscription's send group atpriority::in_group(sequence, 0, order). Backends without groups (web-transport adapters, moq-ffi, moq-wasm) ignore the group and order, and drop a datagram they have no room for.Public API impact (breaking)
Datagrams::send(data, group: Option<&SendGroup>, order: i64)replacessend(data, drop: bool). Removed:SendDatagramError::BlockedandEvent::DatagramsUnblocked.send_datagramtakesgroupandorder. Removed:send_datagram_waitandSendDatagram.Connection::send_datagramandweb::Session::send_datagramare added (inherent, with group and order).poll::Session::send_datagramreplacespoll_send_datagram.fuzzfeature): the datagram op carries anorderinstead ofdrop.Wire impact
None on the wire; the draft adds only scheduling guidance (SHOULD).
Tests
low_priority_datagrams_wait_behind_streams(none of the datagrams make it into the flight);high_priority_datagrams_preempt_streams(all of them do).With the old datagrams-first rule, the low-priority test sees all 8 datagrams in the first flight.
saturated.rs): datagrams merge into their group's order, share a group's credit, share bytes across saturated domains, preempt groups when ungrouped, lead the next packet when they don't fit, fit what they can in a clamped packet, and still send from a released group.retain_datagrams.datagrams_rank_within_their_subscriptionfor both the lite and IETF publishers.Benchmark
rs/moq-net/benches/schedule.rsaddsbench_datagram, which sweeps groups (or domains) {1, 16, 256, 4096} against datagrams queued per group {1, 16, 256}. Times are ns per op, short run:schedule_datagram_evict_groupsschedule_datagram_evict_domainsschedule_datagram_sendCost grows logarithmically with the table. Stream
fanoutkeeps its shape (subs_1/125,subs_4096/1642,subs_256/25658).Decided
doc/concept/moq-lite.mdsays the same for users, and its Priority row now counts datagrams in the equal-priority byte split for native moq-quic transports (browser publishers and transports without send groups are not covered). The moq-hang draft says nothing about datagram scheduling.Follow-ups
🤖 Generated with Claude Code
(Written by Opus 5.5)