[kixelated]

Problem

The remaining fastest-reader work after moq-dev/moq#7388 and moq-dev/moq#7624:

• two-hop retention evidence
• qmux heap and throughput evidence
• real relay WebSocket accept-path evidence
• a transport-sized cache pacing window: ingest still paced at a fixed model::pace::AHEAD = 16 MiB, even with 1 MiB of stream credit.

Decisions

Decided 2026-11-03 (maintainer, /quest-unblock):

• ✅ Transport-sized cache window. Keep the decided plan: ingest and its tests take the cache pacing window from the transport's stream credit, replacing the fixed model::pace::AHEAD = 16 MiB. Reason: memory follows what the peer can actually take in flight.
• Accept the fixed 16 MiB and correct the Plan (not chosen).

Approach

• Transport-sized pacing window. Each ingest stream takes its pacing window when it is accepted or opened, from its session's receive_window (model::pace::window, coding::Pause::window). The window is the stream credit, no more than the connection window, capped at the 16 MiB ahead half of the live tail (AHEAD). group::Producer::poll_paced, frame::Producer::poll_ready and recv::Group::poll_admit now take the window. Local writers (group::Producer::poll_ready/ready) keep 16 MiB, since they have no transport. A transport that reports no window never pauses, as before.
  • Reading the window once per stream keeps the per-chunk pacing check lock-free; it costs one windows read per accepted stream.
• Two-hop evidence. Carry a 64 MiB group through two simulated session relays on lite-06, lite-07-wip, and moq-transport-22. Park the final reader, check cache charges and origin backpressure, then verify every byte and clean completion.
• Relay WebSocket accept path. Start an actual Relay from operator configuration and inspect its WebSocket qmux transport parameters before admission. Pin connection, all stream receive windows, and stream counts across qmux-00/01/02.
• qmux receive benchmark (Nightly). Bounded TCP/WebSocket byte pipes, qmux-00/01/02, 1/8/64 sessions, and 1/8 streams. Native unread leases hold 512 KiB each, so eight fill a 4 MiB connection window. Every round verifies payloads, releases credit, and refills.
• Quest and docs. The quest records the decision and what remains. doc/concept/moq-lite.md describes the per-hop window as the stream receive window capped at 16 MB.

Evidence

Two-relay cache charges while parked, with 4 MiB connection and 1 MiB stream credit per hop:

Window	Upstream relay (incl. sent history)	Downstream relay
Transport-sized (this head)	3,481,056-3,546,640 B	985,952-986,000 B
Fixed 16 MiB (previous head)	33,544,512-33,544,560 B	16,737,968-16,738,016 B

With the transport-sized window, the origin stopped after 327 of 1,024 frames. All 64 MiB arrived after reading resumed. Cache charges include frame slots and relay metadata. They exclude transport buffers and RSS.

The one-hop idle reader now holds three 1 MiB frames under 4 MiB of stream credit, and fifteen when the credit is past the 16 MiB cap.

All 36 qmux cases completed with exactly one heap report per case. With 64 sessions and eight streams per session, each receiver held 4,194,304 unread bytes:

Framing	qmux draft	Live heap bytes/session	Payload GiB/s, point estimate
pipe	02	4,506,163	2.8730
ws	02	4,671,849	3.0469
pipe	01	4,506,154	4.2098
ws	01	4,671,843	3.3289
pipe	00	4,506,265	3.1196
ws	00	4,671,979	3.2538

How to read the qmux table:

• Heap counts live requested allocator bytes for both endpoints, tasks, stream tables, pipes, and receive backings, after two untimed refill rounds. It excludes the runtime and the shared source payload.
• Throughput includes allocation counters, payload verification, and copy cost.
• These are shared-host observations, not the required quiet-host comparison or a regression verdict. Criterion's automatic comparisons are not acceptance evidence.
• The fixture isolates receive ownership from TLS, kernel sockets, congestion, and full MoQ protocol costs.

Impact

• Public API: none.
• Wire: none.
• Production behavior: a relay or client now buffers about one stream receive window (1 MiB at the default [quic] windows) of a paced group past its fastest reader, instead of 16 MiB. A transport whose stream credit is 16 MiB or more paces at 16 MiB as before.

Validation

• just ci on this host: all 11,526 Rust tests and every lane except check flake passed.
• Pre-existing failure, check flake: the Nix github-runner image build fails with unshare: write failed /proc/self/uid_map: Operation not permitted.
  • Same failure on this head (just ci) and on main 90980d8 (CHECK_MODULES=flake just ci --all), on this host.
  • quest/m0/ci/userns-uid-map.md owns the fix.
  • No other test host was free to run the image build.
• cargo test --all-features -p moq-net -p moq-relay -p qmux@0.7.0 -p moq-tokio passed, including the six fastest_reader integration tests and the new a_smaller_window_paces_sooner, the_window_is_the_stream_credit_capped, and a_stream_paces_at_its_credit unit tests.
• quest check --base origin/main passed.

Follow-ups

• The broader real-transport MoQ session benchmark series stays in transport x version benches.
• Out of scope: the relay advertises qmux-02 as a bare fallback, while versioned {qmux}.{moq} pairs list only 00/01. Recommend a separate quest to decide whether to advertise versioned qmux-02 pairs; this attempt does not change negotiation.

Waiting

• Maintainer: provide a host without concurrent workloads. An agent then runs repeated alternating group_read_frames, group_contended, group_relay_frames, and group_paced comparisons against pre-pacing base e7a5c10, then against this head, and records them. Keep this draft until the comparison is recorded.
  • None was quiet on 2026-11-03, checked twice between 13:03 and 13:19 UTC:
    • nixos2 (12 cores): load 3.5-4.5, with a rustc build and three Claude sessions.
    • nixos (32 cores): load 20-28, with clippy and Nix builds.
    • kixtop (10 cores): load 8-15.6.
  • An agent cannot free a host other sessions are using.

Review

Codex, gpt-6.1-sol, medium effort, round 1 (new session after the design change), reviewed head cf4ae07e. No findings.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

[kixelated]

Design change: per the maintainer's 2026-11-03 /quest-unblock decision, ingest now paces each group at its stream's receive credit, capped at 16 MiB, instead of the fixed model::pace::AHEAD = 16 MiB. Each stream reads the window from its session's receive_window when it is accepted or opened. Local writers keep 16 MiB.

With 1 MiB of stream credit per hop, the two-relay test's downstream cache drops from about 16.7 MB to about 0.99 MB. The upstream cache, including sent history, drops from about 33.5 MB to about 3.5 MB.

The quiet-host benchmark comparison is still waiting on a free host, so this stays a draft.

(Written by Claude Opus 5.5)