From 908e6175edd108f9199426f73ea6beef2053aed8 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 10 Oct 2026 20:30:14 -0700 Subject: [PATCH 1/2] quest(m1): plan early SUBSCRIBE_DROPs in Rust and JS publishers Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 1 + quest/m1/subscribe-drop-early.md | 59 ++++++++++++++++++++++++++++++++ quest/m1/subscribe-drop.md | 10 ++++-- 3 files changed, 67 insertions(+), 3 deletions(-) create mode 100644 quest/m1/subscribe-drop-early.md diff --git a/quest/m1/README.md b/quest/m1/README.md index ea0f0389b0a..3da40283e72 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -64,6 +64,7 @@ folded into Dropped sources. - [Binding audio delay](/quest/m1/binding-surface.md) - moq-ffi and every wrapper configure and observe audio playout delay - [Per-frame arrivals in @moq/watch](/quest/m1/watch-arrivals.md) - the audio and video decoders expose a window of recent frame arrivals, late and skipped marks included, as a signal - [Bump web-transport-iroh for the capsule close](/quest/m1/iroh-capsule-bump.md) - condition: moq-dev/web-transport#419 ships in a release, then moq's iroh HTTP/3 client reports a peer's close capsule +- [Early drops](/quest/m1/subscribe-drop-early.md) - Rust and JS publishers drop a sequence the moment they give it up, and forward an upstream drop - [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - every stream group in a lite subscription arrives or is dropped by name, and lite-07 drops its stream count for it - [Subscribe ranges](/quest/m1/subscribe-ranges/README.md) - a lite-07 SUBSCRIBE asks for past and live ranges with absolute or relative ends, in either order, and replaces FETCH; merged ranges never starve a subscriber; relays fill misses by range, including over moq-transport - [Reproduce cross-node announce flapping](/quest/m1/cross-node-flapping.md) - a clean close reaches a client on another node as one retraction, checked by a 3+ relay drill on main diff --git a/quest/m1/subscribe-drop-early.md b/quest/m1/subscribe-drop-early.md new file mode 100644 index 00000000000..4603d763c70 --- /dev/null +++ b/quest/m1/subscribe-drop-early.md @@ -0,0 +1,59 @@ +# [L] Publishers drop a group the moment they give it up + +## Goal + +On lite-05 and lite-06, the Rust and `@moq/net` publishers send a +SUBSCRIBE_DROP for a sequence as soon as they give it up, and still name every +sequence the subscription never got just before the FIN. A subscriber to +either publisher, directly or through a relay, settles without waiting out its +grace. Both implementations behave identically. + +Non-goals, kept in [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md): resets +before the group header, lite-07, and surfacing a drop to readers. + +## Plan + +Today only the Rust publisher sends SUBSCRIBE_DROP, and only at the end +(`drop_unserved` in `rs/moq-net/src/lite/publisher.rs`). The JS publisher +never sends one, so every sequence it skips leaves its subscriber waiting out +the grace (raised by Codex on moq-dev/moq#5301). + +Decided (maintainer, 2026-10-11): + +- Rust and JS together, identical behavior, rather than JS catching up to + Rust's end-of-track sweep alone. +- An early DROP covers: + - cursor skips: sequences below a SUBSCRIBE_START pin, below + `position_cursor`/`positionCursor`, or below the pre-06 SUBSCRIBE_UPDATE + re-pin (moq-dev/moq#5301); + - max-delay expiry: a group skipped for aging past the subscriber's max + delay. This also closes the parent's note that end-of-track aging misses + a gap the subscriber still waits on; + - explicit gaps, only when the upstream publisher named them: an incoming + SUBSCRIBE_DROP, or a producer cut. A sequence that simply has not arrived + is never inferred dropped, since a reordered group may still show up. +- Forwarding an upstream drop needs the track model to record a sequence as + dropped. Add that per-sequence mark to both models (`rs/moq-net/src/model/track.rs`, + `js/net/src/track.ts`), set by an incoming SUBSCRIBE_DROP and by a producer + cut, and have both publishers forward it as an early DROP. The parent + later reads the same mark to surface drops to readers. The mark must survive + the cache reclaiming the slot (see the parent's notes from #4998). +- The end-of-track sweep stays as the backstop; a sequence already dropped + early is not named twice. + +Update the lite draft's SUBSCRIBE_DROP text to say a publisher SHOULD send it +as soon as it gives a sequence up. Run `just drafts check`. + +Tests, with mock time: + +- Mirrored Rust and JS publisher unit tests for each trigger: the subscriber's + owed range settles at the skip, before the track ends and without the grace. +- A relay forwards an upstream DROP downstream on both stacks. +- A lite-05 and a lite-06 case in `just test interop --tail`: a JS publisher + skips a group and a Rust subscriber settles without waiting out its grace, + and the reverse. + +## Related + +- [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md) - the parent: lite-07, resets before the header, readers +- [Tail arrivals](/quest/m1/tail-arrivals.md) - per-track live dispositions survive cache eviction diff --git a/quest/m1/subscribe-drop.md b/quest/m1/subscribe-drop.md index 89228a61148..0541cc75ca3 100644 --- a/quest/m1/subscribe-drop.md +++ b/quest/m1/subscribe-drop.md @@ -18,9 +18,9 @@ live readers or recovery; the JS lite-03/04 path does not run that response loop. Once a track ends, the Rust publisher on lite-05 and lite-06 names every sequence from its SUBSCRIBE_START to the end that the subscription never got (skipped, -stale, or missing its head), just before its FIN. Still missing: JS -publishers, drops sent as soon as a group is given up rather than at the end, -and groups reset before their header. The end-of-track drops also miss a gap +stale, or missing its head), just before its FIN. Still missing: groups +reset before their header. JS publishers and drops sent as soon as a group is +given up belong to [Early drops](/quest/m1/subscribe-drop-early.md). The end-of-track drops also miss a gap the publisher aged out of its grace while the subscriber still waits on it: the publisher ages a gap from when it queued the group above it, which can be well before stream credit lets that group's header out. lite-07 (still `moq-lite-07-wip`, @@ -95,6 +95,10 @@ publisher skipped or never opened ends the track without waiting out the grace. Decided in the 2026-09-30 audit: the case moved here so the basic tail interop could land first. +## Required + +- [Early drops](/quest/m1/subscribe-drop-early.md) - Rust and JS publishers drop a sequence the moment they give it up, and forward an upstream drop + ## Related - [Tail arrivals](/quest/m1/tail-arrivals.md) - per-track live dispositions survive cache eviction From 8e80d760f8fa29a11d3d164ad123f650862f7f58 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 12 Oct 2026 18:33:52 -0700 Subject: [PATCH 2/2] quest(m1): defer pre-FIN settling and draft text to the parent, keep never-arrived gaps for the sweep Co-Authored-By: Claude Opus 5.5 --- quest/m1/subscribe-drop-early.md | 33 ++++++++++++++++++++++---------- quest/m1/subscribe-drop.md | 13 +++++++------ 2 files changed, 30 insertions(+), 16 deletions(-) diff --git a/quest/m1/subscribe-drop-early.md b/quest/m1/subscribe-drop-early.md index 4603d763c70..a35fe915930 100644 --- a/quest/m1/subscribe-drop-early.md +++ b/quest/m1/subscribe-drop-early.md @@ -5,11 +5,12 @@ On lite-05 and lite-06, the Rust and `@moq/net` publishers send a SUBSCRIBE_DROP for a sequence as soon as they give it up, and still name every sequence the subscription never got just before the FIN. A subscriber to -either publisher, directly or through a relay, settles without waiting out its -grace. Both implementations behave identically. +either publisher, directly or through a relay, settles at the FIN without +waiting out its grace. Both implementations behave identically. Non-goals, kept in [SUBSCRIBE_DROP](/quest/m1/subscribe-drop.md): resets -before the group header, lite-07, and surfacing a drop to readers. +before the group header, lite-07 and the draft text, surfacing a drop to +readers, and settling before the FIN. ## Plan @@ -26,9 +27,8 @@ Decided (maintainer, 2026-10-11): - cursor skips: sequences below a SUBSCRIBE_START pin, below `position_cursor`/`positionCursor`, or below the pre-06 SUBSCRIBE_UPDATE re-pin (moq-dev/moq#5301); - - max-delay expiry: a group skipped for aging past the subscriber's max - delay. This also closes the parent's note that end-of-track aging misses - a gap the subscriber still waits on; + - max-delay expiry: a buffered group skipped for aging past the + subscriber's max delay; - explicit gaps, only when the upstream publisher named them: an incoming SUBSCRIBE_DROP, or a producer cut. A sequence that simply has not arrived is never inferred dropped, since a reordered group may still show up. @@ -40,14 +40,27 @@ Decided (maintainer, 2026-10-11): the cache reclaiming the slot (see the parent's notes from #4998). - The end-of-track sweep stays as the backstop; a sequence already dropped early is not named twice. +- A gap that never arrived stays unresolved until that sweep names it. The + publisher must not count it delivered on its own grace clock (Rust marks a + group served before its stream opens, and `Tail::expire` folds aged gaps). + Max-delay expiry has no buffered group to drop for it, and it is not an + upstream DROP, so only the sweep covers it. This closes the parent's note + that end-of-track drops miss a gap the publisher aged out. -Update the lite draft's SUBSCRIBE_DROP text to say a publisher SHOULD send it -as soon as it gives a sequence up. Run `just drafts check`. +The draft's SUBSCRIBE_DROP section describes lite-07, which removed it, so the +SHOULD-send-early text lands with the parent's lite-07 restoration. Tests, with mock time: -- Mirrored Rust and JS publisher unit tests for each trigger: the subscriber's - owed range settles at the skip, before the track ends and without the grace. +- Mirrored Rust and JS publisher unit tests for each trigger: the publisher + emits the DROP at the skip and the subscriber accounts the range then, and + the owed range settles right after the FIN without the grace. Settling + before the FIN needs reader disposition and is tested in the parent. +- Mirrored never-arrived gap test: serve group 0, leave group 1 absent, queue + group 2 behind exhausted stream credit for longer than the publisher's + grace, then release credit and finish. Group 1 is named before the FIN + without waiting for the subscriber's later-started grace. +- No inferred DROP for group 1 while it could still arrive (live reordering). - A relay forwards an upstream DROP downstream on both stacks. - A lite-05 and a lite-06 case in `just test interop --tail`: a JS publisher skips a group and a Rust subscriber settles without waiting out its grace, diff --git a/quest/m1/subscribe-drop.md b/quest/m1/subscribe-drop.md index 0541cc75ca3..fcc3a52e2cc 100644 --- a/quest/m1/subscribe-drop.md +++ b/quest/m1/subscribe-drop.md @@ -20,10 +20,9 @@ publisher on lite-05 and lite-06 names every sequence from its SUBSCRIBE_START to the end that the subscription never got (skipped, stale, or missing its head), just before its FIN. Still missing: groups reset before their header. JS publishers and drops sent as soon as a group is -given up belong to [Early drops](/quest/m1/subscribe-drop-early.md). The end-of-track drops also miss a gap -the publisher aged out of its grace while the subscriber still waits on it: -the publisher ages a gap from when it queued the group above it, which can be -well before stream credit lets that group's header out. lite-07 (still `moq-lite-07-wip`, +given up belong to [Early drops](/quest/m1/subscribe-drop-early.md), which +also stops the end-of-track drops missing a gap the publisher aged out of its +grace. lite-07 (still `moq-lite-07-wip`, unpublished) removed SUBSCRIBE_DROP for a `Stream Count` on SUBSCRIBE_END (#4224). @@ -78,14 +77,16 @@ Learned from the shelved [#4998](https://github.com/moq-dev/moq/pull/4998) flakes under load (`Ok(None)` with 10 of 20 frames, #4332): the code can still end it clean until this lands, and it should pass reliably after. -Update `drafts/draft-lcurley-moq-lite.md` (SUBSCRIBE_DROP, SUBSCRIBE_END, the +Update `drafts/draft-lcurley-moq-lite.md` (SUBSCRIBE_DROP, including that a +publisher SHOULD send it as soon as it gives a sequence up; SUBSCRIBE_END; the lite-07 changelog), `doc/concept/moq-lite.md`, and the Rust and JS lite publishers, subscribers, and tail accounting. Run `just drafts check` and `just test interop --all`. Regression tests: a publisher that expires a group or resets a stream before its header; on each version the subscriber settles without waiting out the -grace. A skipped sequence at the end is covered on lite-05 through lite-07 +grace. An early DROP settles the dropped group for its reader before the +track's FIN, on both stacks. A skipped sequence at the end is covered on lite-05 through lite-07 (`track_tail::skipped_groups_end_without_the_grace`). Add the lite-07 drop case to the tail interop harness