Skip to content

feat(net): lite-07 GROUP Frame End continues bounded groups - #6447

Closed
kixelated wants to merge 7 commits into
mainfrom
quest/m1/subscribe-ranges/bounded-tail
Closed

kixelated wants to merge 7 commits into
mainfrom
quest/m1/subscribe-ranges/bounded-tail

Conversation

@kixelated

@kixelated kixelated commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Through one relay, subscriber A reads group 5 frames 0..3, then subscriber B asks for 3..5. The stream serving A stops at A's end frame, its FIN falsely completes the relay's cached group, and B never gets frames 3 and 4. The publisher half was missing too: a lite publisher marked a capped group served and moved its cursor past it, so a widening SUBSCRIBE_UPDATE never reached the tail even on one hop, and the lite draft forbade a second Group Stream for a sequence on one subscription.

Approach

Lite-07 end to end (PR 1 of the split below).

  • Wire (drafts/draft-lcurley-moq-lite.md): GROUP gains Frame End on lite-07, exclusive and encoded like SUBSCRIBE's (0 = runs to the end of the group). A FIN before it ends the group; a FIN at it ends only the stream. When SUBSCRIBE_UPDATE moves the end past such a group, the publisher continues it on a new Group Stream whose Frame Start is the earlier stream's Frame End, amending the one-Group-Stream rule for that case only. Frame End at or below Frame Start, or a FRAME at or past it, is malformed.
  • Model (rs/moq-net/src/model): group::Producer::suspend stops writing without ending the group; the copy keeps its write permission so readers wait past the last frame. track::Producer::poll_resume_group hands that permission to the stream continuing the group, waiting first for a stream it overtook. All pub(crate), marked // private: capability gap.
  • Rust subscriber: a FIN at Frame End suspends the copy; a continuation appends to it (plain and compressed), reads and discards a replayed prefix, and refuses a FRAME past its end. A suspended copy is aborted with Cancel once its subscription and streams are gone, so no reader waits on a continuation that cannot come. An evicted copy is fetched anew.
  • Publishers (Rust and js/net): a stream capped at the subscription's end is labelled with Frame End unless the group is already known to end by then. When an update moves the end past it, the tail is served on a new stream, which opens only after the capped stream wrote its FIN so the receiver never sees a continuation overtake the stream it continues. JS subscribers never request frame bounds, so they reject a Frame End as unrequested.

Decisions

Maintainer, 2026-10-23:

  • How a lite-07 stream distinguishes bounded suspension from source completion:
    • ✅ Explicit stream-level indication on lite-07-wip: GROUP Frame End (exclusive, 0 = to the group end; FIN below it ends the group, FIN at it ends only the stream; a later Group Stream serves the tail after widening).
    • Keep the wire unchanged and correlate separate requests.
    • A trailing terminal marker (rejected: FRAME has no type byte to distinguish it).
  • Lite-06, which has no such field:
  • IETF:
  • JS:
    • ✅ Mirror the change in js/net.
  • Scope:
    • ✅ Three PRs, lite-07 first: this PR, then lite-06 correlation [M] and IETF has_end plus Object-ID/ordinal handling [M], each on this PR's append path.
    • Two PRs, lite-07 then lite-06 and IETF together.
    • Model append ownership alone first.

Picked without a prompt (challenge if wrong):

  • The model hooks carry // private: capability gap: a custom subscriber cannot continue a cut copy until resuming goes public. No quest tracks making it public yet.
  • A continuation whose replayed prefix would need ietf extension records is refused as Duplicate; the IETF quest owns Object-ID continuity.

Impact

  • Public API: none. lite is private in Rust and unexported from @moq/net; the model additions are pub(crate).
  • Wire: lite-07-wip (unpublished) GROUP gains Frame End; a second Group Stream per sequence is allowed only to continue one that stopped at its Frame End. Lite-01..06 and moq-transport unchanged. Migration: none.
  • Behavior: a relay copy cut at one request's end now accepts the tail a later request asks for on lite-07. No performance claim.

Validation

  • rs/moq-net/tests/integration/bounded_tail.rs (mock time): plain and compressed relayed tails, local readers, early and exact real ends, repeated widening, and a tail after the subscription ended.
  • Subscriber unit tests: FIN at vs before Frame End, a continuation overtaking the stream it continues, replayed prefix not written twice, a replay ending early, a frame past Frame End, an evicted copy, and teardown with the subscription. Publisher tests in Rust and JS cover labelling and continuation; codec tests cover lite-07-only encoding.
  • just check passed except one moq-uring allocation test that hit the shared RLIMIT_MEMLOCK; just rs test --workspace then passed 9853/9853. just drafts check passed. just test interop --all passed (172 cases) after merging main; the run before the merge had one rust -> js-native-node audio timestamp gap on a Rust publisher and JS subscriber, a path this change does not alter (no subscriber there requests a frame bound).

Follow-ups

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 3 commits October 22, 2026 19:35
Preserve failing relay regressions and passing source-end controls while documenting the unresolved stream completion signal.

Co-Authored-By: GPT-6 <codex@openai.com>
Include lite06 in the unresolved completion-signal compatibility scope.

Co-Authored-By: GPT-6 <codex@openai.com>
@kixelated kixelated changed the title A copy that stopped at a request's end bound accepts its tail test(net): reproduce bounded group tail loss Oct 23, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision: park. Leave bounded tails unsupported for now; this draft stays as the checkpoint and the quest keeps the Goal. Do not pursue either wire option until the quest is picked up again.

(Written by Claude Opus 5.5)

kixelated and others added 4 commits October 23, 2026 11:20
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A lite-07 Group Stream capped at a subscription's end frame now says so
with `Frame End`. A FIN there suspends the receiver's cached copy instead
of finishing it, and the publisher serves the rest on a later Group
Stream once SUBSCRIBE_UPDATE moves the end past it, so a relay's copy
cut for one request accepts the tail another request asks for.

Lite-06 and moq-transport continuations move to their own quests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title test(net): reproduce bounded group tail loss feat(net): lite-07 GROUP Frame End continues bounded groups Oct 23, 2026
@kixelated
kixelated marked this pull request as ready for review October 23, 2026 20:51
@coderabbitai

coderabbitai Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 25 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 738ea9fa-cd30-4f18-8a13-dadd0d5f442a
📥 Commits

Reviewing files that changed from the base of the PR and between 46372b6 and a71e151.

📒 Files selected for processing: 23
  • drafts/draft-lcurley-moq-lite.md
  • js/net/src/lite/group.ts
  • js/net/src/lite/message.test.ts
  • js/net/src/lite/publisher.test.ts
  • js/net/src/lite/publisher.ts
  • js/net/src/lite/subscriber.ts
  • js/net/src/lite/version.ts
  • quest/m1/compress/ietf-range-widen.md
  • quest/m1/subscribe-ranges/README.md
  • quest/m1/subscribe-ranges/bounded-tail-ietf.md
  • quest/m1/subscribe-ranges/bounded-tail-lite06.md
  • quest/m1/subscribe-ranges/bounded-tail.md
  • rs/moq-net/src/fuzz.rs
  • rs/moq-net/src/lite/group.rs
  • rs/moq-net/src/lite/publisher.rs
  • rs/moq-net/src/lite/subscriber.rs
  • rs/moq-net/src/lite/version.rs
  • rs/moq-net/src/model/group.rs
  • rs/moq-net/src/model/track.rs
  • rs/moq-net/src/recv.rs
  • rs/moq-net/src/test_interop.rs
  • rs/moq-net/tests/integration/bounded_tail.rs
  • rs/moq-net/tests/integration/main.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-23T20:53:55.462900Z a71e151 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a71e15165b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +3969 to +3971
let Some(capped) = self.capped.take() else {
return;
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reapply updates to continuations still waiting to open

If two widening SUBSCRIBE_UPDATEs arrive before the capped stream releases its reader, the first moves self.capped into self.tail, and the second returns here without updating that queued tail's frame bounds. For example, widening an exclusive end from 2 to 4 and then 6 leaves the tail capped at 4, while serve() derives its GROUP Frame End from the latest end, 6. Its FIN at 4 then falsely declares the source group complete, losing subsequent frames. Update or reposition the queued tail under the latest bounds before opening its stream.

Useful? React with 👍 / 👎.

Comment on lines +117 to +118
groups.retain(|held| held.is_suspended() && !held.is_clone(&group));
groups.push(group);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Release suspended handles after their groups are aborted

On a long-lived subscription receiving successive capped groups, evicted or expired groups remain in this vector indefinitely: is_suspended() only reads the suspended flag, and neither commit_abort() nor release() clears it. Each new capped group therefore retains all previously aborted handles and scans them again, causing unbounded retained state and quadratic cumulative work even after the cache releases their payloads. Prune terminal groups as well as resumed ones so cache eviction actually releases this ownership, consistent with the parked-clone rule in rs/AGENTS.md.

AGENTS.md reference: rs/AGENTS.md:L15-L15

Useful? React with 👍 / 👎.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by moq-dev/moq#6465, which made every range and fetch start at a group's first frame and deleted quest/m1/subscribe-ranges/bounded-tail.md. A relay's upstream copy is now always requested whole, so a capped stream's FIN can no longer falsely complete the cached group, and the resume case is covered by #6465's crate-private hole fill.

The remaining gap, a third-party lite-07 peer widening a mid-group Frame End on one hop, can FETCH the tail; it doesn't justify GROUP Frame End plus the continuation and suspend/resume machinery. The follow-up quests this branch would have added (bounded-tail-lite06.md, bounded-tail-ietf.md) exist only here, so nothing on main needs deleting.

(Written by Claude Opus 5.5)

@kixelated kixelated closed this Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant