Skip to content

fix(watch,publish): keep the delay across a rename and close frames read during teardown; cover renames in media QA - #5635

Merged
kixelated merged 9 commits into
mainfrom
quest/m1/browser-rename-check
Oct 17, 2026
Merged

kixelated merged 9 commits into
mainfrom
quest/m1/browser-rename-check

Conversation

@kixelated

@kixelated kixelated commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Unit tests do not prove that real WebCodecs playback follows the fresh track minted for a decoder identity change. This adds browser coverage, and fixes the two playback failures that coverage surfaced.

Audio stalls after a codec rename. Interop failed audio tone: codec rename 71/81 samples. A codec change renames the video track, and the replacement's Container.Consumer restarted its jitter estimate from the cold-start prior. The shared "auto" delay is the max over targets, and video's target follows that estimate. It dropped at promotion, by about 240 ms in the reproduction and about 560 ms in CI. It then climbed back as the new track relearned the same path's lateness. The audio ring follows that delay. It skipped buffered audio on the drop and parked playback on each rise. In CI the audio timestamp jumped 5144 to 5706 ms at the rename, followed by a 0.55 s stall.

Unclosed VideoFrame. CI logged A VideoFrame was garbage collected without being closed from the publisher page once. That was a real @moq/publish leak with two causes:

  1. Racing reads against teardown. The video encoder, preview, preview transcoder and stats-tab loops did effect.race(reader.read()). Teardown aborts synchronously, so a read that had already resolved, but whose continuation had not run yet, lost the race. Nothing closed the frame it carried.
  2. Replay wrapper enqueue after cancel. Capture's per-reader wrapper replays the held picture to a new reader. It awaited a live read, then called controller.enqueue after its consumer had cancelled. The enqueue threw and the frame leaked.

Both are reachable whenever a reader tears down while a frame is in flight: a subscriber leaving (demand gap), a codec rename swapping the track, or the preview being hidden.

Approach

Keep the estimate across a rename. The path's arrivals do not change with the track name, so a replacement consumer continues its predecessor's estimate. Container.Consumer takes a predecessor. Its learned histogram and startup ramp carry over through Jitter.fork(), and the reference window restarts, so the new track's opening backlog is measured against itself rather than against the old track's arrivals. The video decoder hands the on-screen rendition's consumer to the one replacing it, only when both read the same broadcast path: a source switch or a broadcast override to another path starts from the prior (Codex review). doc/concept/audio-jitter.md (normative) now states this under "Across renditions".

Leak fix. Read without racing. Each loop already cancels its reader on teardown, which ends a pending read. A frame that resolves after teardown is closed rather than encoded or kept. The replay wrapper records its cancellation and closes a frame it can no longer deliver.

Regression tests.

  • decoder.test.ts warms a real consumer on a manual clock: 33 ms frames flushed in bursts, which settles the delay above 300 ms. It then renames the track and plays a postMessage AudioRingBuffer that follows the shared delay in 20 ms steps. Without the fix it skips 280 ms and renders 13 silent ticks; with it, neither, and the delay never drops.
  • A second decoder.test.ts case warms the same bursty path, then moves the media to another path with on-time frames and asserts the delay falls. Without the path guard it holds at 380 ms.
  • jitter.test.ts checks that a fork keeps the estimate but measures the replacement's backlog against its own arrivals (copying the history instead fails it).
  • capture.test.ts sweeps teardown across every microtask offset after a push, for the encoder and for the preview, and asserts only the capture's held picture stays open. Each step settles with one setImmediate turn rather than a timer, since the pipeline runs on promises alone (Codex review). Reverting the leak fixes fails both sweeps at offset 2.

Media QA. Extend the existing deterministic media fixture with an AVC-to-VP8 change and a VP8 resize from 320x240 to 640x480. Check the catalog's resolved track, actual decoded and presented dimensions, frame progress and A/V sync. Source-generation bits in the presented pixels reject pictures from before each change. The codec-change generation advances only once the fixture's catalog advertises VP8, so pictures still encoded as AVC on the old track never carry it (Codex review).

The default media run includes these cases. just test media --media-cases rename selects them independently, explicitly omitting cold-start grading and unrelated controls. Default grading and every tone, timing and skew threshold stay unchanged, including the 90% tone floor.

Impact

  • Public API (@moq/hang): one new optional prop, ConsumerProps.predecessor?: Consumer. Jitter.fork() and the DecoderTrack fields are package-internal. @moq/watch: none.
  • Wire: none. Package versions: unchanged.
  • Test interface: optional --media-cases selection and a source-generation marker in the internal fixture.
  • Corrects the quest's stale resize expectation: fix(publish): mint a new track name when a rendition's decoder identity changes #5552 keeps coded-size growth on the existing track when the codec identity holds.

Decisions

  • ✅ Continue the outgoing estimate, rather than let the audio ring skip only when the delay rises.
    • The ring-side option leaves audio unable to shrink its depth after any rise, which is permanent A/V skew until a reset.
    • It would also be a second limiter on the delay, and audio-jitter.md says the histogram is the only dynamic.
    • Continuing the estimate removes both the drop and the climb at the source.
    • It is a fork with no warm-up threshold: the new track's own evidence takes over at the normal forget rate.
  • ✅ Land the rename-stall fix and the leak fix in this PR, since its media oracle fails on both. Alternative: split each into its own fix PR.
  • ✅ Fix every race(reader.read()) loop over VideoFrames (encoder, preview, transcoder, stats tab), not only the encoder. The audio encoder loop reads plain sample arrays with nothing to close, so it is left alone.
  • ✅ Mark the codec generation from the published catalog rather than tagging each decoded frame with its track.
  • ✅ Name the prop predecessor, not previous or replaces: the replaced consumer's role.

Validation

  • Full Nix just check passes. @moq/watch 372 and @moq/hang 242 unit tests pass.
  • The rename test fails without the fix (skipped 13440 samples, silent 13) and passes with it. The sweep tests fail without their fixes, as above.
  • Five focused real-browser runs (just test media --media-cases rename) on e32f983ae, on a machine at load 47 on 24 cores:
    • Three passed every window at 28.9-30.5 fps with tone 148-153 dB above the floor and no stall.
    • One failed coded-size growth tone (60/81). That window stays on the same video/1, so the predecessor path is not involved: presented video fell behind wall clock and the delay rose with real lateness (325 to 1680 ms).
    • One failed when a SUBSCRIBE_UPDATE for the old video track raced its finish and the relay's stream close (The stream was aborted by the remote server). That is a @moq/net race, listed under follow-ups.
  • Three more runs on e32f983ae passed both rename windows and failed the after-growth tone (48/61, 64/80, 71/81), each while the player logged late video frames (up to 204 ms) and the estimate rose with them.
  • Control on the base da76a4239, same load: one of three runs also failed after-growth tone (68/80), so that window's failure predates this change and tracks machine load. Across all eight runs on the fix, every run that reached the rename windows passed them (7/7).

Follow-ups

  • An audio rename dips the delay to 0. The audio decoder clears its estimate on teardown, before the replacement subscription opens. It wants the same predecessor hand-off.
  • @moq/net: a SUBSCRIBE_UPDATE write that fails because the publisher already finished the track should defer to the track's clean end rather than fail it.
  • Rust parity: check whether moq-mux/hang consumers that replace a track keep their estimate.
  • Retain quest/m1/browser-rename-check.md until smooth playback is established with the complete applicable oracle. quest/m1/publish-audio-channel-count.md already owns the unpinned cold-start tone symptom.

(Written by GPT-6, updated by Claude Opus 5.5)

🤖 Generated with Claude Code

kixelated and others added 2 commits October 15, 2026 18:08
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated
kixelated marked this pull request as ready for review October 16, 2026 05:55
@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 12 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7efafeac-bb5f-414c-8298-6655b19825f9
📥 Commits

Reviewing files that changed from the base of the PR and between 2aa7a95 and 9aca635.

📒 Files selected for processing: 24
  • doc/concept/audio-jitter.md
  • js/hang/src/container/consumer.test.ts
  • js/hang/src/container/consumer.ts
  • js/hang/src/container/jitter.test.ts
  • js/hang/src/container/jitter.ts
  • js/publish/src/preview.ts
  • js/publish/src/ui/components/stats-tab.ts
  • js/publish/src/video/capture.test.ts
  • js/publish/src/video/capture.ts
  • js/publish/src/video/encoder.ts
  • js/watch/src/video/decoder.test.ts
  • js/watch/src/video/decoder.ts
  • quest/m1/browser-rename-check.md
  • test/interop/README.md
  • test/interop/clients/js/media.ts
  • test/interop/clients/js/src/capabilities.test.ts
  • test/interop/clients/js/src/capabilities.ts
  • test/interop/clients/js/src/contract.ts
  • test/interop/clients/js/src/fixture.ts
  • test/interop/clients/js/src/pattern.test.ts
  • test/interop/clients/js/src/pattern.ts
  • test/interop/clients/js/src/probe.ts
  • test/interop/clients/js/src/setup.ts
  • test/interop/interop.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Interop triage before merge (not merging yet):

  • The first Interop attempt failed in media output + lifecycle on FAIL console: A VideoFrame was garbage collected without being closed, logged by the [fixture] (publisher) page during stop/republish, after the new codec-rename and coded-size cases had passed.
  • That message appears in none of the ~40 other recent Interop runs (all branches plus the last main schedule), so it is not a known base-branch flake. The rerun (attempt 2) passed with zero leak warnings, and two local full media runs showed none either, so it is intermittent.
  • Likely reading: a real publisher-side VideoFrame leak on a teardown path, surfaced by this PR's track switch and/or its 640x480 frames raising GC pressure. Candidate spots: Frames.subscribe in js/publish/src/video/capture.ts can controller.enqueue a frame after the outer stream was cancelled, and frames held inside the measure/hold TransformStreams are dropped without close() when the fanout cancels its source. Neither is confirmed.
  • Local runs on this machine also fail audio-tone windows broadly, including in controls that never reach the new case, so they are not a usable oracle here; CI's two attempts passed every tone window.

Holding the merge for a maintainer call on whether to land with the leak as a follow-up or chase it first.

(Written by Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 380bcdd399

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread test/interop/clients/js/src/fixture.ts Resolved
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T23:34:58.760511Z 9aca635 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

kixelated and others added 2 commits October 16, 2026 14:32
# Conflicts:
#	quest/m1/README.md
#	quest/m1/browser-rename-check.md
Two paths dropped a captured frame without closing it when a reader tore
down while the frame was crossing the capture:

- The encoder, preview, transcoder, and stats loops raced each read
  against effect teardown. A read that had already resolved lost the race
  to the synchronous abort, and its frame was never seen again.
- A new reader's replay wrapper enqueued a frame its consumer cancelled
  while the read was resolving; the enqueue threw and the frame leaked.

Read without racing and let the existing cancel end a pending read, close
anything that resolves after teardown, and close the frame the replay
wrapper can no longer deliver. A sweep test tears the encoder and the
preview down at every microtask offset after a push.

The media fixture also marks the codec-change generation once the catalog
advertises VP8, rather than on request, so old-codec pictures never carry
the new generation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title test(media): cover decoder identity changes and resize fix(publish): close frames read during teardown; cover renames in media QA Oct 16, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da76a4239b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/publish/src/video/capture.test.ts Resolved
kixelated and others added 2 commits October 16, 2026 15:24
A codec change renames the video track. The replacement's container
consumer restarted its jitter estimate from the cold-start prior, so the
shared "auto" delay dropped at promotion (340ms to 100ms in the
reproduction) and climbed back as the new track relearned the same
lateness. The audio ring follows that delay: it skipped buffered audio on
the drop and parked playback on each rise, which is the post-rename tone
failure the media QA caught.

The path's arrivals do not change with the track name, so a replacement
now continues its predecessor's estimate. `Container.Consumer` takes a
`predecessor`, whose learned histogram and startup ramp carry over
through `Jitter.fork()`, while the reference window restarts so the new
track's opening backlog is measured against itself. The video decoder
hands the on-screen rendition's consumer to the one replacing it.

A live-consumer test warms a real estimate, renames the track, and plays
a postMessage audio ring that follows the shared delay: without the fix
it skips 280ms and renders 13 silent ticks; with it, neither.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title fix(publish): close frames read during teardown; cover renames in media QA fix(watch,publish): keep the delay across a rename and close frames read during teardown; cover renames in media QA Oct 16, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

The new commit e32f983ae fixes the post-rename audio stall: a replacement video consumer continues its predecessor's jitter estimate through Jitter.fork(), so the shared delay neither drops nor climbs at a rename. Please focus on js/hang/src/container/{jitter,consumer}.ts and js/watch/src/video/decoder.ts.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5fd6e1f29d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread js/watch/src/video/decoder.ts Resolved
A source switch or a rendition `broadcast` override can move video to
another path, whose arrivals have nothing to do with what the outgoing
consumer learned. Inheriting its histogram there held a high delay for the
estimator's decay period, or underbuffered after a low-to-high switch.
Only hand the outgoing consumer to a replacement read from the same
broadcast path; another path starts from the prior. (Codex review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

eee6d87c9 addresses the path finding: the arrival estimate carries over only to a replacement on the same broadcast path.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: eee6d87c9b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# Conflicts:
#	test/interop/clients/js/media.ts
#	test/interop/interop.sh
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

8f8255b26 merges main. The conflict was main's multi-engine media QA (#5640) against this PR's --media-cases and VP8 capability probes: --media-cases now runs per engine, and encode vp8 / decode vp8 moved into src/capabilities.ts as publisher and player requirements.

(Written by Claude Opus 5.5)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 8f8255b264

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

The capture tests waited a 10ms timer per step, so the teardown sweeps
spent over a second sleeping and assumed the pipeline settles within
10ms on a loaded worker. The pipeline runs on promises alone, so one
macrotask turn drains it deterministically. The sweeps still fail
without the leak fix, now at offset 2. (Codex review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

@codex review

9aca63579 addresses the P1 on the teardown sweep: capture.test.ts now settles the pipeline with one setImmediate turn instead of a 10 ms timer. All earlier findings are addressed and resolved.

(Written by Claude Opus 5.5)

kixelated added a commit that referenced this pull request Oct 16, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 9aca63579b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary for 9aca63579:

  • Rename stall: a replacement video consumer on the same broadcast path continues its predecessor's arrival estimate (ConsumerProps.predecessor, Jitter.fork()), so the shared delay neither drops nor climbs at a codec rename. The normative audio-jitter.md says the same.
  • Leak: @moq/publish reader loops no longer race reads against teardown, and the Capture replay wrapper closes a frame it can no longer deliver.
  • Media QA: the AVC-to-VP8 rename and VP8 coded-size growth cases, selectable with --media-cases, now run per engine after the merge with main's multi-engine QA.
  • Decisions (✅ in the body):
    • continue the estimate rather than limit ring skips;
    • same path only;
    • the prop is named predecessor;
    • both fixes in this PR.
  • API: one new optional prop, ConsumerProps.predecessor. Wire: none.
  • Reviews: every Codex finding is addressed and resolved; Codex is clean on this head.
  • Follow-ups: planned in quest(m1): plan the audio rename estimate and the update-after-finish race #5788 (audio rename estimate; a subscribe update racing a finished track).

(Written by Claude Opus 5.5)

@kixelated
kixelated enabled auto-merge October 16, 2026 23:35
@kixelated
kixelated added this pull request to the merge queue Oct 16, 2026
Merged via the queue into main with commit 902c2bf Oct 17, 2026
13 checks passed
@kixelated
kixelated deleted the quest/m1/browser-rename-check branch October 17, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant