Skip to content

fix(engine): hold the last video frame at the inclusive clip end#1564

Merged
miguel-heygen merged 2 commits into
mainfrom
fix/engine-clip-end-frame-visibility
Jun 18, 2026
Merged

fix(engine): hold the last video frame at the inclusive clip end#1564
miguel-heygen merged 2 commits into
mainfrom
fix/engine-clip-end-frame-visibility

Conversation

@miguel-heygen

Copy link
Copy Markdown
Collaborator

What & why

The FrameLookupTable active set deactivated a video on an exclusive end-bound (globalTime < end), while the runtime keeps an element visible through currentTime <= end (core/runtime/init.ts:2202). At t === end, the runtime shows the element but the engine doesn't inject a video frame — the region renders blank for that one frame.

This is the same render-vs-runtime boundary alignment as #1339 / #1340, applied to the video frame injector.

Changes

  • Make the active window inclusive of the end (globalTime <= end) in all 4 comparison sites in FrameLookupTable (refreshActiveSet x3 + getFrame x1)
  • At t === end, serve the last extracted frame when frameIndex >= totalFrames (matching the runtime holding the element's final frame)
  • Mid-clip source exhaustion (t < end, the deliberate blank-tail) is unchanged

Verification

  • 3 new tests: inclusive end, short-source end hold, adjacent boundary overlap
  • All 5 FrameLookupTable tests pass, full engine suite green

Based on #1563 by @calcarazgre646 — added getFrame boundary consistency fix on top.

calcarazgre646 and others added 2 commits June 18, 2026 15:43
The frame-lookup active set deactivated a video on an exclusive end-bound
(globalTime < end), while the runtime keeps an element visible through
currentTime <= end (core/runtime init.ts). The rendered frame landing
exactly on a clip's end went blank even though the runtime still showed
the element on its final frame: one blank frame at the end of every clip
whose end lands on a frame boundary.

Make the active window inclusive of the end to match the runtime, and at
t === end serve the last extracted frame (the runtime holds the element's
final frame there too). Mid-clip source exhaustion (t < end) stays blank,
unchanged.
Make getFrame's end-bound inclusive (> instead of >=) for consistency
with the refreshActiveSet changes. getFrame is currently unused
externally but should match the same contract.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — surgical fix, well-grounded in the runtime/injector mechanism.

Verified the load-bearing claim. core/runtime/init.ts does use currentTime <= end (audio-source attach path) — confirmed by reading the file at the PR's head SHA, not just trusting the description. That's the boundary the engine has to match.

Mechanism confirmed in videoFrameInjector.ts. hideVideoElements/showVideoElements apply visibility: hidden !important to BOTH the native <video> AND the __render_frame_<id>__ image when inactive, and syncVideoFrameVisibility is driven by the active-set Map. So an exclusive end-bound on the active set wasn't just "missing a frame in the lookup" — it forced both layers to !important-hidden at t === end, and the runtime's non-important visibility: visible couldn't show through either. Same render-vs-runtime alignment story as #1339 / #1340 (audio clock / hardSyncAllMedia / razor split), now closed for the video frame injector — nice to see the full set arrive.

Edge cases I walked through end-to-end (all clean):

  • Active-set filter pass still uses strict globalTime > video.end for deactivation, so the new inclusive-add can't drift past the end — the per-frame iteration in getActiveFramePayloads only sees start <= globalTime <= end. The new branch's globalTime >= video.end is therefore effectively === end (the >= is float-safe defense-in-depth, not a real upper-bound widen).
  • Loop branch wins for looping clips. video.loop && frameIndex >= totalFrames fires before the new branch, so loop semantics are preserved — at t === end with loopDuration = source duration >= clip length, loop clips serve frame 0 via the modulo on localTime, which is the right loop behavior.
  • Zero-extracted-frame clips are guarded by extracted.totalFrames > 0 — no framePaths.get(-1) risk.
  • Short-source / blank-tail (test #2) is the right discriminator — globalTime >= video.end is false at mid-clip exhaustion, so the deliberate blank-tail intent is preserved.
  • Adjacent boundary a.end === b.start (test #3) — both clips active, matching the runtime's inclusive overlap. Good.
  • VFR/CFR or HDR-normalization frame-count drift — if extraction produces fewer frames than (end-start) * fps, the new branch holds the last extracted frame at t === end instead of going blank. Quietly robust.

One non-blocking observation I want to surface. FrameLookupTable.getFrame(videoId, globalTime) got the parity flip (>=>) but the helper it delegates to, getFrameAtTime, still returns null at t === end for non-loop clips (no hold-last-frame branch). So getFrame() and getActiveFramePayloads() now diverge on the boundary frame — getFrame() returns null where getActiveFramePayloads() returns the last frame payload.

I grep'd the repo and FrameLookupTable.getFrame has zero callers outside this file (the production render path is videoFrameInjectorgetActiveFramePayloads). The unrelated getFrame symbol elsewhere in the repo is iframe-handle code in sdk-playground/main.ts, getFrameElement in studio's manualOffsetDrag.ts, and getFrameBlob in shader-transitions — none are the FrameLookupTable method. So this asymmetry is dead-code-only today and ships safely.

The cleanup candidate (for a future PR, not blocking this one): either remove FrameLookupTable.getFrame since it's unreferenced, or extend getFrameAtTime's non-loop branch to hold-last-frame symmetric with the active-set path — the latter is riskier because getFrameAtTime IS exported externally per packages/engine/src/index.ts and docs. The dead-code-removal path is the safer cleanup. I'd lean toward that when convenient — leaving a method behind that disagrees with the production path is a footgun for the next agent who reaches for it expecting parity.

The 3 new tests cover the right invariants and the existing 745-test engine suite still passes per the PR description. Ship it.

@miguel-heygen miguel-heygen merged commit fdb8f33 into main Jun 18, 2026
42 of 43 checks passed
@miguel-heygen miguel-heygen deleted the fix/engine-clip-end-frame-visibility branch June 18, 2026 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants