fix(engine): hold the last video frame at the inclusive clip end#1564
Conversation
The frame-lookup active set deactivated a video on an exclusive end-bound (globalTime < end), while the runtime keeps an element visible through currentTime <= end (core/runtime init.ts). The rendered frame landing exactly on a clip's end went blank even though the runtime still showed the element on its final frame: one blank frame at the end of every clip whose end lands on a frame boundary. Make the active window inclusive of the end to match the runtime, and at t === end serve the last extracted frame (the runtime holds the element's final frame there too). Mid-clip source exhaustion (t < end) stays blank, unchanged.
Make getFrame's end-bound inclusive (> instead of >=) for consistency with the refreshActiveSet changes. getFrame is currently unused externally but should match the same contract. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jrusso1020
left a comment
There was a problem hiding this comment.
Approving — surgical fix, well-grounded in the runtime/injector mechanism.
Verified the load-bearing claim. core/runtime/init.ts does use currentTime <= end (audio-source attach path) — confirmed by reading the file at the PR's head SHA, not just trusting the description. That's the boundary the engine has to match.
Mechanism confirmed in videoFrameInjector.ts. hideVideoElements/showVideoElements apply visibility: hidden !important to BOTH the native <video> AND the __render_frame_<id>__ image when inactive, and syncVideoFrameVisibility is driven by the active-set Map. So an exclusive end-bound on the active set wasn't just "missing a frame in the lookup" — it forced both layers to !important-hidden at t === end, and the runtime's non-important visibility: visible couldn't show through either. Same render-vs-runtime alignment story as #1339 / #1340 (audio clock / hardSyncAllMedia / razor split), now closed for the video frame injector — nice to see the full set arrive.
Edge cases I walked through end-to-end (all clean):
- Active-set filter pass still uses strict
globalTime > video.endfor deactivation, so the new inclusive-add can't drift past the end — the per-frame iteration ingetActiveFramePayloadsonly seesstart <= globalTime <= end. The new branch'sglobalTime >= video.endis therefore effectively=== end(the>=is float-safe defense-in-depth, not a real upper-bound widen). - Loop branch wins for looping clips.
video.loop && frameIndex >= totalFramesfires before the new branch, so loop semantics are preserved — att === endwithloopDuration = source duration >= clip length, loop clips serve frame 0 via the modulo onlocalTime, which is the right loop behavior. - Zero-extracted-frame clips are guarded by
extracted.totalFrames > 0— noframePaths.get(-1)risk. - Short-source / blank-tail (test #2) is the right discriminator —
globalTime >= video.endis false at mid-clip exhaustion, so the deliberate blank-tail intent is preserved. - Adjacent boundary
a.end === b.start(test #3) — both clips active, matching the runtime's inclusive overlap. Good. - VFR/CFR or HDR-normalization frame-count drift — if extraction produces fewer frames than
(end-start) * fps, the new branch holds the last extracted frame att === endinstead of going blank. Quietly robust.
One non-blocking observation I want to surface. FrameLookupTable.getFrame(videoId, globalTime) got the parity flip (>= → >) but the helper it delegates to, getFrameAtTime, still returns null at t === end for non-loop clips (no hold-last-frame branch). So getFrame() and getActiveFramePayloads() now diverge on the boundary frame — getFrame() returns null where getActiveFramePayloads() returns the last frame payload.
I grep'd the repo and FrameLookupTable.getFrame has zero callers outside this file (the production render path is videoFrameInjector → getActiveFramePayloads). The unrelated getFrame symbol elsewhere in the repo is iframe-handle code in sdk-playground/main.ts, getFrameElement in studio's manualOffsetDrag.ts, and getFrameBlob in shader-transitions — none are the FrameLookupTable method. So this asymmetry is dead-code-only today and ships safely.
The cleanup candidate (for a future PR, not blocking this one): either remove FrameLookupTable.getFrame since it's unreferenced, or extend getFrameAtTime's non-loop branch to hold-last-frame symmetric with the active-set path — the latter is riskier because getFrameAtTime IS exported externally per packages/engine/src/index.ts and docs. The dead-code-removal path is the safer cleanup. I'd lean toward that when convenient — leaving a method behind that disagrees with the production path is a footgun for the next agent who reaches for it expecting parity.
The 3 new tests cover the right invariants and the existing 745-test engine suite still passes per the PR description. Ship it.
What & why
The
FrameLookupTableactive set deactivated a video on an exclusive end-bound (globalTime < end), while the runtime keeps an element visible throughcurrentTime <= end(core/runtime/init.ts:2202). Att === end, the runtime shows the element but the engine doesn't inject a video frame — the region renders blank for that one frame.This is the same render-vs-runtime boundary alignment as #1339 / #1340, applied to the video frame injector.
Changes
globalTime <= end) in all 4 comparison sites inFrameLookupTable(refreshActiveSetx3 +getFramex1)t === end, serve the last extracted frame whenframeIndex >= totalFrames(matching the runtime holding the element's final frame)t < end, the deliberate blank-tail) is unchangedVerification
Based on #1563 by @calcarazgre646 — added
getFrameboundary consistency fix on top.