Creator workflow suite: greenscreen facecam pipeline, scene styles, and recording reliability fixes by justmaiko12 · Pull Request #676 · webadderallorg/Recordly

justmaiko12 · 2026-06-12T04:13:23Z

Summary

A creator-workflow feature suite built and battle-tested while producing real facecam-over-screen content (courses, presentations, demos) with a green screen and an iPhone Continuity Camera. It contains two halves:

Recording reliability fixes that affect every user (root-caused, not patched)
Creator features: a full facecam greenscreen pipeline and per-segment "scene styles"

Everything renders identically in the editor preview and both exporters, persists with projects (older project files open unchanged), and degrades gracefully (no WebGL → raw webcam; missing assets → sensible fallbacks).

I'm happy to split this into smaller PRs per feature if that's easier to review — say the word and I'll carve it up.

Bug fixes (likely the easiest wins)

Webcam frame gaps from camera device renegotiation

Recordings could contain multi-second frozen-facecam gaps (we measured 24s missing from a 72s clip via ffprobe). Root cause: the floating preview and the recorder each held their own getUserMedia with different constraints; any consumer churn (HUD compact transition, popover open/close) made Chromium restart the physical camera, and slow-renegotiation devices (iPhone Continuity Camera) stall 3–12s while the recorder's track receives no frames.

Fix: a refcounted shared webcam session (src/lib/webcamSession.ts) — one getUserMedia per device; the preview attaches to it and the recorder records a cloned track, so the device never restarts mid-session. Also: dead-stream self-healing (ended tracks are never reused — Continuity Camera disconnects recover on reselect) and a [webcam-watchdog] that logs track mute/unmute with timestamps so future stalls leave evidence. Verified: post-fix recordings show 0 frame gaps.

HUD invisible over fullscreen apps (macOS)

The HUD overlay never opted into fullscreen Spaces, so it vanished when presenting. Fixed with setVisibleOnAllWorkspaces(visibleOnFullScreen) + screen-saver window level, matching the update-toast window's existing pattern.

Notch black-bar auto-crop

Fullscreen apps on notched MacBooks render the menu-bar band pure black, which screen captures faithfully record. Fresh recordings are now sampled (3 frames, conservative thresholds — dark themes/black slides are never falsely detected; unit-tested) and the bars cropped automatically.

Creator features

Facecam greenscreen pipeline (WebGL2)

Chroma key with eyedropper + auto key-color detection (samples the frame border for the dominant screen color — real screens shift far from pure green under lighting), edge softness, fixed spill suppression, and background-image compositing. Garbage matte with box or bezier pen mask (insert-on-path, smooth/corner toggling, mirrored handles, fullscreen editor dialog) and feather. Facecam color controls (brightness/contrast/highlights/shadows) applied to the keyed foreground. The per-pixel math lives in a pure, unit-tested module (chromaKeyMath.ts) that the shader mirrors exactly.

Scene styles: per-segment fill-frame ("Fullscreen") regions

Time ranges where the recording covers the full canvas edge-to-edge (presentation mode) vs the framed look (demo mode), with an animated transition between the two layouts using the existing easing. Controlled by a timeline track, ⌥+. / ⌥+, editor shortcuts, header toggle buttons — and the same hotkeys during recording, persisted as a sidecar event file and imported as pre-split segments on project open (mirrors the webcam-layout-events pattern). Zooms are automatically suppressed inside fullscreen segments. "Remove background" now rides this system (cover-crop at the project aspect) instead of switching to native aspect.

Editor ergonomics

Facecam FPS preference (24/30/60), instant preview updates for webcam setting changes, marquee multi-select with batch delete on the timeline, condensed track rows, teleprompter auto-close when recording stops, background image no longer mirrored by "Mirror webcam".

Testing

~120 new unit tests (key/matte/color math, mask path geometry, session manager semantics, fill-frame regions/persistence, black-bar detection, marquee selection); suite at 1028 passing
tsc --noEmit and biome clean on all touched files
Manually verified end-to-end on macOS (Apple Silicon): record with hotkeys → edit → export, including Continuity Camera disconnect/reconnect scenarios

🤖 Generated with Claude Code

On macOS, when recording mic-only (no system audio), the native helper writes mic audio both into the video's inline track and a .mic sidecar. The preview was playing both sources simultaneously, causing an audible echo. Mute the embedded preview whenever sidecar tracks exist but the video path was not listed as a distinct audio source (hasEmbeddedSourceAudio === false). Export behavior (includeEmbeddedInExport) is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>