Skip to content

test(verify): opt-in 30-minute soak recordings - #2666

Merged
richiemcilroy merged 2 commits into
mainfrom
ci/recording-soak
Oct 17, 2026
Merged

richiemcilroy merged 2 commits into
mainfrom
ci/recording-soak

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

Adds opt-in 30-minute soak recordings to the real-capture oracle, for drift, frame loss and sync over a real session length.

What it adds

  • CAP_VERIFY_SOAK_SECS=<seconds> (at most 1800) adds two cases to crates/verify/tests/recording.rs:
    • recording/soak/studio-camera: gated Studio, fragmented, with camera, microphone and system audio, and two holds (3 s and 10 s).
    • recording/soak/instant: gated Instant with microphone and system audio, and a 5 s hold.
  • Past 30 minutes the microphone's marker indices (12 bits, one every 0.5 s) run out, so the length is capped there.
  • Verify recording gains a soak_secs dispatch input, and the job timeout rises to 180 minutes when it is set. Without the variable nothing changes: the cases are not in any tier.

First run

gh workflow run verify-recording.yml -f soak_secs=1800 -f case_filter='recording/soak/*' -f runners=blacksmith, run 38998821647:

Blacksmith macOS 26 Blacksmith macOS 15
Studio + camera, 30 min pass pass
Instant, 30 min pass pass
Screen drift vs host clock +0.01 / -0.01 ms/min +0.01 / +0.04 ms/min
Camera frames 54,011 of 54,011, all distinct 54,010, all distinct
Microphone markers 3,600 of 3,600 3,600 of 3,600
Camera vs microphone -6.6 ms +0.0 ms
Screen vs microphone (Studio / Instant) +13.0 / +28.7 ms +35.9 / +19.9 ms

The macOS 26 Instant recording lost one isolated screen frame, within the miss budget; the others lost none. Analysis of a 30-minute recording takes 10 to 20 minutes per case on these runners.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; both previous findings are fixed and no new actionable issues were found.

Summary

Adds opt-in long recordings to check drift, frame loss, and audio/video sync.

  • The recording checks can now run two opt-in, session-length recordings.

Reviews (2) · Last reviewed commit: "test(verify): reject an invalid CAP_VERI..." · Reviewed by Greptile

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

Open in Capy

Comment thread crates/verify/tests/recording.rs
Comment thread crates/verify/tests/recording.rs Outdated
Comment thread crates/verify/tests/recording.rs Outdated
@richiemcilroy

Copy link
Copy Markdown
Member Author

Both addressed in dead545: a nonempty CAP_VERIFY_SOAK_SECS that is not a positive number of seconds (1800s, -1) now panics with the value instead of adding no cases, and the comment is down to why the length is capped (the 12-bit microphone marker index).

@richiemcilroy

Copy link
Copy Markdown
Member Author

@greptileai review

@vercel

vercel Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
cap-status Ready Ready Preview Oct 17, 2026 5:27am UTC

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy
richiemcilroy merged commit 9ed74ad into main Oct 17, 2026
47 checks passed

This branch was successfully deployed

2 active deployments
Preview – cap-web — dead545d Deployed Oct 17, 2026 by vercel[bot]
Preview – cap-status — dead545d Deployed Oct 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant