Skip to content

feat(playground): add MJPEG live stream for iOS playground#2004

Merged
quanru merged 4 commits intomainfrom
feat/ios-mjpeg-live-stream
Feb 13, 2026
Merged

feat(playground): add MJPEG live stream for iOS playground#2004
quanru merged 4 commits intomainfrom
feat/ios-mjpeg-live-stream

Conversation

@quanru
Copy link
Collaborator

@quanru quanru commented Feb 12, 2026

Summary

  • Add /mjpeg endpoint to PlaygroundServer that streams screenshots via multipart/x-mixed-replace (standard MJPEG protocol), replacing the 5-second polling with real-time screen preview for iOS playground
  • Support configurable fps via ?fps=N query parameter (default 10, max 30), with error backoff to handle transient session unavailability during task execution
  • Update ScreenshotViewer component with an optional mjpegUrl prop — when provided, renders a native <img> tag consuming the MJPEG stream and hides polling UI to stay consistent with Android's ScrcpyPlayer

Test plan

  • Start iOS playground (cd packages/ios && pnpm run playground) and confirm the right panel shows a live screen stream
  • Visit http://localhost:5800/mjpeg directly to verify the raw MJPEG stream works in the browser
  • Execute a task and confirm the stream recovers automatically after the agent session is recreated
  • Start Android playground and confirm it still uses the existing polling/scrcpy behavior (no mjpegUrl passed)

Replace 5-second polling with real-time MJPEG streaming for iOS playground
screen preview, providing a much smoother experience similar to Android's
scrcpy stream.
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Feb 12, 2026

Deploying midscene with  Cloudflare Pages  Cloudflare Pages

Latest commit: 2f5f56e
Status: ✅  Deploy successful!
Preview URL: https://6a874161.midscene.pages.dev
Branch Preview URL: https://feat-ios-mjpeg-live-stream.midscene.pages.dev

View logs

- Add wdaMjpegPort option to IOSDeviceOpt (default 9100)
- Probe WDA MJPEG server once, cache result to avoid repeated failures
- Fall back to polling mode seamlessly when native stream unavailable
- Add clear logging to indicate which streaming mode is active
Also configure WDA MJPEG settings (quality, framerate, scaling) during
iOS session setup to ensure the MJPEG server is ready when port 9100
is forwarded.
@quanru quanru force-pushed the feat/ios-mjpeg-live-stream branch from fc4262a to 2f5f56e Compare February 12, 2026 13:12
@quanru quanru merged commit 7a21471 into main Feb 13, 2026
8 checks passed
@quanru quanru deleted the feat/ios-mjpeg-live-stream branch February 13, 2026 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants