Skip to content

iOS: <Camera> records 4K — preview resolutionBias overrides the video output's targetResolution (failing repro)#4036

Open
radko93 wants to merge 1 commit into
mrousavy:mainfrom
radko93:fix/preview-bias-overrides-video-target
Open

iOS: <Camera> records 4K — preview resolutionBias overrides the video output's targetResolution (failing repro)#4036
radko93 wants to merge 1 commit into
mrousavy:mainfrom
radko93:fix/preview-bias-overrides-video-target

Conversation

@radko93

@radko93 radko93 commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Relates to #3963. Builds on #3965.

#3965 is a great root fix for the imperative case in #3963 — switching from a summed weighted penalty to a lexicographic penalty vector means a higher-priority constraint can't be numerically swamped by a lower-priority one. 👍

But it makes constraint order decisive, and that exposes a second path #3965 doesn't cover: the auto-injected preview resolutionBias. This PR adds a failing harness test for it (test-only — the Device Farm run is the reproduction). A proposed fix is in the <details> at the bottom.

The gap

In real <Camera> usage the resolver receives constraints in roughly this order (Camera.tsx prepends the preview to outputs; useCameraController.ts appends one { resolutionBias } per output):

[ ...userConstraints, { resolutionBias: previewOutput }, { resolutionBias: videoOutput }, ...internal ]

So the preview bias outranks the video bias. HybridCameraPreviewOutput.targetResolution is .min(screenSize) with streamType = .video, which gives every ≥screen-size format (4K) penalty 0 and penalizes the requested 720p/1080p. Under #3965's lexicographic comparison, for any formats that tie on fps the preview term is compared before the video term and still selects 4K — targetResolution is overridden again.

#3965's harness test uses [{ resolutionBias: videoOutput }, { fps: 60 }] with no preview, so this path is currently untested.

Test

Adds an it(...) to visioncamera.constraints.harness.ts that mirrors <Camera>: a preview output + a Frame output (targetResolution 720p), with [{ resolutionBias: preview }, { resolutionBias: frame }]. It asserts the delivered Frame is not ~4K. Frame dimensions are the only device-readable proof of the negotiated format (CameraSessionConfig exposes no resolution). Expected to fail on main and also with #3965 applied — happy to rebase onto #3965's branch to show that directly. Repro device: iPhone 17 Pro / iOS 26.5.1, front camera (camera-agnostic — any device with a format larger than the screen).

Proposed fix (preview should not outrank capture outputs) — direction for discussion, not verified

The .min(screenSize) rule is correct for a preview-only session (you want a screen-sized format there), so the fix is about priority, not the rule: when a capture output is present, its resolution target should outrank the preview's.

Under #3965's lexicographic model that just means ordering the auto-appended biases so capture outputs come before preview outputs. In useCameraController.ts:

const stableConstraints = useMemo<Constraint[]>(() => {
  const captureBiases = stableOutputs
    .filter((o) => !isPreviewOutput(o))
    .map<Constraint>((o) => ({ resolutionBias: o }))
  const previewBiases = stableOutputs
    .filter(isPreviewOutput)
    .map<Constraint>((o) => ({ resolutionBias: o }))
  // capture targets first (higher priority), preview last
  return [...constraints, ...captureBiases, ...previewBiases]
}, [/* ... */])

(Equivalently, drop the preview's resolutionBias entirely when a capture output is present — a preview can downscale from any format.) I haven't verified the exact JS predicate/ordering on a device, so treat this as the direction rather than a finished patch.

@vercel

vercel Bot commented Jun 22, 2026

Copy link
Copy Markdown

@radko93 is attempting to deploy a commit to the Margelo Team on Vercel.

A member of the Team first needs to authorize it.

…ideo output targetResolution

A capture output's `targetResolution` is ignored when a preview is attached.
`<Camera>` always injects a preview output and auto-appends one `{ resolutionBias }`
per output; the preview output's `.min(screenSize)` bias gives every >=screen-size
format (e.g. 4K) zero penalty and penalizes the capture output's `.closestTo(720p)`.
Because ConstraintResolver sums every bias into the total penalty, the preview bias
dominates and a 1280x720 target negotiates the device's max (4K) format.

This test asserts that a 720p Frame-output target does not deliver ~4K frames when a
preview is attached. It is expected to FAIL on current main (Frame dimensions are the
only device-readable proof of the negotiated format). A verified fix is described in
the PR description and can be applied to this branch.
@radko93 radko93 force-pushed the fix/preview-bias-overrides-video-target branch from 3f25f04 to 6b7b29b Compare June 22, 2026 19:05
@radko93 radko93 changed the title fix(ios): don't let preview resolutionBias override a video output's targetResolution Failing harness repro: preview resolutionBias overrides a video output's targetResolution (fix ready) Jun 22, 2026
@radko93 radko93 marked this pull request as ready for review June 22, 2026 19:07
@radko93 radko93 changed the title Failing harness repro: preview resolutionBias overrides a video output's targetResolution (fix ready) Failing harness repro: preview resolutionBias overrides a video output's targetResolution Jun 22, 2026
@radko93 radko93 changed the title Failing harness repro: preview resolutionBias overrides a video output's targetResolution Follow-up to #3965: <Camera> still records 4K — preview resolutionBias outranks the video target (failing repro) Jun 22, 2026
@radko93 radko93 changed the title Follow-up to #3965: <Camera> still records 4K — preview resolutionBias outranks the video target (failing repro) iOS: <Camera> records 4K — preview resolutionBias overrides the video output's targetResolution (failing repro) Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant