iOS: <Camera> records 4K — preview resolutionBias overrides the video output's targetResolution (failing repro)#4036
Open
radko93 wants to merge 1 commit into
Conversation
|
@radko93 is attempting to deploy a commit to the Margelo Team on Vercel. A member of the Team first needs to authorize it. |
…ideo output targetResolution
A capture output's `targetResolution` is ignored when a preview is attached.
`<Camera>` always injects a preview output and auto-appends one `{ resolutionBias }`
per output; the preview output's `.min(screenSize)` bias gives every >=screen-size
format (e.g. 4K) zero penalty and penalizes the capture output's `.closestTo(720p)`.
Because ConstraintResolver sums every bias into the total penalty, the preview bias
dominates and a 1280x720 target negotiates the device's max (4K) format.
This test asserts that a 720p Frame-output target does not deliver ~4K frames when a
preview is attached. It is expected to FAIL on current main (Frame dimensions are the
only device-readable proof of the negotiated format). A verified fix is described in
the PR description and can be applied to this branch.
3f25f04 to
6b7b29b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relates to #3963. Builds on #3965.
#3965 is a great root fix for the imperative case in #3963 — switching from a summed weighted penalty to a lexicographic penalty vector means a higher-priority constraint can't be numerically swamped by a lower-priority one. 👍
But it makes constraint order decisive, and that exposes a second path #3965 doesn't cover: the auto-injected preview
resolutionBias. This PR adds a failing harness test for it (test-only — the Device Farm run is the reproduction). A proposed fix is in the<details>at the bottom.The gap
In real
<Camera>usage the resolver receives constraints in roughly this order (Camera.tsxprepends the preview tooutputs;useCameraController.tsappends one{ resolutionBias }per output):So the preview bias outranks the video bias.
HybridCameraPreviewOutput.targetResolutionis.min(screenSize)withstreamType = .video, which gives every ≥screen-size format (4K) penalty0and penalizes the requested 720p/1080p. Under #3965's lexicographic comparison, for any formats that tie on fps the preview term is compared before the video term and still selects 4K —targetResolutionis overridden again.#3965's harness test uses
[{ resolutionBias: videoOutput }, { fps: 60 }]with no preview, so this path is currently untested.Test
Adds an
it(...)tovisioncamera.constraints.harness.tsthat mirrors<Camera>: a preview output + a Frame output (targetResolution720p), with[{ resolutionBias: preview }, { resolutionBias: frame }]. It asserts the delivered Frame is not ~4K. Frame dimensions are the only device-readable proof of the negotiated format (CameraSessionConfigexposes no resolution). Expected to fail onmainand also with #3965 applied — happy to rebase onto #3965's branch to show that directly. Repro device: iPhone 17 Pro / iOS 26.5.1, front camera (camera-agnostic — any device with a format larger than the screen).Proposed fix (preview should not outrank capture outputs) — direction for discussion, not verified
The
.min(screenSize)rule is correct for a preview-only session (you want a screen-sized format there), so the fix is about priority, not the rule: when a capture output is present, its resolution target should outrank the preview's.Under #3965's lexicographic model that just means ordering the auto-appended biases so capture outputs come before preview outputs. In
useCameraController.ts:(Equivalently, drop the preview's
resolutionBiasentirely when a capture output is present — a preview can downscale from any format.) I haven't verified the exact JS predicate/ordering on a device, so treat this as the direction rather than a finished patch.