Skip to content

fix: Fix unknown Pixel Formats (specifically rgb) #4064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mrousavy merged 4 commits into from
Jul 1, 2026
Merged

fix: Fix unknown Pixel Formats (specifically rgb) #4064

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fe31d28
fix: Fix unknown Pixel Formats (specifically rgb)
mrousavy Jul 1, 2026
4ab5ca7
docs
mrousavy Jul 1, 2026
778ef3a
Update PixelFormat+fromImageFormat.kt
mrousavy Jul 1, 2026
f2599e2
remove pixelFormat constraint test
mrousavy Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 5 additions & 40 deletions apps/simple-camera/__tests__/visioncamera.constraints.harness.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -238,11 +238,14 @@ describe('VisionCamera - Constraints', () => {
const targetEdges = getEdges(targetResolution)
const maxAllowedEdges = getEdges(maxAllowedResolution)
const uhdEdges = getEdges(uhdResolution)
const supportedVideoResolutions = backDevice.getSupportedResolutions('video')
const supportedVideoResolutions =
backDevice.getSupportedResolutions('video')
const supportsTargetResolution = supportedVideoResolutions.some(
(resolution) => {
const edges = getEdges(resolution)
return edges.short === targetEdges.short && edges.long === targetEdges.long
return (
edges.short === targetEdges.short && edges.long === targetEdges.long
)
},
)
const supportsUhdResolution = supportedVideoResolutions.some(
Expand Down Expand Up @@ -472,44 +475,6 @@ describe('VisionCamera - Constraints', () => {
await session.stop()
})

it('honors a pixelFormat constraint matching a supported device format', async (context) => {
const candidate = backDevice.supportedPixelFormats.find(
(f) => f !== 'unknown',
)
if (candidate == null) {
return context.skip(
'pixelFormat constraint: device has no non-unknown formats',
)
}
const session = await VisionCamera.createCameraSession(false)
const frameOutput = VisionCamera.createFrameOutput({
targetResolution: CommonResolutions.HD_16_9,
pixelFormat: 'native',
enablePreviewSizedOutputBuffers: false,
enablePhysicalBufferRotation: false,
enableCameraMatrixDelivery: false,
allowDeferredStart: false,
dropFramesWhileBusy: true,
})
let received: CameraSessionConfig | undefined
await session.configure([
{
input: backDevice,
outputs: [{ output: frameOutput, mirrorMode: 'auto' }],
constraints: [{ pixelFormat: candidate }],
onSessionConfigSelected: (config) => {
received = config
},
},
])
await waitUntil(() => received != null, { timeout: 5_000 })
console.log(
`requested pixelFormat=${candidate} resolved=${received?.nativePixelFormat}`,
)
expect(received?.nativePixelFormat).toBe(candidate)
await session.stop()
})

it('resolves the same config via VisionCamera.resolveConstraints and session.configure', async () => {
const photoOutput = VisionCamera.createPhotoOutput({
targetResolution: CommonResolutions.HD_4_3,
Expand Down
6 changes: 6 additions & 0 deletions ...c/main/java/com/margelo/nitro/camera/extensions/converters/PixelFormat+fromImageFormat.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package com.margelo.nitro.camera.extensions.converters
import android.graphics.ImageFormat
import com.margelo.nitro.camera.PixelFormat
import com.margelo.nitro.camera.utils.PixelRange
import android.graphics.PixelFormat as AndroidPixelFormat

fun PixelFormat.Companion.fromImageFormat(
imageFormat: Int,
Expand All @@ -11,6 +12,11 @@ fun PixelFormat.Companion.fromImageFormat(
return when (imageFormat) {
ImageFormat.FLEX_RGB_888 -> PixelFormat.RGB_RGB_8_BIT
ImageFormat.FLEX_RGBA_8888 -> PixelFormat.RGB_RGBA_8_BIT
AndroidPixelFormat.RGBA_8888 -> PixelFormat.RGB_RGBA_8_BIT
AndroidPixelFormat.RGBX_8888 -> PixelFormat.RGB_RGB_8_BIT
AndroidPixelFormat.RGB_888 -> PixelFormat.RGB_RGB_8_BIT
ImageFormat.RAW_SENSOR -> PixelFormat.RAW_BAYER_UNPACKED_16_BIT
ImageFormat.RAW12 -> PixelFormat.RAW_BAYER_PACKED96_12_BIT
ImageFormat.DEPTH16 -> PixelFormat.DEPTH_16_BIT
ImageFormat.DEPTH_POINT_CLOUD -> PixelFormat.DEPTH_POINT_CLOUD_32_BIT
ImageFormat.YUV_420_888 -> {
Expand Down
36 changes: 18 additions & 18 deletions packages/react-native-vision-camera/src/specs/common-types/VideoPixelFormat.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,24 +7,24 @@ import type { PixelFormat } from './PixelFormat'
* Video Formats are either **YUV** or **RGB**.
* The most commonly used format is `yuv-420-8-bit-video`.
*
* | Format | Colorspace | Sampling | Bit depth | Color Range | Layout |
* |-----------------------------|------------|----------|-----------|--------------|---------------------|
* | `yuv-420-8-bit-video` | YUV | 4:2:0 | 8-bit | Video | Planar |
* | `yuv-420-8-bit-full` | YUV | 4:2:0 | 8-bit | Full | Planar |
* | `yuv-420-10-bit-video` | YUV | 4:2:0 | 10-bit | Video | Planar |
* | `yuv-420-10-bit-full` | YUV | 4:2:0 | 10-bit | Full | Planar |
* | `yuv-422-8-bit-video` | YUV | 4:2:2 | 8-bit | Video | Planar |
* | `yuv-422-8-bit-full` | YUV | 4:2:2 | 8-bit | Full | Planar |
* | `yuv-422-10-bit-video` | YUV | 4:2:2 | 10-bit | Video | Planar |
* | `yuv-422-10-bit-full` | YUV | 4:2:2 | 10-bit | Full | Planar |
* | `yuv-444-8-bit-video` | YUV | 4:4:4 | 8-bit | Video | Planar |
* | `yuv-444-8-bit-full` | YUV | 4:4:4 | 8-bit | Full | Planar |
* | `rgb-bgra-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved BGRA |
* | `rgb-rgba-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved RGBA |
* | `rgb-rgb-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved RGB |
* | `raw-bayer-packed96-12-bit` | RAW Bayer | Mosaic | 12-bit | Sensor | Packed96 Bayer |
* | `raw-bayer-unpacked-16-bit` | RAW Bayer | Mosaic | 16-bit | Sensor | Unpacked 16-bit |
* | `unknown` | — | — | — | — | Unknown |
* | Format | Colorspace | Sampling | Bit depth | Color Range | Layout |
* |-----------------------------|------------|----------|-----------|--------------|----------------------|
* | `yuv-420-8-bit-video` | YUV | 4:2:0 | 8-bit | Video | Planar |
* | `yuv-420-8-bit-full` | YUV | 4:2:0 | 8-bit | Full | Planar |
* | `yuv-420-10-bit-video` | YUV | 4:2:0 | 10-bit | Video | Planar |
* | `yuv-420-10-bit-full` | YUV | 4:2:0 | 10-bit | Full | Planar |
* | `yuv-422-8-bit-video` | YUV | 4:2:2 | 8-bit | Video | Planar |
* | `yuv-422-8-bit-full` | YUV | 4:2:2 | 8-bit | Full | Planar |
* | `yuv-422-10-bit-video` | YUV | 4:2:2 | 10-bit | Video | Planar |
* | `yuv-422-10-bit-full` | YUV | 4:2:2 | 10-bit | Full | Planar |
* | `yuv-444-8-bit-video` | YUV | 4:4:4 | 8-bit | Video | Planar |
* | `yuv-444-8-bit-full` | YUV | 4:4:4 | 8-bit | Full | Planar |
* | `rgb-bgra-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved BGRA |
* | `rgb-rgba-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved RGBA |
* | `rgb-rgb-8-bit` | RGB | 4:4:4 | 8-bit | Full | Interleaved RGB/RGBX |
* | `raw-bayer-packed96-12-bit` | RAW Bayer | Mosaic | 12-bit | Sensor | Packed96 Bayer |
* | `raw-bayer-unpacked-16-bit` | RAW Bayer | Mosaic | 16-bit | Sensor | Unpacked 16-bit |
* | `unknown` | — | — | — | — | Unknown |
*/
export type VideoPixelFormat =
| 'yuv-420-8-bit-video'
Expand Down
2 changes: 1 addition & 1 deletion packages/react-native-vision-camera/src/specs/instances/Frame.nitro.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ export interface Frame
/**
* Gets the {@linkcode PixelFormat} of this Frame's pixel data.
* Common formats are {@linkcode PixelFormat | 'yuv-420-8-bit-video'}
* for native YUV Frames, {@linkcode PixelFormat | 'rgb-bgra-32-bit'}
* for native YUV Frames, {@linkcode PixelFormat | 'rgb-bgra-8-bit'}
* for processed RGB Frames, or {@linkcode PixelFormat | 'private'} for
* zero-copy GPU-only Frames.
*
Expand Down
Loading