feat: Allocate GPU-sampleable YUV buffers on Android for zero-copy GPU Frame import

[wcandillon]

What

On Android, pixelFormat: 'yuv' Frame Outputs now allocate their YUV_420_888 buffers with USAGE_CPU_READ_OFTEN | USAGE_GPU_SAMPLED_IMAGE (API 29+, when HardwareBuffer.isSupported(...) says the device can), via a new YuvImageReaderProxy mirroring the existing PrivateImageReaderProxy. Frames stay fully CPU-readable (planes are exposed unchanged), and can now additionally be imported zero-copy as GPU textures through Frame.getNativeBuffer().

Also fixes a stale error message: the physical-buffer-rotation error in HybridFrameOutput.kt referenced enableGpuBuffers, an option that no longer exists; it now points at pixelFormat: 'native' / enablePhysicalBufferRotation.

Why

CameraX's default ImageReaders allocate CPU-only gralloc buffers. Vulkan derives importable texture usages from the AHardwareBuffer usage bits, so without USAGE_GPU_SAMPLED_IMAGE the buffer cannot be sampled by the GPU. Concretely, Dawn (WebGPU) maps AHB usage to WebGPU usages in AHBFunctions.cpp and only grants TextureBinding when USAGE_GPU_SAMPLED_IMAGE is present. The same applies to Skia's Vulkan backend and any SamplerYcbcrConversion-based import.

Today this means the docs' promise of zero-copy GPU import of Frames (via NativeBuffer) only holds for pixelFormat: 'native' on Android. ML/vision pipelines that want CPU access and a GPU preview/effects pass (e.g. react-native-webgpu, Skia) have no working format. With this change, 'yuv' serves both.

Behavior / compatibility

• API < 29, or devices where HardwareBuffer.isSupported returns false for the GPU-sampled combination: falls back to a plain ImageReader, identical behavior to today.
• CPU consumers are unaffected: getPlanes() works as before.
• 'rgb' is unchanged (CameraX's RGBA conversion writes via CPU into its own reader; that pipeline isn't compatible with a custom GPU-usage reader).
• Like the existing PrivateImageReaderProxy, this uses CameraX's restricted ImageReaderProxyProvider API and a placeholder ImageInfo (CameraX wraps the proxy in a SettableImageProxy with the correct rotation before delivering it).

Testing

Untested on a physical device so far; review with that in mind. The structure deliberately mirrors the proven PrivateImageReaderProxy/PrivateImageProxy pair.

🤖 Generated with Claude Code

[vercel]

@wcandillon is attempting to deploy a commit to the Margelo Team on Vercel.

A member of the Team first needs to authorize it.

[mrousavy]

Hey - thanks for the PR!
I think for GPU sampleable images we should prefer to use 'native' Image Format, which uses PRIVATE, GPU Buffers on Android, and in the next coming CameraX release it has a native implementation - which doesn't require us shipping a custom ImageReaderProxy. I think that would be best, but do you have any reason to prefer YUV here? I guess I could also ask the CameraX team to expose flags like GPU_READ_OFTEN for ImageAnalyzer?

[wcandillon]

native is the better choice here (at least for RN WebGPU/Skia) but in guess the user chooses 'yuv' it allows for GPU only import. But I agree that this not important for WebGPU/Skia.
If you wouldn't like to merge this, if you want I can make a spearate PR for the error message fix.

[mrousavy]

Hey! I reported this feature request to the CameraX team here: https://issuetracker.google.com/u/1/issues/492934501 (in the latest comment down in this issue)

I feel like this makes most sense. The next CameraX release will have GPU buffer support in PRIVATE ImageFormat, while the current one has a slower CPU based route. I believe the HardwareBuffers that are streamed in YUV right now are still accessible on the GPU via Vulkan/OpenGL, it's just a slower path for now.

Not sure what we want to do, if I want to merge this custom YUV ImageReader in the meantime or not... 🤔

[mrousavy]

The build is failing in CI:

> Task :react-native-vision-camera:compileDebugKotlin
e: file:///home/runner/work/react-native-vision-camera/react-native-vision-camera/node_modules/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/utils/YuvImageProxy.kt:23:3 Platform declaration clash: The following declarations have the same JVM signature (getPlanes()[Landroidx/camera/core/ImageProxy$PlaneProxy;):

> Task :react-native-vision-camera:compileDebugKotlin FAILED
    fun `<get-planes>`(): Array<ImageProxy.PlaneProxy> defined in com.margelo.nitro.camera.utils.YuvImageProxy
    fun getPlanes(): Array<out ImageProxy.PlaneProxy> defined in com.margelo.nitro.camera.utils.YuvImageProxy
e: file:///home/runner/work/react-native-vision-camera/react-native-vision-camera/node_modules/react-native-vision-camera/android/src/main/java/com/margelo/nitro/camera/utils/YuvImageProxy.kt:83:3 Platform declaration clash: The following declarations have the same JVM signature (getPlanes()[Landroidx/camera/core/ImageProxy$PlaneProxy;):
    fun `<get-planes>`(): Array<ImageProxy.PlaneProxy> defined in com.margelo.nitro.camera.utils.YuvImageProxy
    fun getPlanes(): Array<out ImageProxy.PlaneProxy> defined in com.margelo.nitro.camera.utils.YuvImageProxy
> Task :react-native-nitro-modules:buildCMakeDebug[arm64-v8a]

[mrousavy]

The error message fix can be a separate PR I can merge right now, do you want to do that or should I? :)