Performance

import { Tab, Tabs } from 'fumadocs-ui/components/tabs'

VisionCamera's modern architecture, super-fast native bindings via Nitro Modules, and best practices in-place allows it to be highly optimized for every Camera situation. As with all advanced tools, there are still ways to mis-use it. This guide covers common tips to improve Camera performance in VisionCamera.

Simpler Camera Device

Selecting a "simpler" CameraDevice (i.e. a Camera Device with less physicalDevices) allows the Camera to initialize faster as it does not have to start multiple devices at once. You can prefer a simple 'wide-angle' Camera over something like a 'triple-camera' to significantly speed up initialization time.

<Tabs items={['Hooks API', 'Imperative API']}> <Tab value="Hooks API">

const fasterDevice = useCameraDevice('back', {
  physicalDevices: ['wide-angle-camera'],
})
const slowerDevice = useCameraDevice('back', {
  physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera'],
})

const devices = getAllCameraDevices()
const fasterDevice = getCameraDevice(devices, 'back', {
  physicalDevices: ['wide-angle-camera'],
})
const slowerDevice = getCameraDevice(devices, 'back', {
  physicalDevices: ['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera'],
})

[!TIP] See "Camera Devices" for more information.

No Video HDR

Video HDR uses 10-bit formats and/or additional processing steps that come with additional computation overhead. Disable Video HDR for higher efficiency.

[!TIP] See "Video HDR" for more information.

Video Stabilization

Video Stabilization requires additional overhead to start the algorithm and introduces capture latency. Disabling Video Stabilization can significantly speed up the Camera initialization time.

[!TIP] See "Video Stabilization" for more information.

Pixel Format

The Camera's native PixelFormat is a YUV-like format. If you set a CameraFrameOutput's pixelFormat to 'rgb', the pipeline will need to convert the YUV buffers to RGB, which introduces additional overhead and consumes more memory.

If you are using any Frame Processor Plugins that work with RGB, try to replace them with YUV-based plugins instead and set your pixelFormat to 'yuv'.

[!TIP] See "Pixel Formats Map" for more information.

Barcode Scanner vs Object Output

The Barcode Scanner is a custom processing pipeline using MLKit for barcode detection. On iOS, the Object Output is a native Camera pipeline, which also allows barcode detection and is much more lightweight and efficient.

Consider using the Object Output instead of the Barcode Scanner on iOS for better performance if it suits your needs, and only fall back to the Barcode Scanner otherwise (e.g. on Android).

[!TIP] See "The Barcode Scanner" or "The Object Output" for more information.

Code Scanning Formats

When using the Barcode Scanner or the Object Output, only enable the code formats you actually need to scan to improve performance.

No physical Buffer rotation

Instead of physically rotating buffers, your Frame Processor Pipeline should be able to work with orientation and mirroring via flags. Ensure your CameraFrameOutput is not configured with enablePhysicalBufferRotation, as this causes latency and requires conversion overhead.

[!TIP] See "Orientation" for more information.

Disable unneeded Outputs

Only attach CameraOutputs that are actually needed. The more outputs you add to a CameraSession, the more processing power is required.

No Skia Frame Processor

If you are not using Skia, use the regular <Camera /> view instead of the <SkiaCamera /> view, as the <SkiaCamera /> view requires much more processing power.

Using isActive

The isActive property controls whether the Camera should actively stream frames. Instead of fully unmounting the <Camera /> component and remounting it again, keep it mounted and just switch isActive on or off. This allows the Camera to resume much faster as it internally keeps the session warmed up.

[!TIP] See "Lifecycle" for more information.

Fast Photos

If you need to take photos as fast as possible, use a qualityPrioritization of 'speed' to speed up the photo pipeline:

const photoOutput = usePhotoOutput({
  qualityPrioritization: 'speed'
})

Prepare Photo Settings

Some photo captures require the native Camera pipeline to allocate additional buffers or reconfigure parts of the photo render path before the capture can be serviced efficiently. On iOS, this can especially matter for expensive captures such as maximum-resolution photos, RAW photos, depth data, flash captures, or qualityPrioritization set to 'quality'.

If you already know which CapturePhotoSettings you are going to use, call prepareSettings(...) before the actual capture:

await photoOutput.prepareSettings([
  { flashMode: 'off' }
])

const photo = await photoOutput.capturePhoto(
  { flashMode: 'off' },
  {}
)

Preparing settings is optional, but it allows VisionCamera to warm up the native photo pipeline for the exact settings you plan to capture with. On Android, prepareSettings(...) is a no-op.

Snapshot Capture

If photo capture is still too slow for your use-case, consider capturing a Snapshot via takeSnapshot() instead:

const preview = ...
const snapshot = await preview.takeSnapshot()

[!TIP] See "Snapshot Capture" for more information.

Appropriate Resolution

Choose resolutions efficiently. If your backend can only handle 1080p videos, don't use a 4k resolution if you have to downsize it later anyways - instead configure the Camera for 1080p already via a { resolutionBias: ... } constraint, or by setting a matching targetResolution on your outputs.

Binned Formats

Binned formats combine multiple neighboring sensor pixels into one larger effective pixel. This usually improves low-light sensitivity and reduces noise. Additionally, binned formats are more performant as they use significantly less bandwidth. The tradeoff is that binned formats can reduce fine detail compared to a full-resolution non-binned readout.

If performance is more important than maximum spatial detail, prefer a binned format with a { binned: true } constraint. If fine detail is more important, prefer a non-binned format with a { binned: false } constraint.

[!TIP] See "Constraints API" for more information.

Appropriate FPS

Same as with resolutions, also record at the frame rate you expect. Setting higher frame rate uses more memory and could heat up the battery. If your backend can only handle 30 FPS, there is no need to record at 60 FPS, instead set the Camera's FPS to 30 via an { fps: ... } constraint.

[!TIP] See "FPS" for more information.