Skia Frame Processors

import { Tab, Tabs } from 'fumadocs-ui/components/tabs'

Skia Frame Processors allow using @Shopify/react-native-skia to draw on a Frame in realtime.

[!DEPENDENCY] Skia Frame Processors require react-native-vision-camera-skia to be installed.

Using Skia with VisionCamera

After installing @Shopify/react-native-skia and react-native-vision-camera-skia, you can use the <SkiaCamera /> view to quickly render a Camera feed using Skia:

import { SkiaCamera } from 'react-native-vision-camera-skia'

function App() {
  return (
    <SkiaCamera
      style={StyleSheet.absoluteFill}
      isActive={true}
      device="back"
      onFrame={(frame, render) => {
        'worklet'
        // ... custom Frame processing logic
        render(({ frameTexture, canvas }) => {
          // ... custom drawing operations
          canvas.drawImage(frameTexture, 0, 0)
        })
        frame.dispose()
      }}
    />
  )
}

In the onFrame(...) callback you receive two parameters:

• The frame (a Frame), which you can use for any kind of Frame Processing, like Face Detection.
• A render(...) function, which allows you to render the Frame (as an SkImage) to a Skia SkCanvas. The SkCanvas will be in the Frame's coordinate system, and will ultimately be rendered to the screen via the preview Skia <Canvas />.

Pixel Formats

Just like in a regular CameraFrameOutput, you can configure the <SkiaCamera />'s pixel format via pixelFormat. The pixel format affects your Skia Camera's performance, as well as the pixel format of the frame.

• Choose 'yuv' as a good default - Skia supports rendering 8-bit and 10-bit YUV buffers, and most native Frame Processing libraries support YUV.
• Choose 'rgb' as a fallback if 'yuv' is not supported by your native Frame Processor libraries, as 'rgb' requires conversion overhead.
• Choose 'native' only if you ensure the negotiated CameraSessionConfig's nativePixelFormat is renderable by Skia. As of today, Skia supports 8-bit and 10-bit YUV formats, RGB formats, as well as 'private' on Android.

[!TIP] See "The Frame Output: Choosing a Pixel Format" for more information.

Differences to a regular <Camera />

The <SkiaCamera /> is different than the <Camera /> in two important ways:

• A <SkiaCamera /> does not render to a PreviewView - instead it renders a Skia <Canvas /> and draws Frames manually, which you can control via render(...).
• A <SkiaCamera /> always has a CameraFrameOutput attached.

Rendering yourself

If you want to render Frames yourself, without <SkiaCamera />, use the NativeBuffer APIs and Skia's MakeImageFromNativeBuffer(...) API:

const frameOutput = useFrameOutput({
  pixelFormat: 'native',
  onFrame(frame) {
    'worklet'
    const surface = // create a Skia.Surface & cache it across onFrame(...)
    const nativeBuffer = frame.getNativeBuffer()
    const frameTexture = Skia.Image.MakeImageFromNativeBuffer(nativeBuffer.pointer)
    // render `frameTexture` to `surface`/canvas
    frameTexture.dispose()
    nativeBuffer.release()
    frame.dispose()
  }
})

[!TIP] See "A Frame's NativeBuffer" for more information about the NativeBuffer API.