Video manipulation - Remotion

import {VideoCanvasExamples} from '../components/GreenscreenExamples/index';

You can draw frames of a <OffthreadVideo>, <Video> or a <Html5Video> onto a <canvas> element using the drawImage() API.

:::note During preview, makes use of the requestVideoFrameCallback() API.
Browser support: Firefox 130 (August 2024), Chrome 83, Safari 15.4. :::

Basic example

In this example, an <OffthreadVideo> is rendered and made invisible.
Every frame that is emitted is drawn to a Canvas and a grayscale filter is applied.

tsx

import React, {useCallback, useEffect, useRef} from 'react';
import {AbsoluteFill, useVideoConfig, OffthreadVideo} from 'remotion';
// ---cut---
export const VideoOnCanvas: React.FC = () => {
  const video = useRef<HTMLVideoElement>(null);
  const canvas = useRef<HTMLCanvasElement>(null);
  const {width, height} = useVideoConfig();

  // Process a frame
  const onVideoFrame = useCallback(
    (frame: CanvasImageSource) => {
      if (!canvas.current) {
        return;
      }
      const context = canvas.current.getContext('2d');

      if (!context) {
        return;
      }

      context.filter = 'grayscale(100%)';
      context.drawImage(frame, 0, 0, width, height);
    },
    [height, width],
  );

  return (
    <AbsoluteFill>
      <AbsoluteFill>
        <OffthreadVideo
          // Hide the original video tag
          style={{opacity: 0}}
          onVideoFrame={onVideoFrame}
          src="https://remotion.media/BigBuckBunny.mp4"
        />
      </AbsoluteFill>
      <AbsoluteFill>
        <canvas ref={canvas} width={width} height={height} />
      </AbsoluteFill>
    </AbsoluteFill>
  );
};

Alternative: Effects

You can also apply effects to canvas-based components such as <Video> using @remotion/effects. For example, colorKey() removes a chosen color — see the Greenscreen page.

Before v4.0.190

Before v4.0.190, the onVideoFrame prop of <OffthreadVideo> and <Html5Video> was not supported.
You could only manipulate a <Html5Video> using the requestVideoFrameCallback API.
Click here to see the old version of this page.