Jetson Inference: Video Streaming - Jetson Inference

| | Jetson Inference

DNN Vision Library |

Video Streaming Utilities Library (jetson-utils)

videoSource and videoOutput APIs for input and output video streams. More...

Classes

| | struct | URI | | | Resource URI of a video device, IP stream, or file/directory. More...
| | | | struct | videoOptions | | | The videoOptions struct contains common settings that are used to configure and query videoSource and videoOutput streams. More...
| | | | class | videoOutput | | | The videoOutput API is for rendering and transmitting frames to video input devices such as display windows, broadcasting RTP network streams to remote hosts over UDP/IP, and saving videos/images/directories to disk. More...
| | | | class | videoSource | | | The videoSource API is for capturing frames from video input devices such as MIPI CSI cameras, V4L2 cameras, video/images files from disk, directories containing a sequence of images, and from RTP/RTSP network video streams over UDP/IP. More...
| | |

Macros

Detailed Description

videoSource and videoOutput APIs for input and output video streams.

Class Documentation

◆URI

| struct URI |

Resource URI of a video device, IP stream, or file/directory.

The URI object is used by videoSource, videoOutput, and videoOptions to identify which resource is being streamed. It will parse a string into protocol, path, and port/extension components.

URI protocols for videoSource input streams include MIPI CSI cameras (csi://), V4L2 cameras (v4l2://), RTP/RTSP networking streams (rtp:// and rtsp://), and disk-based videos/images/directories (file://)

- `csi://0` for MIPI CSI cameras, where `0` can be replaced with the camera port.
   It is also assumed that if a number with no protocol is specified (e.g. `"0"`),
   this means to use the MIPI CSI camera of that number (`"0"` -> `csi://0`)

- `v4l2:///dev/video0` for V4L2 cameras, where `/dev/video0` can be replaced with
   a different video device (e.g. `v4l2:///dev/video1` for V4L2 video device `1`).
   If no protocol is specified but the string begins with `/dev/video`, then it is
   assumed that the protocol is V4L2 (`/dev/video0` -> `v4l2:///dev/video0`)

- `rtp://@:1234` to recieve an RTP network stream, where `1234` is the port and `@` is shorthand
   for localhost. `@` can also be substituted for the IP address of a multicast group.
   Note that it is important to manually specify the codec and width/height when using RTP,
   as these values cannot be discovered from the RTP stream itself and need to be provided.
   @see videoOptions for more info about `--input-codec`, `--input-width`, and `--input-height`.

- `rtsp://username:password@<remote-host>:1234` to subscribe to an RTSP network stream, where
   `<remote-host>` should be substituted for the remote host's IP address or hostname, and 
   `1234` is the port. For example, `rtsp://192.168.1.2:5000`. The `username` and `password` 
   are optional, and are only used for RTSP streams that require authentication.

- `file:///home/user/my_video.mp4` for disk-based videos, images, and directories of images.
   You can leave off the `file://` protocol identifier and it will be deduced from the path.
   It can be a relative or absolute path. If a directory is specified that contains images,
   those images will be loaded in sequence (sorted alphanumerically). The path can also
   contain wildcard characters, for example `"images/*.jpg"` - however when using wildcards
   from the command line, enclose the string in quotes otherwise the OS will pre-expand them.
   Supported video formats for loading include MKV, MP4, AVI, and FLV. Supported codecs for 
   decoding include H.264, H.265, VP8, VP9, MPEG-2, MPEG-4, and MJPEG. Supported image formats
   for loading include JPG, PNG, TGA, BMP, GIF, PSD, HDR, PIC, and PNM (PPM/PGM binary).

URI protocols for videoOutput streams include rendering to displays (display://), broadcasting RTP/RTSP streams (rtp://, rtsp://), WebRTC streams (webrtc://), and saving videos/images to disk (file://)

- `display://0` for rendering to display using OpenGL, where `0` corresponds to the display number.
   By default, an OpenGL window will be created, unless the `--headless` command line option is used.

- `rtp://<remote-host>:1234` to broadcast a compressed RTP stream to a remote host, where you should
   substitute `<remote-host>` with the remote host's IP address or hostname, and `1234` is the port.

- `rtsp://@:8554/my_stream` to serve a compressed RTSP stream at the specified port and stream path.
   RTSP clients can connect to the stream at the specified path. Using this will create a RTSP server
   that can handle multiple videoOutput streams on the same port but with different paths
   (e.g. `rtsp://<hostname>:8554/my_stream_1`, `rtsp://<hostname>:8554/my_stream_2`, ect)

- `webrtc://@:1234/my_stream` to serve a compressed WebRTC stream at the specified port and path
   that browsers can connect to and view. Users will be able to navigate their browser to
   `http://<hostname>:1234/my_stream` and view a rudimentary video player that plays the stream.
   More advanced web front-ends can be created by using standard client-side Javascript WebRTC APIs.

- `file:///home/user/my_video.mp4` for saving videos, images, and directories of images to disk.
   You can leave off the `file://` protocol identifier and it will be deduced from the path.
   It can be a relative or absolute path. You can output a sequence of images using a path of
   the form `my_dir/image_%i.jpg` (where `%i` can include printf-style modifiers like `%03i`).  
   The `%i` will be replaced with the image number in the sequence. If just a directory is
   specified, then by default it will create a sequence of the form `%i.jpg` in that directory.
   Supported video formats for saving include MKV, MP4, AVI, and FLV. Supported codecs for 
   encoding include H.264, H.265, VP8, VP9, and MJPEG. Supported image formats for saving 
   include JPG, PNG, TGA, and BMP.

The URI strings used should take one of the above forms for input/output streams to be parsed correctly.

Public Member Functions

Public Attributes

Constructor & Destructor Documentation

◆URI() [1/2]

| URI::URI | ( | | ) | |

Default constructor.

◆URI() [2/2]

| URI::URI | ( | const char * | uri | ) | |

Construct a new URI from the given resource string.

See alsothe documentation above for valid string formats.

Member Function Documentation

◆c_str()

| const char* URI::c_str | ( | | ) | const |

| inline |

Cast to C-style string (const char*)

◆operator const char *()

| URI::operator const char * | ( | | ) | const |

| inline |

Cast to C-style string (const char*)

◆operator std::string()

| URI::operator std::string | ( | | ) | const |

| inline |

Cast to std::string

◆operator=() [1/2]

| void URI::operator= | ( | const char * | uri | ) | |

| inline |

Assignment operator (parse URI string)

◆operator=() [2/2]

| void URI::operator= | ( | const std::string & | uri | ) | |

| inline |

Assignment operator (parse URI string)

◆Parse()

| bool URI::Parse | ( | const char * | uri | ) | |

Parse the URI from the given resource string.

See alsothe documentation above for valid string formats.

◆Print()

Log the URI, with an optional prefix label.

Member Data Documentation

◆extension

| std::string URI::extension |

File extension (for files only, otherwise empty)

◆location

| std::string URI::location |

Path, IP address, or device name.

◆path

| std::string URI::path |

Path (for a network URI this comes after the port)

◆port

| int URI::port |

IP port, camera port, ect.

◆protocol

| std::string URI::protocol |

Protocol string (e.g.

file, csi, v4l2, rtp, ect)

◆string

| std::string URI::string |

Full resource URI (what was originally parsed)

◆videoOptions

| struct videoOptions |

The videoOptions struct contains common settings that are used to configure and query videoSource and videoOutput streams.

Public Types

| | enum | DeviceType {
DEVICE_DEFAULT = 0, DEVICE_V4L2, DEVICE_CSI, DEVICE_IP,
DEVICE_FILE, DEVICE_DISPLAY
} | | | Device interface types. More...
| | | | enum | IoType { INPUT = 0, OUTPUT } | | | Input/Output stream type. More...
| | | | enum | FlipMethod {
FLIP_NONE = 0, FLIP_COUNTERCLOCKWISE, FLIP_ROTATE_180, FLIP_CLOCKWISE,
FLIP_HORIZONTAL, FLIP_UPPER_RIGHT_DIAGONAL, FLIP_VERTICAL, FLIP_UPPER_LEFT_DIAGONAL,
FLIP_DEFAULT = FLIP_NONE
} | | | Settings of the flip method used by MIPI CSI cameras and compressed video inputs. More...
| | | | enum | Codec {
CODEC_UNKNOWN = 0, CODEC_RAW, CODEC_H264, CODEC_H265,
CODEC_VP8, CODEC_VP9, CODEC_MPEG2, CODEC_MPEG4,
CODEC_MJPEG
} | | | Video codecs. More...
| | | | enum | CodecType {
CODEC_CPU = 0, CODEC_OMX, CODEC_V4L2, CODEC_NVENC,
CODEC_NVDEC
} | | | Video codec engines. More...
| | | |

Public Member Functions

Static Public Member Functions

Public Attributes

Member Enumeration Documentation

◆Codec

| enum videoOptions::Codec |

Video codecs.

Enumerator
CODEC_UNKNOWN

Unknown/unsupported codec.

| | CODEC_RAW |

Uncompressed (e.g.

RGB)

| | CODEC_H264 |

H.264.

| | CODEC_H265 |

H.265.

| | CODEC_VP8 |

VP8.

| | CODEC_VP9 |

VP9.

| | CODEC_MPEG2 |

MPEG2 (decode only)

| | CODEC_MPEG4 |

MPEG4 (decode only)

| | CODEC_MJPEG |

MJPEG.

◆CodecType

| enum videoOptions::CodecType |

Video codec engines.

Enumerator
CODEC_CPU

CPU-based implementation using libav (e.g.

avdec_h264 / x264enc)

| | CODEC_OMX |

aarch64 & JetPack 4 only - OMX hardware plugins (e.g.

omxh264dec/omxh264enc)

| | CODEC_V4L2 |

aarch64 & JetPack 5 only - V4L2 hardware plugins (e.g.

nvv4l2decoder/nvv4l2h264enc)

| | CODEC_NVENC |

x86 only - NVENC hardware plugin (not currently implemented)

| | CODEC_NVDEC |

x86 only - NVDEC hardware plugin (not currently implemented)

◆DeviceType

| enum videoOptions::DeviceType |

Device interface types.

Enumerator
DEVICE_DEFAULT

Unknown interface type.

| | DEVICE_V4L2 |

V4L2 webcam (e.g.

/dev/video0)

| | DEVICE_CSI |

MIPI CSI camera.

| | DEVICE_IP |

IP-based network stream (e.g.

RTP/RTSP)

| | DEVICE_FILE |

Disk-based stream from a file or directory of files.

| | DEVICE_DISPLAY |

OpenGL output stream rendered to an attached display.

◆FlipMethod

| enum videoOptions::FlipMethod |

Settings of the flip method used by MIPI CSI cameras and compressed video inputs.

Enumerator
FLIP_NONE

Identity (no rotation)

| | FLIP_COUNTERCLOCKWISE |

Rotate counter-clockwise 90 degrees.

| | FLIP_ROTATE_180 |

Rotate 180 degrees.

| | FLIP_CLOCKWISE |

Rotate clockwise 90 degrees.

| | FLIP_HORIZONTAL |

Flip horizontally.

| | FLIP_UPPER_RIGHT_DIAGONAL |

Flip across upper right/lower left diagonal.

| | FLIP_VERTICAL |

Flip vertically.

| | FLIP_UPPER_LEFT_DIAGONAL |

Flip across upper left/lower right diagonal.

| | FLIP_DEFAULT |

Default setting (none)

◆IoType

| enum videoOptions::IoType |

Input/Output stream type.

Enumerator
INPUT

Input stream (e.g.

camera, video/image file, ect.)

| | OUTPUT |

Output stream (e.g.

display, video/image file, ect.)

Constructor & Destructor Documentation

◆videoOptions()

| videoOptions::videoOptions | ( | | ) | |

Constructor using default options.

Member Function Documentation

◆CodecFromStr()

| static Codec videoOptions::CodecFromStr | ( | const char * | str | ) | |

| static |

Parse a Codec enum from a string.

◆CodecToStr()

| static |

Convert a Codec enum to a string.

◆CodecTypeFromStr()

| static CodecType videoOptions::CodecTypeFromStr | ( | const char * | str | ) | |

| static |

Parse a Codec enum from a string.

◆CodecTypeToStr()

| static |

Convert a CodecType enum to a string.

◆DeviceTypeFromStr()

| static DeviceType videoOptions::DeviceTypeFromStr | ( | const char * | str | ) | |

| static |

Parse a DeviceType enum from a string.

◆DeviceTypeToStr()

| static |

Convert a DeviceType enum to a string.

◆FlipMethodFromStr()

| static FlipMethod videoOptions::FlipMethodFromStr | ( | const char * | str | ) | |

| static |

Parse a FlipMethod enum from a string.

◆FlipMethodToStr()

| static |

Convert a FlipMethod enum to a string.

◆IoTypeFromStr()

| static IoType videoOptions::IoTypeFromStr | ( | const char * | str | ) | |

| static |

Parse an IoType enum from a string.

◆IoTypeToStr()

| static |

Convert an IoType enum to a string.

◆Parse() [1/4]

◆Parse() [2/4]

◆Parse() [3/4]

◆Parse() [4/4]

◆Print()

Log the video settings, with an optional prefix label.

Member Data Documentation

◆bitRate

| uint32_t videoOptions::bitRate |

The encoding bitrate for compressed streams (only applies to video codecs like H264/H265).

For videoOutput streams, this option can be set from the command line using --bitrate=N.

Notethe default bitrate for encoding output streams is 4Mbps (target VBR).

◆codec

| Codec videoOptions::codec |

Indicates the codec used by the stream.

This is only really applicable to compressed streams, otherwise it will be CODEC_RAW.

videoSource input streams will attempt to discover the codec type (i.e. from video file), however RTP streams need this to be explitly set using the --input-codec=xyz option (where xyz is a string like h264, h265, vp8, vp9, mpeg2, mpeg4, or mjpeg).

A compressed videoOutput stream will default to H.264 encoding, but can be set using the --output-codec=xyz command line option (same values for xyz as above).

◆codecType

| CodecType videoOptions::codecType |

Indicates the underlying hardware/software engine used by the codec.

For input streams, this can be set with --decode=cpu or --decode=v4l2 for example. For output streams, this can be set with --encode=cpu or --encode=v4l2 for example. The default setting is to use hardware-acceleration on Jetson (aarch64) and CPU on x86.

◆deviceType

| DeviceType videoOptions::deviceType |

Indicates the type of device interface used by this stream.

◆flipMethod

| FlipMethod videoOptions::flipMethod |

The flip method controls if and how an input frame is flipped/rotated in pre-processing from a MIPI CSI camera or compressed video input.

Other types of streams will ignore this.

This option can be set from the command line using --flip-method=xyz, where xyz is one of the strings below:

none (Identity, no rotation)
counterclockwise (Rotate counter-clockwise 90 degrees)
rotate-180 (Rotate 180 degrees)
clockwise (Rotate clockwise 90 degrees)
horizontal-flip (Flip horizontally)
vertical-flip (Flip vertically)
upper-right-diagonal (Flip across upper right/lower left diagonal)
upper-left-diagonal (Flip across upper left/lower right diagonal)

◆frameCount

| uint64_t videoOptions::frameCount |

The number of frames that have been captured or output on this interface.

◆frameRate

| float videoOptions::frameRate |

The framerate of the stream (the default is 30Hz).

This option can be set from the command line using --input-rate=N or --output-rate=N for input and output streams, respectively. The --framerate=N option sets it for both.

◆height

| uint32_t videoOptions::height |

The height of the stream (in pixels).

This option can be set from the command line using --input-height=N for videoSource streams, or --output-height=N for videoOutput streams.

◆ioType

| IoType videoOptions::ioType |

Indicates if this stream is an input or an output.

◆latency

| int videoOptions::latency |

Number of milliseconds of video to buffer for network RTSP or WebRTC streams.

The default setting is 10ms (which is lower than GStreamer's default settings). If you have connection/buffering problems, try increasing the latency setting. It can be set from the command line using --input-latency=N or --output-latency=N

◆loop

| int videoOptions::loop |

Control the number of loops for videoSource disk-based inputs (for example, the number of times that a video should loop).

Other types of streams will ignore it.

The following values are are valid:

-1 = loop forever 0 = don't loop >0 = set number of loops

This option can be set from the command line using --loop=N.

Noteby default, looping is disabled (set to 0).

◆numBuffers

| uint32_t videoOptions::numBuffers |

The number of ring buffers used for threading.

This option can be set from the command line using --num-buffers=N.

Notethe default number of ring buffers is 4.

◆resource

| URI videoOptions::resource |

The resource URI of the device, IP stream, or file/directory.

See alsoURI for details about accepted protocols and URI formats.

◆save

| URI videoOptions::save |

Optional path to save the compressed stream to a video file on disk, which is to be used in addition to the primary resource URI above.

This option can be set from the command-line using --input-save for videoSource streams, or --output-save for videoOutput streams.

◆sslCert

| std::string videoOptions::sslCert |

Path to a file containing a PEM-encoded SSL/TLS certificate.

This is used for enabling HTTPS in the WebRTC server. It can be set from the command-line using the --ssl-cert or --https-cert options. You can make your own self-signed certificate by running a command like: openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365

◆sslKey

| std::string videoOptions::sslKey |

Path to a file containing a PEM-encoded private key.

This is used for enabling HTTPS in the WebRTC server. It can be set from the command-line using the --ssl-key or --https-key options. You can make your own self-signed certificate by running a command like: openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365

◆stunServer

| std::string videoOptions::stunServer |

URL of STUN server used for WebRTC.

This can be set using the --stun-server command-line argument. STUN servers are used during ICE/NAT and allow a local device to determine its public IP address. If this is left blank and WebRTC is used, then a default STUN server will be assigned.

◆width

| uint32_t videoOptions::width |

The width of the stream (in pixels).

This option can be set from the command line using --input-width=N for videoSource streams, or --output-width=N for videoOutput streams.

◆zeroCopy

| bool videoOptions::zeroCopy |

If true, indicates the buffers are allocated in zeroCopy memory that is mapped to both the CPU and GPU.

Otherwise, the buffers are only accessible from the GPU.

Notethe default is true (zeroCopy CPU/GPU access enabled).

◆videoOutput

| class videoOutput |

The videoOutput API is for rendering and transmitting frames to video input devices such as display windows, broadcasting RTP network streams to remote hosts over UDP/IP, and saving videos/images/directories to disk.

videoOutput interfaces are implemented by glDisplay, gstEncoder, and imageWriter.
The specific implementation is selected at runtime based on the type of resource URI. An instance can have multiple sub-streams, for example simultaneously outputting to a display and encoded video on disk or RTP stream.

videoOutput supports the following protocols and resource URI's:

- `display://0` for rendering to display using OpenGL, where `0` corresponds to the display number.
   By default, an OpenGL window will be created, unless the `--headless` command line option is used.

- `rtp://<remote-host>:1234` to broadcast a compressed RTP stream to a remote host, where you should
   substitute `<remote-host>` with the remote host's IP address or hostname, and `1234` is the port.

- `rtsp://@:8554/my_stream` to serve a compressed RTSP stream at the specified port and stream path.
   RTSP clients can connect to the stream at the specified path. Using this will create a RTSP server
   that can handle multiple videoOutput streams on the same port but with different paths
   (e.g. `rtsp://<hostname>:8554/my_stream_1`, `rtsp://<hostname>:8554/my_stream_2`, ect)

- `webrtc://@:1234/my_stream` to serve a compressed WebRTC stream at the specified port and path
   that browsers can connect to and view. Users will be able to navigate their browser to
   `http://<hostname>:1234/my_stream` and view a rudimentary video player that plays the stream.
   More advanced web front-ends can be created by using standard client-side Javascript WebRTC APIs.

- `file:///home/user/my_video.mp4` for saving videos, images, and directories of images to disk.
   You can leave off the `file://` protocol identifier and it will be deduced from the path.
   It can be a relative or absolute path. You can output a sequence of images using a path of
   the form `my_dir/image_%i.jpg` (where `%i` can include printf-style modifiers like `%03i`).  
   The `%i` will be replaced with the image number in the sequence. If just a directory is
   specified, then by default it will create a sequence of the form `%i.jpg` in that directory.
   Supported video formats for saving include MKV, MP4, AVI, and FLV. Supported codecs for 
   encoding include H.264, H.265, VP8, VP9, and MJPEG. Supported image formats for saving 
   include JPG, PNG, TGA, and BMP.

See alsoURI for info about resource URI formats. videoOptions for additional options and command-line arguments.

Inheritance diagram for videoOutput:

Public Member Functions

Static Public Member Functions

| | static videoOutput * | Create (const videoOptions &options) | | | Create videoOutput interface from a videoOptions struct that's already been filled out. More...
| | | | static videoOutput * | Create (const char *URI, const videoOptions &options=videoOptions()) | | | Create videoOutput interface from a resource URI string and optional videoOptions. More...
| | | | static videoOutput * | Create (const char *URI, const commandLine &cmdLine) | | | Create videoOutput interface from a resource URI string and parsing command line arguments. More...
| | | | static videoOutput * | Create (const char *URI, const int argc, char **argv) | | | Create videoOutput interface from a resource URI string and parsing command line arguments. More...
| | | | static videoOutput * | Create (const int argc, char **argv, int positionArg=-1) | | | Create videoOutput interface by parsing command line arguments, including the resource URI. More...
| | | | static videoOutput * | Create (const commandLine &cmdLine, int positionArg=-1) | | | Create videoOutput interface by parsing command line arguments, including the resource URI. More...
| | | | static videoOutput * | CreateNullOutput () | | | Create videoOutput interface that acts as a NULL output and does nothing with incoming frames. More...
| | | | static const char * | Usage () | | | Usage string for command line arguments to Create()More...
| | | | static const char * | TypeToStr (uint32_t type) | | | Convert a class type to a string. More...
| | | |

Protected Member Functions

| | | videoOutput (const videoOptions &options) | | | |

Protected Attributes

Constructor & Destructor Documentation

◆~videoOutput()

| virtual videoOutput::~videoOutput | ( | | ) | |

| virtual |

Destroy interface and release all resources.

◆videoOutput()

| protected |

Member Function Documentation

◆AddOutput()

| inline |

Add an output sub-stream.

When a frame is rendered to this stream, it will be rendered to each sub-stream.

◆Close()

| virtual void videoOutput::Close | ( | | ) | |

| virtual |

Stop streaming the device.

NoteClose() is automatically called by the videoOutput destructor when it gets deleted, so you do not explicitly need to call Close() before exiting the program if you delete your videoSource object.

Reimplemented in gstEncoder.

◆Create() [1/6]

| static videoOutput* videoOutput::Create | ( | const char * | URI, | | | | const commandLine & | cmdLine | | | ) | | |

| static |

Create videoOutput interface from a resource URI string and parsing command line arguments.

See alsovideoOptions for valid command-line arguments to be parsed. the documentation above and the URI struct for more info about resource URI's.

◆Create() [2/6]

| static |

Create videoOutput interface from a resource URI string and parsing command line arguments.

See alsovideoOptions for valid command-line arguments to be parsed. the documentation above and the URI struct for more info about resource URI's.

◆Create() [3/6]

| static videoOutput* videoOutput::Create | ( | const char * | URI, | | | | const videoOptions & | options = videoOptions() | | | ) | | |

| static |

Create videoOutput interface from a resource URI string and optional videoOptions.

See alsothe documentation above and the URI struct for more info about resource URI's.

◆Create() [4/6]

| static videoOutput* videoOutput::Create | ( | const commandLine & | cmdLine, | | | | int | positionArg = -1 | | | ) | | |

| static |

Create videoOutput interface by parsing command line arguments, including the resource URI.

Parameters

| positionArg | indicates the positional argument number in the command line of the resource URI (or -1 if a positional argument isn't used, and should instead be parsed from the --input= option). |

See alsovideoOptions for valid command-line arguments to be parsed. the documentation above and the URI struct for more info about resource URI's.

◆Create() [5/6]

| static |

Create videoOutput interface by parsing command line arguments, including the resource URI.

Parameters

See alsovideoOptions for valid command-line arguments to be parsed. the documentation above and the URI struct for more info about resource URI's.

◆Create() [6/6]

| static |

Create videoOutput interface from a videoOptions struct that's already been filled out.

It's expected that the supplied videoOptions already contain a valid resource URI.

◆CreateNullOutput()

| static videoOutput* videoOutput::CreateNullOutput | ( | | ) | |

| static |

Create videoOutput interface that acts as a NULL output and does nothing with incoming frames.

CreateNullOutput() can be used when there are no other outputs created and programs expect one to run.

◆GetFrameCount()

| uint64_t videoOutput::GetFrameCount | ( | | ) | const |

| inline |

Return the number of frames output.

◆GetFrameRate()

| float videoOutput::GetFrameRate | ( | | ) | const |

| inline |

Return the framerate, in Hz or FPS.

◆GetHeight()

| uint32_t videoOutput::GetHeight | ( | | ) | const |

| inline |

Return the height of the stream, in pixels.

◆GetNumOutputs()

| uint32_t videoOutput::GetNumOutputs | ( | | ) | const |

| inline |

Return the number of sub-streams.

◆GetOptions()

| const videoOptions& videoOutput::GetOptions | ( | | ) | const |

| inline |

Return the videoOptions of the stream.

◆GetOutput()

| inline |

Return a sub-stream.

◆GetResource()

| const URI& videoOutput::GetResource | ( | | ) | const |

| inline |

Return the resource URI of the stream.

◆GetType()

| virtual uint32_t videoOutput::GetType | ( | | ) | const |

| inlinevirtual |

Return the interface type of the stream.

This could be one of the following values:

Reimplemented in glDisplay, gstEncoder, and imageWriter.

◆GetWidth()

| uint32_t videoOutput::GetWidth | ( | | ) | const |

| inline |

Return the width of the stream, in pixels.

◆IsStreaming()

| bool videoOutput::IsStreaming | ( | | ) | const |

| inline |

Check if the device is actively streaming or not.

Returnstrue if the device is streaming (open), or false if it's closed or has reached EOS (End Of Stream).

◆IsType() [1/2]

template<typename T >

| bool videoOutput::IsType | ( | | ) | const |

| inline |

Check if a this stream is of a particular type.

Can be used with glDisplay, gstEncoder, and imageWriter. For example:

if( stream->IsType<glDisplay>() ) glDisplay* display = (glDisplay*)stream; // safe to cast

◆IsType() [2/2]

| inline |

Check if this stream is of a particular type.

◆Open()

| virtual bool videoOutput::Open | ( | | ) | |

| virtual |

Begin streaming the device.

After Open() is called, frames from the device can begin to be rendered.

Open() is not stricly necessary to call, if you call one of the Render() functions they will first check to make sure that the stream is opened, and if not they will open it automatically for you.

Returnstrue on success, false if an error occurred opening the stream.

Reimplemented in gstEncoder, and glDisplay.

◆Render() [1/2]

template<typename T >

| inline |

Render and output the next frame to the stream.

The image formats supported by this templated version of Render() include the following:

uchar3 (IMAGE_RGB8)
uchar4 (IMAGE_RGBA8)
float3 (IMAGE_RGB32F)
float4 (IMAGE_RGBA32F)

The image format will automatically be deduced from these types. If other types are used with this overload, a static compile-time error will be asserted.

Parameters

Returnstrue on success, false on error.

◆Render() [2/2]

| virtual |

Render and output the next frame to the stream.

The image formats supported by Render() are IMAGE_RGB8 (uchar3), IMAGE_RGBA8 (uchar4), IMAGE_RGB32F (float3), and IMAGE_RGBA32F (float4).

See alsoimageFormat for more info.Parameters

See alsoimageFormat) Returnstrue on success, false on error.

Reimplemented in glDisplay, gstEncoder, and imageWriter.

◆SetStatus()

| virtual void videoOutput::SetStatus | ( | const char * | str | ) | |

| virtual |

Set a status string (i.e.

status bar text on display window). Other types of interfaces may ignore the status text.

Reimplemented in glDisplay.

◆TypeToStr() [1/2]

| const char* videoOutput::TypeToStr | ( | | ) | const |

| inline |

Convert this stream's class type to string.

◆TypeToStr() [2/2]

| static |

Convert a class type to a string.

◆Usage()

| static const char* videoOutput::Usage | ( | | ) | |

| inlinestatic |

Usage string for command line arguments to Create()

Member Data Documentation

◆mOptions

| videoOptions videoOutput::mOptions |

| protected |

◆mOutputs

| std::vector<videoOutput*> videoOutput::mOutputs |

| protected |

◆mStreaming

| bool videoOutput::mStreaming |

| protected |

◆videoSource

| class videoSource |

The videoSource API is for capturing frames from video input devices such as MIPI CSI cameras, V4L2 cameras, video/images files from disk, directories containing a sequence of images, and from RTP/RTSP network video streams over UDP/IP.

videoSource interfaces are implemented by gstCamera, gstDecoder, and imageLoader. The specific implementation is selected at runtime based on the type of resource URI.

videoSource supports the following protocols and resource URI's:

- `csi://0` for MIPI CSI cameras, where `0` can be replaced with the camera port.
   It is also assumed that if a number with no protocol is specified (e.g. `"0"`),
   this means to use the MIPI CSI camera of that number (`"0"` -> `csi://0`).

- `v4l2:///dev/video0` for V4L2 cameras, where `/dev/video0` can be replaced with
   a different video device (e.g. `v4l2:///dev/video1` for V4L2 video device `1`).
   If no protocol is specified but the string begins with `/dev/video`, then it is
   assumed that the protocol is V4L2 (`/dev/video0` -> `v4l2:///dev/video0`)

- `rtp://@:1234` to recieve an RTP network stream, where `1234` is the port and `@` is shorthand
   for localhost. `@` can also be substituted for the IP address of a multicast group.
   Note that it is important to manually specify the codec of the stream when using RTP,
   as the codec cannot be discovered from the RTP stream itself and need to be provided.
   @see videoOptions for more info about the `--input-codec` option.

- `rtsp://username:password@<remote-host>:1234` to subscribe to an RTSP network stream, where
   `<remote-host>` should be substituted for the remote host's IP address or hostname, and 
   `1234` is the port. For example, `rtsp://192.168.1.2:5000`. The `username` and `password` 
   are optional, and are only used for RTSP streams that require authentication.

- `file:///home/user/my_video.mp4` for disk-based videos, images, and directories of images.
   You can leave off the `file://` protocol identifier and it will be deduced from the path.
   It can be a relative or absolute path. If a directory is specified that contains images,
   those images will be loaded in sequence (sorted alphanumerically). The path can also
   contain wildcard characters, for example `"images/*.jpg"` - however when using wildcards
   from the command line, enclose the string in quotes otherwise the OS will pre-expand them.
   Supported video formats for loading include MKV, MP4, AVI, and FLV. Supported codecs for 
   decoding include H.264, H.265, VP8, VP9, MPEG-2, MPEG-4, and MJPEG. Supported image formats
   for loading include JPG, PNG, TGA, BMP, GIF, PSD, HDR, PIC, and PNM (PPM/PGM binary).

See alsoURI for info about resource URI formats. videoOptions for additional options and command-line arguments.

Inheritance diagram for videoSource:

Public Types

| | enum | Status { ERROR = -2, EOS = -1, TIMEOUT = 0, OK = 1 } | | | Stream status codes that are optionally returned from Capture()More...
| | | |

Public Member Functions

Static Public Member Functions

| | static videoSource * | Create (const videoOptions &options) | | | Create videoSource interface from a videoOptions struct that's already been filled out. More...
| | | | static videoSource * | Create (const char *URI, const videoOptions &options=videoOptions()) | | | Create videoSource interface from a resource URI string and optional videoOptions. More...
| | | | static videoSource * | Create (const char *URI, const commandLine &cmdLine) | | | Create videoSource interface from a resource URI string and parsing command line arguments. More...
| | | | static videoSource * | Create (const char *URI, const int argc, char **argv) | | | Create videoSource interface from a resource URI string and parsing command line arguments. More...
| | | | static videoSource * | Create (const int argc, char **argv, int positionArg=-1) | | | Create videoSource interface by parsing command line arguments, including the resource URI. More...
| | | | static videoSource * | Create (const commandLine &cmdLine, int positionArg=-1) | | | Create videoSource interface by parsing command line arguments, including the resource URI. More...
| | | | static const char * | Usage () | | | Usage string for command line arguments to Create()More...
| | | | static const char * | TypeToStr (uint32_t type) | | | Convert a class type to a string. More...
| | | |

Static Public Attributes

Protected Member Functions

| | | videoSource (const videoOptions &options) | | | |

Protected Attributes

Member Enumeration Documentation

◆Status

| enum videoSource::Status |

Stream status codes that are optionally returned from Capture()

Enumerator
ERROR

an error occurred

| | EOS |

end-of-stream (EOS)

| | TIMEOUT |

a timeout occurred

| | OK |

frame capture successful

Constructor & Destructor Documentation

◆~videoSource()

| virtual videoSource::~videoSource | ( | | ) | |

| virtual |

Destroy interface and release all resources.

◆videoSource()

| protected |

Member Function Documentation

◆Capture() [1/3]

template<typename T >

| bool videoSource::Capture | ( | T ** | image, | | | | int * | status | | | ) | | |

| inline |

Capture the next image from the video stream, using the default timeout of 1000ms.

The image formats supported by this templated version of Capture() include the following:

uchar3 (IMAGE_RGB8)
uchar4 (IMAGE_RGBA8)
float3 (IMAGE_RGB32F)
float4 (IMAGE_RGBA32F)

The image format will automatically be deduced from these types. If other types are used with this overload, a static compile-time error will be asserted.

Parameters

See alsovideoSource::Status). -2 on ERROR, -1 on EOS, 0 on TIMEOUT, 1 on OK.Returnstrue if a frame was captured, false if there was an error or a timeout occurred.

◆Capture() [2/3]

template<typename T >

| bool videoSource::Capture | ( | T ** | image, | | | | uint64_t | timeout = DEFAULT_TIMEOUT, | | | | int * | status = NULL | | | ) | | |

| inline |

Capture the next image from the video stream.

The image formats supported by this templated version of Capture() include the following:

uchar3 (IMAGE_RGB8)
uchar4 (IMAGE_RGBA8)
float3 (IMAGE_RGB32F)
float4 (IMAGE_RGBA32F)

The image format will automatically be deduced from these types. If other types are used with this overload, a static compile-time error will be asserted.

Parameters

| [out] | image | output pointer that will be set to the memory containing the image. If this interface has it's videoOptions::zeroCopy flag set to true, the memory was allocated in mapped CPU/GPU memory and is be accessible from both CPU and CUDA. Otherwise, it's accessible only from CUDA. | | [in] | timeout | timeout in milliseconds to wait to capture the image before returning. The default is 1000ms. A timeout value of UINT64_MAX will wait forever. A timeout of 0 will return instantly if a frame wasn't immediately ready. | | [out] | status | optional status code returned ( |

See alsovideoSource::Status). -2 on ERROR, -1 on EOS, 0 on TIMEOUT, 1 on OK.Returnstrue if a frame was captured, false if there was an error or a timeout occurred.

◆Capture() [3/3]

| pure virtual |

Capture the next image from the video stream.

The image formats supported by Capture() are IMAGE_RGB8 (uchar3), IMAGE_RGBA8 (uchar4), IMAGE_RGB32F (float3), and IMAGE_RGBA32F (float4).

See alsoimageFormat for more info.Parameters

See alsovideoSource::Status). -2 on ERROR, -1 on EOS, 0 on TIMEOUT, 1 on OK.Returnstrue if a frame was captured, false if there was an error or a timeout occurred.

Implemented in gstCamera, gstDecoder, and imageLoader.

◆Close()

| virtual void videoSource::Close | ( | | ) | |

| virtual |

Stop streaming the device.

NoteClose() is automatically called by the videoSource destructor when it gets deleted, so you do not explicitly need to call Close() before exiting the program if you delete your videoSource object.

Reimplemented in gstCamera, gstDecoder, and imageLoader.

◆Create() [1/6]

| static videoSource* videoSource::Create | ( | const char * | URI, | | | | const commandLine & | cmdLine | | | ) | | |

| static |

Create videoSource interface from a resource URI string and parsing command line arguments.

See alsovideoOptions for valid command-line arguments to be parsed. the documentation above and the URI struct for more info about resource URI's.

◆Create() [2/6]

| static |

Create videoSource interface from a resource URI string and parsing command line arguments.

See alsovideoOptions for valid command-line arguments to be parsed. the documentation above and the URI struct for more info about resource URI's.

◆Create() [3/6]

| static videoSource* videoSource::Create | ( | const char * | URI, | | | | const videoOptions & | options = videoOptions() | | | ) | | |

| static |

Create videoSource interface from a resource URI string and optional videoOptions.

See alsothe documentation above and the URI struct for more info about resource URI's.

◆Create() [4/6]

| static videoSource* videoSource::Create | ( | const commandLine & | cmdLine, | | | | int | positionArg = -1 | | | ) | | |

| static |

Create videoSource interface by parsing command line arguments, including the resource URI.

Parameters

See alsovideoOptions for valid command-line arguments to be parsed. the documentation above and the URI struct for more info about resource URI's.

◆Create() [5/6]

| static |

Create videoSource interface by parsing command line arguments, including the resource URI.

Parameters

See alsovideoOptions for valid command-line arguments to be parsed. the documentation above and the URI struct for more info about resource URI's.

◆Create() [6/6]

| static |

Create videoSource interface from a videoOptions struct that's already been filled out.

It's expected that the supplied videoOptions already contain a valid resource URI.

◆GetFrameCount()

| uint64_t videoSource::GetFrameCount | ( | | ) | const |

| inline |

Return the number of frames captured.

◆GetFrameRate()

| uint32_t videoSource::GetFrameRate | ( | | ) | const |

| inline |

Return the framerate, in Hz or FPS.

◆GetHeight()

| uint32_t videoSource::GetHeight | ( | | ) | const |

| inline |

Return the height of the stream, in pixels.

◆GetLastTimestamp()

| uint64_t videoSource::GetLastTimestamp | ( | | ) | const |

| inline |

Get timestamp of the last captured frame, in nanoseconds.

◆GetOptions()

| const videoOptions& videoSource::GetOptions | ( | | ) | const |

| inline |

Return the videoOptions of the stream.

◆GetRawFormat()

| imageFormat videoSource::GetRawFormat | ( | | ) | const |

| inline |

Get raw image format.

◆GetResource()

| const URI& videoSource::GetResource | ( | | ) | const |

| inline |

Return the resource URI of the stream.

◆GetType()

| virtual uint32_t videoSource::GetType | ( | | ) | const |

| inlinevirtual |

Return the interface type of the stream.

This could be one of the following values:

Reimplemented in gstCamera, gstDecoder, and imageLoader.

◆GetWidth()

| uint32_t videoSource::GetWidth | ( | | ) | const |

| inline |

Return the width of the stream, in pixels.

◆IsStreaming()

| bool videoSource::IsStreaming | ( | | ) | const |

| inline |

Check if the device is actively streaming or not.

Returnstrue if the device is streaming (open), or false if it's closed or has reached EOS (End Of Stream).

◆IsType() [1/2]

template<typename T >

| bool videoSource::IsType | ( | | ) | const |

| inline |

Check if a this stream is of a particular type.

Can be used with gstCamera, gstDecoder, and imageLoader. For example:

if( stream->IsType<gstCamera>() ) gstCamera* camera = (gstCamera*)stream; // safe to cast

◆IsType() [2/2]

| inline |

Check if this stream is of a particular type.

◆Open()

| virtual bool videoSource::Open | ( | | ) | |

| virtual |

Begin streaming the device.

After Open() is called, frames from the device will begin to be captured.

Open() is not stricly necessary to call, if you call one of the Capture() functions they will first check to make sure that the stream is opened, and if not they will open it automatically for you.

Returnstrue on success, false if an error occurred opening the stream.

Reimplemented in gstCamera, gstDecoder, and imageLoader.

◆TypeToStr() [1/2]

| const char* videoSource::TypeToStr | ( | | ) | const |

| inline |

Convert this stream's class type to string.

◆TypeToStr() [2/2]

| static |

Convert a class type to a string.

◆Usage()

| static const char* videoSource::Usage | ( | | ) | |

| inlinestatic |

Usage string for command line arguments to Create()

Member Data Documentation

◆DEFAULT_TIMEOUT

| const uint64_t videoSource::DEFAULT_TIMEOUT =1000 |

| static |

The default Capture timeout (1000ms)

◆mLastTimestamp

| uint64_t videoSource::mLastTimestamp |

| protected |

◆mOptions

| videoOptions videoSource::mOptions |

| protected |

◆mRawFormat

| imageFormat videoSource::mRawFormat |

| protected |

◆mStreaming

| bool videoSource::mStreaming |

| protected |

Macro Definition Documentation

◆LOG_VIDEO

| #define LOG_VIDEO "[video] " |

◆VIDEO_OUTPUT_USAGE_STRING

| #define VIDEO_OUTPUT_USAGE_STRING |

Value:

"videoOutput arguments: \n" \

" output resource URI of the output stream, for example:\n" \

" * file://my_image.jpg (image file)\n" \

" * file://my_video.mp4 (video file)\n" \

" * file://my_directory/ (directory of images)\n" \

" * rtp://<remote-ip>:1234 (RTP stream)\n" \

" * rtsp://@:8554/my_stream (RTSP stream)\n" \

" * webrtc://@:1234/my_stream (WebRTC stream)\n" \

" * display://0 (OpenGL window)\n" \

" --output-codec=CODEC desired codec for compressed output streams:\n" \

" * h264 (default), h265\n" \

" * vp8, vp9\n" \

" * mpeg2, mpeg4\n" \

" * mjpeg\n" \

" --output-encoder=TYPE the encoder engine to use, one of these:\n" \

" * cpu\n" \

" * omx (aarch64/JetPack4 only)\n" \

" * v4l2 (aarch64/JetPack5 only)\n" \

" --output-save=FILE path to a video file for saving the compressed stream\n" \

" to disk, in addition to the primary output above\n" \

" --bitrate=BITRATE desired target VBR bitrate for compressed streams,\n" \

" in bits per second. The default is 4000000 (4 Mbps)\n" \

" --headless don't create a default OpenGL GUI window\n\n"

Standard command-line options able to be passed to videoOutput::Create()

◆VIDEO_SOURCE_USAGE_STRING

| #define VIDEO_SOURCE_USAGE_STRING |

Standard command-line options able to be passed to videoSource::Create()

Generated on Tue Mar 28 2023 14:27:58 for Jetson Inference by 1.8.17