doc/rs400/d401_dual_rgb_gmsl.md
The D401 GMSL builds its color image from its two OV9782 global-shutter stereo imagers, which are shared between depth and color. Each imager can, at any given moment, produce either a mono infrared image or a Bayer color image, never both at once. Because of this, there are two ways to obtain color from the D401 GMSL:
| Mode | Where color is produced | Color streams | Runs together with infrared? |
|---|---|---|---|
| ISP color (legacy) | Firmware ISP (YUYV, converted on the host) | one Color | Yes, with Infrared 1, Infrared 2, Depth |
| Raw dual-RGB | Host-side debayer of raw Bayer (RAW8/BA81) | Color + Color 1 | No, both imagers are producing Bayer |
Firmware requirement. Raw dual-RGB is available only on firmware 5.17.4.13 or newer. On older firmware the camera exposes the ISP color path only; requesting dual-RGB (a second color stream) will fail to resolve, and the Viewer shows a single ISP color stream.
The firmware ISP produces a single processed color stream (YUYV), which the SDK converts to the standard color formats. This is the original D401 GMSL color behavior and it coexists with Depth, Infrared 1, and Infrared 2. Use ISP color when you want color alongside infrared.
Both imagers stream raw Bayer (RAW8, fourcc BA81). The SDK debayers and color-processes each imager on the host and exposes two color streams: Color (index 0, left imager) and Color 1 (index 1, right imager). Because both imagers are producing Bayer color, infrared is not available in this mode. The Viewer additionally offers a stereo-rectification filter (on by default) that aligns the two color streams.
| Stream | Formats | Notes |
|---|---|---|
Depth | Z16 | Independent, always available |
Infrared 1 | Y8, Y16 | Left imager (mono) |
Infrared 2 | Y8, Y16 | Right imager (mono) |
Color (index 0) | RGB8, BGR8, RGBA8, BGRA8, YUYV | Left imager, ISP color on its own; switches to raw when Color 1 is also enabled |
Color 1 (index 1) | RGB8 | Right imager, raw dual-RGB only |
The second color stream (Color 1) selects the mode: enable Color 1 for raw dual-RGB. A single Color stream, in any format, including RGB8, is ISP color and coexists with infrared.
Resolutions (shared across depth, infrared, and color): 1280x720, 848x480, 640x480, 640x360, 480x270, 424x240.
Because the imagers are shared, only certain combinations can stream together. Infrared can run with a single (ISP) Color stream in any format; raw dual-RGB (Color + Color 1) consumes both imagers, so infrared is not available there. Depth is independent and can be added to any row.
| Use case | Depth | Infrared 1 | Infrared 2 | Color | Color 1 |
|---|---|---|---|---|---|
| Depth + stereo IR | Z16 | Y8/Y16 | Y8/Y16 | — | — |
| Depth + IR + ISP color | Z16 | Y8/Y16 | Y8/Y16 | RGB8/BGR8/YUYV/RGBA8/BGRA8 | — |
| ISP color only (any format) | Z16 (opt) | — | — | RGB8/BGR8/YUYV/RGBA8/BGRA8 | — |
| Raw dual-RGB (+ optional depth) | Z16 (opt) | — | — | RGB8 | RGB8 |
| Infrared only | — | Y8/Y16 | Y8/Y16 | — | — |
Raw dual-RGB, both color streams:
rs2::config cfg;
cfg.enable_stream(RS2_STREAM_COLOR, 0, 848, 480, RS2_FORMAT_RGB8, 30); // Color (left imager)
cfg.enable_stream(RS2_STREAM_COLOR, 1, 848, 480, RS2_FORMAT_RGB8, 30); // Color 1 (right imager)
// Depth may be added; infrared cannot run in this mode.
rs2::pipeline pipe;
pipe.start(cfg);
ISP color together with depth and infrared:
rs2::config cfg;
cfg.enable_stream(RS2_STREAM_DEPTH, 848, 480, RS2_FORMAT_Z16, 30);
cfg.enable_stream(RS2_STREAM_INFRARED, 1, 848, 480, RS2_FORMAT_Y8, 30);
cfg.enable_stream(RS2_STREAM_INFRARED, 2, 848, 480, RS2_FORMAT_Y8, 30);
cfg.enable_stream(RS2_STREAM_COLOR, 848, 480, RS2_FORMAT_BGR8, 30); // ISP color
rs2::pipeline pipe;
pipe.start(cfg);
A single
Colorstream is ISP regardless of format,RGB8on its own is ISP color and runs with infrared. Raw dual-RGB is selected by also enablingColor 1.On firmware older than 5.17.4.13, enabling
Color 1throwsCouldn't resolve requests, since raw dual-RGB is not exposed. ISP color (a singleColorstream) continues to work.
In Stereo Module → Available Streams you will see Depth, Infrared 1, Infrared 2, Color, and (on supported firmware) Color 1. The Color 1 checkbox chooses the mode:
Color 1 off → ISP color. Color streams in any format (including RGB8) alongside Infrared 1/2.Color 1 → raw dual-RGB. Infrared 1/2 grey out and Color switches to RGB8.The Viewer greys out any stream that cannot run in the currently selected mode, so it is not possible to pick an unstreamable combination. See the two mode screenshots above.
On the D401 GMSL all controls are exposed on a single Stereo Module sensor (depth, infrared, and color are folded into it). Which controls affect the image depends on the mode:
White Balance / Auto White Balance controls do not apply. When streaming color only (depth off), color exposure may be high; streaming depth alongside color lets the depth exposure govern the shared imager.Color 1 uses the infrared imager. Keep to a single Color stream (do not enable Color 1) if you need color and infrared together.Y8 / Y16 only).