Metadata

Streams are divided into Video and Motion.

Motion streams do not have metadata. They stand alone, are output at a high frequency, and need to be as efficient as possible. Their frequency makes it very hard to synchronize with other streams, and any delay generated by keeping them around until they can be synchronized would be detrimental.

Video streams use metadata to describe the image. Common metadata include per-frame exposure & brightness value, and may contain any information that's relevant to the generation or handling if the image. However, image data and metadata cannot be packaged together: the ROS2 Image format cannot carry additional information besides a timestamp. The metadata must be sent separately and somehow synchronized with the image data.

Metadata Topic

A single topic is used to communicate metadata to a client:

<device-topic-root>/metadata

This topic conveys metadata for all video streams.

Quality of Service

Reliability: BEST_EFFORT
Durability: VOLATILE

Format

Metadata uses flexible messages.

As such, metadata content is itself flexible and easily changed without predefined structures:

JSON

{
  "stream-name":"Color",
  "header":{"frame-number":1234, "timestamp":123456789, "timestamp-domain":0},
  "metadata":{"Exposure":123, "Gain":456}
}

The stream-name is the stream with which this metadata should be synchronized
The header contains important "non-metadata" metadata pertinent for proper client handling:
- frame-number supplied the sequential number of the frame - optional
- timestamp is critical and is the frame timestamp, same as was sent with the image - this is used to synchronize to the image
- timestamp-domain tells the client how to interpret the timestamp - optional (if missing, assuming it's a hardware timestamp)
The metadata contains the actual customizable values for metadata, as name-value mappings: the name is the string name of the metadata field, and the value its value

No requirements are set for the metadata content. However, some agreement is needed between the server and client if the data is to be made use of! For purposes of librealsense, the names must match those returned by rs2_frame_metadata_to_string and the values must be integral long long values.

Metadata that's missing will be marked not-there. Metadata names that're unrecognized will be ignored.

Send Order

It is recommended that images be sent first, then metadata: because the metadata is much smaller (encompassing even a single packet), it will likely arrive before the image transfer is complete.

Lost Messages

Since the topic is best-effort, it is possible that messages will get lost.

Or metadata may simply not get sent.

Images that cannot be synchronized to metadata (either because of non-matching timestamps or because metadata was lost) will simply have no metadata.

Without a frame-number, the client must assume frame-numbers. The timestamp domain may get lost, but video streaming will keep on working.