wrappers/unrealengine4/README.md
Level with examples demonstrating streaming depth, color (IR and external RGB) and pointcloud in Unreal Engine 4.
UnrealEngine4 wrapper shows how to use RealSense features via Blueprints (UE4 version 4.19+). At the moment it provides next functionality:
Download and install RealSense SDK 2.0
Download wrapper and generate project files for RealSenseUE.uproject
Copy C:\Program Files (x86)\Intel RealSense SDK 2.0\bin\x64\realsense2.dll to either:
In case RealSense SDK was installed in custom folder:
string RealSenseDirectory = "REALSENSE_SDK_PATH_HERE"Build RealSenseUE.sln using DevelopmentEditor configuration
Start and play RealSenseUE.uproject to see some live data
Check out prefab BP_RealSenseAPI, it shows functionality exposed to Blueprints and a few examples:
Check out prefab BP_RealSenseInspector, it simplifies stream configuration and management, also binds live data to scene objects. BP_RealSenseAPI should be placed to scene as well if new one is being created.
Hardware inspector allows to check connected hardware and tune sensor options via editor. This feature is for debugging only and changes are not saved automatically. It's possible to load and save configuration presets using corresponding buttons in hardware inspector dropdown or via blueprint functions.
Preset files can be downloaded here.
Picture: hardware inspector
DeviceSerial allows to select specific device by serial. If empty default device will be used.
PipelineMode switches between CaptureOnly/Record/Playback:
EnablePolling switches poll/wait update mode. Poll mode is checking for data with specified rate, wait mode is blocking until data is available.
Picture: mode inspector
Each stream has it's own dropdown with enable/disable flag and video mode selector. These options should be configured before starting RealSenseInspector. Restarting is required to change video mode. Enable/disable flag can be used to pause/resume already running stream.
Depth can provide two textures: raw and colorized. Colorized depth is soft-processed based on Equalize/MinMax/Colormap settings. Avoid using colorized depth on high resolutions/rates.
In case running both depth and infrared they should be configured to same video mode.
Picture: depth stream inspector
Point cloud support is experimental and has a lot of flaws. Current implementation uses third-party plugin for dynamic mesh generation and should be optimized.
Picture: point cloud inspector
RealSenseContext is global object initialized at plugin load. It wraps native RealSense library and provides device management functions.
Picture: get context blueprint
After getting the context devices can be enumerated. Each device contains a few sensors, each sensor has some options. Sensor profile contains video mode and frame rate.
Picture: enumerate devices blueprint
Picture: enumerate sensors blueprint
Picture: enumerate options blueprint
Sensor options can be tuned one by one or using preset load/save functions.
Picture: get and set options value
Picture: save and load preset blueprint
RealSenseInspector provides stream configuration and management functions. It updates textures and PCL mesh.
Picture: find Inspector blueprint
Properties described in StreamConfiguration section also can be modified from Blueprints.
Picture: modify stream mode blueprint
After configuration RealSenseInspector should be started with any PipelineMode.
Picture: set mode blueprint
Time to get texture objects and PCL mesh.
Picture: get stream texture blueprint
Picture: transform PCL mesh blueprint
It's also possible to access wrapper from native code, check RealSenseTestActor.cpp for example:
#include "RealSensePlugin.h"
#include "RealSenseContext.h"
#include "RealSenseDevice.h"
#include "RealSenseNative.h"
ARealSenseTestActor::ARealSenseTestActor()
{
auto Context = IRealSensePlugin::Get().GetContext();
Context->QueryDevices();
for (auto Device : Context->Devices)
{
UE_LOG(LogRealSenseDemo, Display, TEXT("Wrapper device %s"), *(Device->Name));
}
rs2::context_ref RsContext(Context->GetHandle());
auto DeviceList = RsContext.query_devices();
for (auto Device : DeviceList)
{
FString DevName(ANSI_TO_TCHAR(Device.get_info(RS2_CAMERA_INFO_NAME)));
UE_LOG(LogRealSenseDemo, Display, TEXT("Native device %s"), *DevName);
}
}
This project is licensed under the Apache License, Version 2.0.