Back to Serial Studio

Data sources

doc/help/Data-Sources.md

4.0.120.3 KB
Original Source

Data sources

Overview

Serial Studio connects to hardware and software data sources through ten driver types. Three ship in the free GPL edition; seven more need a Pro license. Each driver feeds raw bytes into the frame-parsing pipeline. You pick the active driver in the Setup panel, or (for multi-device projects) in the Project Editor.

For protocol theory, wire framing, and educational primers on each driver, see the dedicated Drivers section: UART, Network, Bluetooth LE, Modbus, CAN Bus, Audio, Raw USB, HID, Process I/O, and MQTT Subscriber. The protocol vocabulary that the MQTT driver assumes (topics, wildcards, QoS, retained messages) lives in MQTT Topics & Semantics under Connectivity; the outbound side is the per-project MQTT Publisher.

The diagram below shows how each driver type feeds into the data pipeline.

mermaid
flowchart TD
    subgraph Free["Free"]
        F1["UART · TCP/UDP · BLE"]
    end
    subgraph Pro["Pro"]
        P1["Audio · Modbus · CAN
USB · HID · Process · MQTT"]
    end
    F1 --> FR["Frame Reader"]
    P1 --> FR
    FR --> FB["Frame Builder"]
    FB --> DASH["Dashboard"]

Free drivers

Serial port (UART)

Talks to physical or virtual serial ports. A good fit for Arduino, ESP32, STM32, any USB-to-serial adapter, RS-232, and RS-485 links.

Configuration:

ParameterOptions / rangeDefault
COM portAuto-detected list of available ports
Baud rate110 to 1,000,000+ (custom values accepted)9600
Data bits5, 6, 7, 88
ParityNone, Even, Odd, Space, MarkNone
Stop bits1, 1.5, 21
Flow controlNone, RTS/CTS (hardware), XON/XOFF (software)None
DTR signalOn / Off. Toggles the Data Terminal Ready lineOn
Auto reconnectOn / Off. Reconnects automatically on disconnectOff

Platform notes. Port names are platform-specific: COM3 on Windows, /dev/ttyUSB0 or /dev/ttyACM0 on Linux, /dev/cu.usbserial-* on macOS. You can register custom port paths manually.

Quick start:

  1. Select your COM port from the dropdown.
  2. Set the baud rate to match your device firmware.
  3. Adjust data bits, parity, stop bits, and flow control if your device needs non-default values.
  4. Turn on DTR if your board needs it for reset (common on some Arduino variants).
  5. Click Connect.

Network socket (TCP/UDP)

Talks over TCP or UDP sockets. Use this for WiFi-enabled microcontrollers (ESP32 in WiFi mode, Raspberry Pi), remote telemetry servers, or network-attached instruments.

Configuration:

ParameterTCPUDP
Socket typeTCPUDP
Remote addressIP or hostnameIP or hostname
PortTCP port (default: 23)Remote port (default: 53)
Local portListening port (0 = auto-assign)
MulticastOn / Off

Protocol differences:

  • TCP sets up a persistent, reliable connection. Data arrives in order, with automatic retransmission of lost packets. Hostname resolution runs asynchronously before the connection is opened.
  • UDP is connectionless, with lower latency and no delivery guarantee. It supports multicast group reception when enabled.

Quick start (TCP):

  1. Select TCP as the socket type.
  2. Enter the remote IP address and port.
  3. Click Connect. Serial Studio opens a TCP session and starts receiving data.

Quick start (UDP):

  1. Select UDP as the socket type.
  2. Enter the remote address and remote port where the device sends data.
  3. Set a local port if your device expects a specific listening port, or leave it at 0 for auto-assignment.
  4. Turn on multicast if you're receiving from a multicast group.
  5. Click Connect.

Bluetooth Low Energy (BLE)

Connects to BLE peripherals using GATT service/characteristic subscriptions. A good fit for BLE sensors, fitness devices, and custom BLE firmware.

Configuration:

ParameterDescription
DeviceDiscovered BLE peripherals (scanning starts automatically)
ServiceGATT service to subscribe to
CharacteristicSpecific characteristic for data notifications

Platform support: macOS (CoreBluetooth), Windows 10+ (WinRT), Linux (BlueZ 5+).

Architecture notes:

  • Device discovery state is shared across all BLE driver instances. The device list is append-only during scanning, so indices stay stable.
  • Each driver instance keeps its own connection (controller, service, characteristic subscriptions).
  • Service and characteristic names discovered on one instance are propagated to all other instances targeting the same device.

Quick start:

  1. Open the Setup panel. BLE scanning starts automatically.
  2. Pick a device from the dropdown.
  3. Wait for service discovery to finish, then pick a service.
  4. Pick the characteristic that carries your data.
  5. Click Connect.

Pro drivers

The next seven drivers need a Serial Studio Pro license.

Audio input

Captures audio samples from system input devices through the miniaudio library. Useful for microphone analysis, acoustic sensors, vibration monitoring via piezo elements, and analog signal visualization within the audio frequency range.

Configuration:

ParameterOptions
Input deviceSystem audio inputs (microphone, line-in, and so on)
Sample rateDevice-dependent (common: 8, 22.05, 44.1, 48, 96, 192 kHz)
Sample formatPCM unsigned 8-bit, signed 16/24/32-bit, float 32-bit
Channel configMono, stereo, or any layout the device supports
Output deviceSystem audio outputs (optional, for loopback)

Quick start:

  1. Select Audio Input as the data source.
  2. Choose an input device and sample rate.
  3. Set the sample format and channel configuration.
  4. Click Connect. Audio samples flow into the frame pipeline as CSV rows.

Tips. Use line-in instead of mic-in to avoid automatic gain control. Turn off OS-level noise cancellation and audio effects for clean signals. Grant microphone permissions if your OS asks for them.

Modbus (RTU and TCP)

Polls registers from Modbus-compatible PLCs, sensors, and industrial controllers. Supports both Modbus RTU (serial, RS-485/RS-232) and Modbus TCP.

Common configuration:

ParameterRange / options
ProtocolModbus RTU, Modbus TCP
Slave address1 to 247
Poll interval10 to 60,000 ms
Register groupsType + start address + count (multiple groups)

Register types: coil (read/write discrete), discrete input (read-only discrete), holding register (read/write 16-bit), input register (read-only 16-bit).

RTU-specific parameters. Serial port, baud rate, data bits, parity, stop bits. Identical to the UART driver settings.

TCP-specific parameters. Host address and TCP port (default: 5020).

Quick start (RTU):

  1. Connect an RS-485 adapter to the Modbus device and your computer.
  2. Select Modbus RTU. Configure the serial port and line parameters.
  3. Set the slave address and add one or more register groups.
  4. Set the poll interval.
  5. Click Connect. Register values are polled on a timer and delivered as frames.

Quick start (TCP):

  1. Enter the device IP address and port.
  2. Set the slave address (unit ID) and register groups.
  3. Click Connect.

CAN Bus

Receives and transmits CAN frames through platform-specific CAN interface plugins. Used for automotive ECU diagnostics, industrial control networks, and vehicle telemetry.

Configuration:

ParameterOptions
PluginSocketCAN, PEAK PCAN, Vector CAN, Systec, others
Interfacecan0, can1, PCAN_USBBUS1, and so on (plugin-dependent)
Bitrate10K to 1M (common: 125K, 250K, 500K, 1M)
CAN FDOn / Off. Enables flexible data-rate frames (up to 64 bytes)

Platform support:

  • Linux: SocketCAN is built into the kernel. Configure with ip link set can0 type can bitrate 500000 && ip link set can0 up.
  • Windows: Requires a third-party CAN adapter with a Qt CAN Bus plugin (PEAK, Vector, Systec).
  • macOS: Limited support. Requires third-party drivers.

Frame format. Standard CAN uses 11-bit identifiers (0x000 to 0x7FF); IDs above 0x7FF are sent as 29-bit extended frames. CAN FD raises the payload limit to 64 data bytes per frame (classic CAN caps at 8). The driver outputs standard frames as [ID_high, ID_low, DLC, data...] byte arrays; extended frames carry the full 29-bit identifier in four bytes with bit 7 of the first byte set ([0x80|ID_28..24, ID_23..16, ID_15..8, ID_7..0, DLC, data...]).

Quick start:

  1. Connect your CAN adapter and install its drivers.
  2. Select CAN Bus as the data source.
  3. Pick the plugin that matches your hardware and select the interface.
  4. Set the bitrate to match your CAN network exactly. Mismatched bitrates cause bus errors.
  5. Turn on CAN FD if your network uses it.
  6. Click Connect.

Raw USB

Direct USB access via libusb, bypassing OS serial and HID abstractions. Supports bulk, control, and isochronous transfers for custom USB devices and high-bandwidth sensors.

Configuration:

ParameterOptions
DeviceEnumerated USB devices (VID:PID, product name)
Transfer modeBulk Stream (default), Advanced Control, Isochronous
IN endpointUSB endpoint to read from
OUT endpointUSB endpoint to write to
ISO packet sizePacket size in bytes (isochronous mode only)

Transfer modes:

  • Bulk stream. Standard bulk IN/OUT transfers. Best for most custom USB firmware.
  • Advanced control. Bulk plus control transfers, for devices that need vendor-specific USB control commands.
  • Isochronous. Time-sensitive, fixed-rate streaming transfers. Use this for real-time audio, video, or other isochronous devices.

Device enumeration. Uses libusb hotplug callbacks where the OS supports them (Linux, macOS, Windows). Falls back to a 2-second polling timer when hotplug isn't available.

Quick start:

  1. Plug in your USB device.
  2. Select Raw USB as the data source.
  3. Pick your device from the enumerated list.
  4. Select Bulk Stream as the transfer mode (unless your device specifically needs something else).
  5. Select the IN and OUT endpoints that match your device firmware.
  6. Click Connect.

Platform notes:

  • Linux: You may need udev rules (for example SUBSYSTEM=="usb", ATTR{idVendor}=="1234", MODE="0666") or root access.
  • macOS: The kernel HID or serial driver may have to be detached before libusb can claim the device.
  • Windows: A WinUSB or libusb-compatible driver must be installed (for example via Zadig).

HID device

Connects to Human Interface Devices (gamepads, joysticks, custom HID sensors) through hidapi. Works on Windows, macOS, and Linux without extra drivers for most devices.

Configuration:

ParameterDescription
DeviceEnumerated HID devices (VID:PID, product name)
Usage pageRead-only. HID Usage Page reported by the device
UsageRead-only. HID Usage reported by the device

Enumeration. The device list refreshes automatically every 2 seconds. The list includes a placeholder entry at index 0 ("Select Device").

Data format. The driver reads 65-byte HID reports directly from the device. Unexpected disconnections (unplugging, driver errors) are handled cleanly, and the UI is notified.

Quick start:

  1. Plug in your HID device.
  2. Select HID Device as the data source.
  3. Pick your device from the list (check Usage Page and Usage to confirm the right interface on multi-interface devices).
  4. Click Connect.

Platform notes:

  • Linux: Add a udev rule for hidraw access (for example SUBSYSTEM=="hidraw", GROUP="plugdev", MODE="0664").
  • macOS and Windows: most HID devices work without extra configuration.

Process I/O

Reads data from a child process's stdout or from a named pipe/FIFO. Any script or program that writes to stdout can feed data into Serial Studio dashboards this way.

Modes:

ModeDescription
LaunchSpawns a child process and reads its combined stdout and stderr
Named pipeOpens an existing named pipe or FIFO for reading

Launch mode parameters:

ParameterDescription
ExecutablePath to the program to launch
ArgumentsCommand-line arguments
Working dirWorking directory for the child process

Named pipe parameters:

ParameterDescription
Pipe pathPath to the FIFO or named pipe (e.g., /tmp/mydata or \\.\pipe\mydata)

Quick start (Launch mode):

  1. Select Process I/O as the data source.
  2. Pick Launch mode.
  3. Enter the path to your script or executable.
  4. Add command-line arguments if needed.
  5. Click Connect. Serial Studio spawns the process and reads its stdout.

Quick start (Named Pipe mode):

  1. Create a FIFO or named pipe from your external process.
  2. Select Process I/O and pick Named Pipe mode.
  3. Enter the pipe path.
  4. Click Connect.

Tips:

  • The child process has to write data in a format Serial Studio can parse (for example CSV lines or delimited frames).
  • For Python scripts, use the -u flag or call flush() after each print() to disable output buffering.
  • If the child process crashes, Serial Studio prints a warning and disconnects.
  • Use Named Pipe mode when you want to connect to a long-running external process without Serial Studio managing its lifecycle.

MQTT subscriber

Subscribes to a broker topic filter and feeds each received payload into the frame pipeline as if the bytes had arrived over UART or TCP. The right transport when the telemetry is already on a broker, or when several Serial Studio instances need to consume the same data without each one talking to the device directly.

Configuration:

ParameterDescription
HostnameBroker address (IP or hostname). Default 127.0.0.1.
PortBroker port. 1883 plaintext, 8883 TLS.
Client IDIdentifier sent on CONNECT. Auto-generated; Regenerate picks a new one.
Username/PasswordOptional broker authentication.
Topic filterTopic to subscribe to. Supports + (one level) and # (rest) wildcards.
MQTT version3.1, 3.1.1, or 5.0.
Clean sessionDiscard persistent session state on CONNECT. Default on.
Keep aliveSeconds between PING packets when idle.
SSL / TLSMaster toggle, protocol family, peer-verify mode, peer-verify depth.
CA certificatesLoad extra PEM certificates for self-signed brokers.

Quick start:

  1. Select MQTT Subscriber as the bus type for a source.
  2. Enter broker hostname, port, and credentials.
  3. Set the topic filter to the topic the device publishes on (or a wildcard pattern that covers it).
  4. Click Connect. Incoming messages are routed to the project's frame parser.

Tips:

  • One MQTT message is one chunk of bytes. The broker preserves payload boundaries, so each publish usually becomes one frame and No Delimiters is a common choice.
  • A wildcard filter like sensors/+/temp multiplexes many publishers onto a single source. If you need to tell them apart, encode the publisher's identity in the payload or use one source per publisher.
  • See MQTT Subscriber for the full protocol primer and MQTT Topics & Semantics for the topic / wildcard / QoS reference.

Multi-device mode

When a project file defines multiple sources, Serial Studio runs in multi-device mode. Each source sets its own bus type, connection settings, frame delimiters, and (optionally) a frame parser in Lua or JavaScript.

Key behaviors:

  • All configured devices connect at the same time when you click Connect.
  • Each device's data routes to its own groups and datasets in the dashboard.
  • Sources are configured in the Project Editor under the Sources section.
  • The Setup panel shows a "Multi-Device Project" notice with a link to open the Project Editor.
  • Driver toolbar buttons are disabled while a multi-device project is active.
  • Multi-device mode needs a Pro license.

Picking a data source

Single-device mode. Use the Setup panel's "I/O Interface" dropdown or the driver buttons in the toolbar to pick a data source type.

Multi-device mode. Each source is configured individually in the Project Editor. The Setup panel shows the active project but doesn't allow per-source driver selection.

Troubleshooting

DriverCommon issues
Serial portWrong COM port or baud rate mismatch. Check that settings match device firmware.
NetworkFirewall blocking the port. Check that IP address and port are reachable.
BLEDevice out of range or not advertising. Check the Bluetooth adapter is enabled.
Audio inputInput device muted or permissions denied. Check OS audio and privacy settings.
ModbusSlave ID or register address mismatch. For RTU, check wiring and termination.
CAN BusBitrate mismatch causes bus errors. Check CAN-H/CAN-L wiring and termination.
Raw USBMissing udev rules (Linux) or kernel driver conflict (macOS). Check endpoints.
HID deviceDevice claimed by another application. Add udev rules on Linux.
Process I/OExecutable not found or stdout buffered. Use -u for Python; check the path.
MQTTBroker unreachable, wrong topic filter, or client-ID collision. Run Test Connection on the MQTT Publisher page to probe credentials.