doc/help/Data-Flow.md
Understanding how data moves through Serial Studio helps you configure it correctly and troubleshoot issues. This page traces the journey of a single byte from your device to a rendered widget on screen.
The following diagram shows the complete data flow from hardware device to rendered dashboard widgets, including the optional export path for CSV, MDF4, and API output.
flowchart TD
A["Device"] -->|bytes| B["Driver"]
B --> C["Buffer · SPSC"]
C --> D["Frame Reader"]
D --> E["Frame Builder"]
E -->|"const Frame&"| F["Dashboard"]
E -->|export| G["CSV · MDF4 · API"]
Your device sends raw bytes over one of nine supported transports: UART, TCP/UDP, Bluetooth LE, Audio Input, Modbus RTU/TCP, CAN Bus, USB (libusb), HID (hidapi), or Process I/O.
The selected driver receives bytes on a dedicated I/O thread managed by DeviceManager. No parsing happens here — the driver's only job is raw byte transport. Each driver type handles its own protocol: serial framing, TCP streams, BLE characteristic notifications, audio sample buffers, and so on.
Drivers are not singletons. ConnectionManager holds one UI-configuration instance per driver type (used by the QML interface), while DeviceManager creates a fresh live instance for each active connection.
A 10 MB lock-free single-producer single-consumer (SPSC) ring buffer sits between the driver and the frame reader.
The SPSC design is a hard architectural constraint. There is exactly one producer (the driver) and one consumer (the frame reader). Adding a second producer or consumer would violate the lock-free guarantee and corrupt data.
The buffer handles burst data without dropping bytes. If data arrives faster than it can be consumed, an overflow counter increments so you can detect the condition.
The frame reader runs on the I/O thread alongside the driver. Its job is to scan the circular buffer for frame boundaries and extract complete frames.
Delimiter detection uses the KMP (Knuth-Morris-Pratt) string matching algorithm for O(n + m) performance, where n is the buffer size and m is the delimiter length. Four detection modes are available:
After extraction, the frame reader optionally validates a checksum (CRC-8, CRC-16, or CRC-32). Valid frames are placed into a lock-free queue with a capacity of 4096 entries. The main thread is signaled when frames are ready.
The frame reader is configured once before being moved to its thread. If configuration changes (different delimiters, different checksum), the entire frame reader is destroyed and recreated via resetFrameReader(). Mutexes are never used.
The frame builder runs on the main thread. It dequeues frames from the lock-free queue and processes them according to the current operation mode.
No project file is needed. This mode is designed for rapid prototyping with CSV-formatted serial output.
/* and */).parse(frame) function via QJSEngine.In multi-device projects, each device (source) has its own frame reader. The frame builder routes data by source ID through hotpathRxSourceFrame(sourceId, data). Each source maintains its own per-source Frame and its own JavaScript engine instance. Source frames are published independently to the dashboard.
The dashboard receives a const Frame& — a read-only reference. No copying. No heap allocation. This is the critical performance constraint that enables Serial Studio to sustain data rates of 256 KHz and above.
The dashboard updates all active widgets with new values. The UI refresh rate is capped at 20 Hz for optimal performance. Time-series data (plots, FFT, GPS trajectory) is appended to fixed-capacity circular buffers that automatically discard the oldest samples.
All dashboard operations run on the main thread. Widget rendering is handled by Qt Quick's scene graph.
The export path is the only place where a heap allocation occurs per frame. When CSV export, MDF4 export, or the API server is active:
TimestampedFrame is created with a nanosecond timestamp (one make_shared call).Export worker configuration (CSV example):
The API server on port 7777 serializes frames to JSON and broadcasts to connected clients using MCP (JSON-RPC 2.0) or the legacy protocol.
| Operation | Complexity |
|---|---|
| Circular buffer append | O(1) amortized |
| KMP delimiter search | O(n + m), n = buffer size, m = delimiter length |
| JavaScript parse | O(n) interpreter time per frame |
| Dashboard update | O(d), d = total datasets (zero-copy) |
| Export enqueue | O(1) lock-free |
Total hotpath memory allocations: zero on the dashboard path. One shared_ptr per frame on the export path only when export is active.
| Component | Thread | Constraint |
|---|---|---|
| Driver | I/O thread (QThread) | One driver per DeviceManager |
| Circular Buffer | Shared (write: I/O, read: I/O) | SPSC only, never MPMC |
| Frame Reader | I/O thread | Configured once, then immutable. Recreate on config change. |
| Frame Builder | Main thread | Dequeues from lock-free queue |
| Dashboard | Main thread | Zero-copy const Frame& only |
| CSV/MDF4/API Export | Worker threads | Lock-free enqueue from main, batch dequeue on worker |
No data in console: Check driver configuration — correct port, baud rate, IP address, or BLE characteristic.
Data in console but no dashboard: Verify the operation mode. Check that frame delimiters match what your device actually sends. In Project File mode, confirm the JavaScript parser returns valid arrays.
Garbled data: Wrong baud rate, wrong decoder method, or mismatched delimiters. Compare raw console output against your expected format.
Partial frames: Delimiter mismatch. Your device may be sending \r\n while you configured only \n, or vice versa. Inspect the raw hex in the console.
Dashboard not updating: Check that dataset Frame Index values in the project file match the positions in your parsed data array. Index 1 maps to the first element returned by parse().
High CPU with no dashboard: The frame reader may be finding too many false frames. Tighten your delimiters or add checksum validation.
Export files empty: Export workers only write when a device is connected. Check that the export was started before disconnecting.