doc/claude/architecture/scripting.md
Part of the architecture corpus (index). Read this file in full before touching the JS/Lua/Native parser engines, per-dataset transforms, the data-table store, the control script, or the code-editor QML.
IScriptEngine is the abstraction. Three impls:
JsScriptEngine — QJSEngine with ConsoleExtension + GarbageCollectionExtension
only (not AllExtensions). Watchdog: always route through
JsScriptEngine::guardedCall() (concrete engine — the IScriptEngine interface does
not carry it); never call parseFunction.call() directly.LuaScriptEngine — Lua 5.4 (lib/lua/lua54), one lua_State* per source.CFrameParser — native C++ parametrized templates (SerialStudio::Native = 2). The
"script" is a canonical JSON descriptor {"params": {...}, "template": "<id>"} built by
CFrameParser::buildDescriptor() (compact + key-sorted, so the BackupManager SHA stays
byte-stable). Template registry: Scripting/NativeTemplates/ — stateless
INativeTemplate descriptors (id, tr()'d metadata, param schema, makeParser()) +
per-source stateful INativeParser instances (latch buffers live in the instance, never
in the registry singleton). Catalog/schema for QML + AI:
CFrameParser::templateCatalog() / templateSchema(id).IScriptEngine::language() (the old dynamic_cast
bool check silently broke with 3 languages). FrameParser::parseMultiFrame* never falls
back to source 0 across language boundaries.Source::frameParserTemplate (string id) +
Source::frameParserParams (JSON object). FrameParser::scriptForSource() builds the
descriptor for native sources (empty template id falls back to the default delimited
comma config).FrameParser::nativeEquivalentForFile() / fileForNativeTemplate() map script template
file basenames ⇄ native ids (+ delimited separator params for the CSV/TSV/pipe/semicolon
variants); JsCodeEditor::switchNativeLanguage uses them, so JS → Native → Lua lands on
the equivalent Lua template, never stale wrong-language code. Custom code that matches no
template falls back to the default template (same semantics as the JS ↔ Lua switch).SourceFrameParserView.qml is the only parser editor view — every parser tree
item (source 0 included) routes through selectSourceParserItem →
SourceFrameParserView (the old project-level FrameParserView.qml was dead code and
was deleted). In native mode it swaps the code editor for NativeParserPane.qml (param
form + Markdown documentation page) and hides the code-editor toolbar row; the native
template combo lives in the secondary toolbar next to a Help button and "Test With
Sample Data". Per-template docs ship at
app/rcc/scripts/native/<id>.md (exposed via
FrameParserModel::templateDocumentation). The bridge is
DataModel::FrameParserModel (registered as QML type FrameParserModel in the
SerialStudio module, see ModuleManager.cpp:545).app/qml/Dialogs/FrameParserTest.qml, one dialog
for all three languages), backed by the same FrameParserModel bridge: pipeline
get/setters write through ProjectModel::updateSource, and dryRun() branches by source
language (JS/Lua → live engines via runFrameParserPipeline, Native →
runNativeTemplatePipeline). The old QWidget FrameParserTestDialog was deleted;
JsCodeEditor::prepareParserTest() only loads/validates the script before QML opens
the dialog. The dialog is hosted by a DialogLoader in ProjectEditor.qml
(parserTestLoader.openTester(sourceId)) — never instantiate it inside the
Loader-managed views, or any updateSource triggered from the dialog rebuilds the view
and destroys the window mid-interaction.QTimer on the thread running QJSValue::call()
can never fire — the event loop is blocked while the script runs (this was a real,
shipped no-op against while(true){}). JsWatchdogThread (a dedicated QThread polling
armed JsWatchdogs every 20 ms) flips setInterrupted(true) from off-thread, which Qt
documents as thread-safe. Lua uses an in-engine LUA_MASKCOUNT hook + QDeadlineTimer
instead. Every JS engine (parser, transform, Painter, Output, MQTT) holds a JsWatchdog
that registers with the thread; arm()/disarm() are lock-free atomic-deadline stores
safe on the hotpath. setInterrupted(true) may appear only in JsWatchdogThread.cpp
— code-verify.py:js-interrupt-off-thread blocks it anywhere else.frameParserLanguage (0 = JS, 1 = Lua, 2 = Native) picks the engine in
FrameParser::engineForSource(). JS/Lua templates in app/rcc/scripts/parser/{js,lua}/
templates.json; native templates are compiled in (nativeTemplates() registry,
delimited/comma is the default).seedDefaultFrameParser in
ProjectModel.cpp: language = Native, template = delimited, params = schema defaults).
frameParserCode is not seeded; the switch mapping generates the equivalent script
template on demand.Binary-decoder scripts get byte tables, not strings. Writing string-based byte access
(string.byte, string.unpack, charCodeAt, split) in a Lua/JS parser that runs under
the Binary decoder — including the Lua templates the importers generate (DBCImporter,
ModbusMapImporter) — breaks every frame: with SerialStudio::Binary,
LuaScriptEngine::parseBinary pushes the frame as a 1-indexed table of byte integers and
JsScriptEngine::parseBinary passes an Array of bytes — not a string. Every frame dies with
bad argument #1 to 'byte' (string expected, got table) (Lua) or frame.split is not a function (JS). Index the table/Array directly (frame[1], sawtooth/LSB walks like the DBC
template); when string.unpack is genuinely needed (multi-byte/float fields, Modbus), convert
once at the top of parse():
if type(frame) == "table" then frame = string.char(table.unpack(frame)) end. The shipped
parser templates (app/rcc/scripts/parser/) and ProtoImporter's bytesToString are the
reference patterns; both importers shipped this bug in 2026-06.
The QML-embedded code editors (JsCodeEditor & siblings) are an offscreen QCodeEditor
grabbed into a QQuickPaintedItem, so three invariants hold them together:
renderWidget() first calls syncWidgetPosition() (moves the hidden top-level to the
item's screen position — completer popups and drag auto-scroll resolve global coordinates
through it).event() forwards ShortcutOverride to the widget (editing keys handled natively —
without it QML Shortcut soup decides, and duplicated sequences across the same window go
ambiguous and silently dead).keyPressEvent reroutes to completer()->popup() while it's visible (the popup never has
real focus in the embedded setup).Input handlers call renderWidget() directly — dropping that re-introduces one-timer-tick
lag on every keystroke/drag.
transformCode string (language matches its source). The user
defines function transform(value) -> number.FrameBuilder::compileTransforms() runs on project load /
connection open. One Lua state or QJSEngine per source; per-dataset function refs cached
in m_transformEngines.luaL_dostring once; top-level locals become upvalues in the
transform closure, so two datasets sharing the same Lua state don't clobber each other.vars are
closure-scoped per dataset.applyTransform(language, uniqueId, rawValue, info) → cached per-source
engine pointer (m_luaEngineForSource / m_jsEngineForSource, refreshed on sourceId
change in applyDatasetValues; no std::map::find per dataset per frame) →
lua_pcall / QJSValue::call. Single-arg transforms skip the info table / object
allocation: acceptsInfo is detected at compile time via lua_getinfo(">u") (Lua) and
function.length (JS) and stored on the per-dataset ref.applyDatasetValues arms the active source's
per-engine watchdog (m_jsEngineForSource->jsWatchdog->arm()) once, runs the dataset loop,
and disarms it. The 100 ms budget covers the whole frame's transforms collectively, and the
interrupt is delivered off-thread by JsWatchdogThread (see Frame Parser). On timeout
applyTransformJs sets m_jsTransformTimedOut; the user-facing notification is posted once
from the main thread after the loop, never from the watchdog thread.[[unlikely]] guarded) and rawValue is returned.DatasetTransformEditor prefills a multiline-comment placeholder when the
dataset has no transform; onApply runs validateTransform(code, language, error) which
returns a TransformStatus (Ok / SyntaxError / NoFunction). SyntaxError and
NoFunction (the placeholder or any code that doesn't define transform()) block
persistence and keep the dialog open with a warning, so the placeholder never persists.DataModel::DataTableStore: flat pre-allocated register store, no hotpath allocation.__datasets__: auto-generated. Two registers per dataset:
raw:<uniqueId>, final:<uniqueId>. Populated by FrameBuilder during parsing."tables". Registers are Constant
(read-only at runtime) or Computed (writable by transforms, persists across frames —
no automatic per-frame reset). The defaultValue is the value at project load only.
Computed registers are the natural place for filter state, integrators, edge counters,
and latched flags; for a "clear me each frame" effect, the transform writes the reset
value explicitly at the top of an early dataset.tableGet, tableSet, datasetGetRaw,
datasetGetFinal. Lua = C closures; JS = TableApiBridge QObject.applyTransform returns QVariant (double or QString). Dataset has
rawNumericValue/rawValue snapshots and virtual_ (no frame index — transform-only).The control script runs on a worker thread with apiCall only: ControlScriptWorker evaluates
setup()/loop() in its own QJSEngine and installs ONLY __ss_bridge (apiCall marshalled to the
GUI thread via BlockingQueuedConnection) — never the direct helper bridges (__ss,
__ss_db, ... wrap main-thread singletons and must not execute off-thread). The SDK prelude
(app/rcc/api/prelude.js, embedded into SerialStudio.js by scripts/generate-sdk.py)
defines control-mode fallbacks that re-route the friendly globals through apiCall:
notify* → notifications.*, tableGet/tableSet → project.dataTable.getValue/setValue
(live store values; DataTablesHandler, backed by FrameBuilder::tableStore()).
datasetGetRaw/datasetGetFinal remain parser/transform-only. io.getLatestFrame returns
ageMs (steady-clock ms since capture) for staleness watchdogs — never compare its monotonic
timestampMs against Date.now(). A new control-script global must follow the apiCall
fallback pattern; installing a direct bridge on the worker engine is a threading bug. Dataset
transforms re-run only on frame arrival, so table writes made while the device is silent
don't render until the next frame. Two SDK calls close that gap, both running the same
transform-only pass (reprocessDatasetValues) over the live frames (per-source frames when
populated, else m_frame) and sharing the private republishFrames(bool feedExports) helper:
refreshDashboard() (dashboard.reprocess → FrameBuilder::reprocessFrames, feedExports
false) publishes to Dashboard::hotpathRxFrame directly, skipping the hotpathTxFrame export
fan-out so a synthetic refresh never re-records frames already exported on arrival;
dashboardTick() (dashboard.tick → FrameBuilder::dashboardTick, feedExports true)
publishes through hotpathTxFrame, so a control-script simulation that owns its values via
data tables both renders and feeds the CSV/MDF4/session/MQTT/API exports (still gated on
m_anyAsyncSink). dashboardTick also seeds each source frame from the project template when
none has arrived yet, so it works from the very first loop().
The control-script lifecycle is per-connection and force-restarted: ControlScript
edge-tracks shouldRun() (m_shouldRun) and on every rising edge queues stop-then-start, so
each connection gets a fresh engine (top-level state resets, setup() re-runs). stopWorker()
always queues the idempotent worker stop even when m_running is false — a worker/GUI
desync must never keep an old engine alive across a cycle — and a setup() exception stops
the worker (loop never arms with the GUI showing stopped). FrameBuilder::onConnectedChanged
clears m_latestFrames on BOTH edges: with the API server on, m_captureLatestFrame stays
true across a disconnect, and a stale retained frame would otherwise leak into the next
connection's io.getLatestFrame/newFrame(). controlscript.dryRun compile-checks source
in a throwaway GUI-thread engine (stub __ss_bridge + SDK prelude + JsWatchdog) without
installing it; controlscript.getCode/setCode are registry aliases of get/set. The
agent-facing globals reference is :/ai/docs/control_script_js.md
(meta.fetchScriptingDocs{kind:'control_script_js'}; allow-lists in
ContextBuilder::scriptingDocFor AND ToolDispatcher::getScriptingDocs, plus rcc.qrc).