sdk/runanywhere-electron/README.md
On-device LLM, VLM, STT, TTS, and embeddings for Electron and Node on Windows. The SDK is a native N-API addon over the RunAnywhere rac_* C ABI and llama.cpp / ONNX Runtime / Sherpa-ONNX. Inference runs in an isolated Electron utility process, streaming results to the renderer over a MessagePort.
Status: Unpublished preview (
private: true, version0.1.0). Windows x64 only — not available on npm. Build from source in this repository.
This package is not published to npm. Clone the repository and build the native addon on Windows:
git clone https://github.com/RunanywhereAI/runanywhere-sdks.git
cd runanywhere-sdks/sdk/runanywhere-electron
npm install
npm run build
npm run bundle:native # copies .node + DLLs into prebuilds/win32-x64/
Prerequisites: MSVC, Node.js, and a windows-release build of runanywhere-commons with backends enabled. See docs/DEVELOPMENT.md for CUDA builds, integration tests, and contributor details.
After building, require the local package from your app or the example:
const { RunAnywhere } = require('@runanywhere/electron');
RunAnywhere.initialize();
const llm = await RunAnywhere.loadLLM('qwen2.5-0.5b'); // catalog id or local path
for await (const t of llm.generate('Explain on-device AI in one sentence.')) {
process.stdout.write(t);
}
llm.unload();
RunAnywhere.shutdown();
Point at a custom native build with RUNANYWHERE_NATIVE_PATH if you are not using prebuilds/win32-x64/.
const person = await llm.generateStructured(
'Extract the person: "Ada Lovelace, 36, English mathematician."',
{
schema: {
type: 'object',
properties: {
name: { type: 'string' },
age: { type: 'integer' },
interests: { type: 'array', items: { type: 'string' }, maxItems: 5 },
},
required: ['name', 'age', 'interests'],
},
}
);
const tools = [{
name: 'get_weather',
description: 'Current weather for a city',
parameters: {
type: 'object',
properties: { city: { type: 'string' } },
required: ['city'],
},
execute: ({ city }) => fetchWeather(city),
}];
const run = await llm.generateWithTools('Weather in Tokyo?', tools);
// { name, arguments, result }
const chat = RunAnywhere.createChat(llm, { system: 'You are concise.' });
await chat.sendText('My name is Aman.');
await chat.sendText('What is my name?');
Main process:
const { RunAnywhereMain } = require('@runanywhere/electron/main');
const ra = new RunAnywhereMain({ nativePath: /* optional path to .node */ });
win.webContents.on('did-finish-load', () => ra.connect(win.webContents));
Renderer preload — set webPreferences.preload to @runanywhere/electron/preload. It exposes window.runanywhere with async APIs (loadLLM, generate, transcribe, synthesize, …).
Renderers that bundle the SDK can import audio helpers:
import { MicRecorder, SpeakerPlayer } from '@runanywhere/electron/audio';
When packaging with electron-builder, unpack native artifacts from the asar:
"asarUnpack": ["**/node_modules/@runanywhere/electron/prebuilds/**"]
loadLLM, loadVLM, loadEmbedder, loadSTT, and loadTTS accept a catalog id (auto-downloaded on first use) or a local path. Built-in ids include smollm2-135m, qwen2.5-0.5b, smolvlm-256m, minilm, whisper-tiny, and piper-lessac.
From the repo root on Windows:
examples\electron\RunAnywhereAI\run-demo.cmd
Or:
set RUNANYWHERE_NATIVE_PATH=sdk\runanywhere-electron\prebuilds\win32-x64\runanywhere_native.node
npx electron examples/electron/RunAnywhereAI
The example covers chat/streaming, vision, embeddings, and a mic → STT → LLM → TTS → speaker voice loop. Source: examples/electron/RunAnywhereAI/.
Failures throw SDKException with .code, .category, and .recoverySuggestion, consistent with other RunAnywhere SDKs.
Build and test details: docs/DEVELOPMENT.md.
See the repository LICENSE.