packages/docs/apps/desktop.md
The Eliza desktop app wraps the companion UI in a native Electrobun shell, adding system-level features like tray icons, global keyboard shortcuts, native notifications, and native OS capability bridges. Electrobun can either launch the canonical Eliza runtime locally or connect the UI to an already-running local or remote runtime.
Download the .dmg file from the GitHub releases page. Open the DMG and drag Eliza to your Applications folder.
public.app-category.productivity).Download the .exe installer (NSIS) from the releases page.
eliza-code-sign certificate profile).Download the .AppImage or .deb package from the releases page.
git clone https://github.com/elizaOS/eliza.git && cd eliza
bun install && bun run build
bun run dev:desktop
For why the desktop dev commands spawn multiple processes, how Ctrl-C and Quit behave, environment variables (ELIZA_DESKTOP_VITE_WATCH, ELIZA_RENDERER_URL, etc.), and IDE/agent observability (GET /api/dev/stack, aggregated console log, screenshot proxy — why loopback, defaults, and opt-out), see Desktop local development.
In development mode, the Electrobun app resolves the Eliza distribution from the repository root's dist/ directory. In packaged builds, assets are copied into the app bundle under Resources/app/eliza-dist/.
On macOS, the main window uses hiddenInset (no classic title bar; traffic lights inset). The WKWebView fills the client area, so window move and inner-edge resize are implemented with native NSView overlays above the web view — not with CSS resize cursors alone. Why: WebKit owns the pointer over page pixels; tracking areas on the contentView underneath led to unreliable cursors and flicker when AppKit and WebKit both tried to set NSCursor.
Strip thickness can track the current NSScreen when the host passes height: 0 into native layout (see main-process applyMacOSWindowEffects and FFI setNativeDragRegion). Full architecture, z-order, and file map: Electrobun macOS window chrome.
Electrobun may log [WebGPU Browser] macOS … using os.release() (Darwin). Why document: on macOS 26, Darwin is still 25.x; a naive Darwin − 9 mapping shows 16 and mis-gates WKWebView WebGPU. Eliza maps Darwin to the marketing major in code; rationale and table: Darwin vs macOS version (Electrobun WebGPU).
Product framing: Eliza targets strong visuals when you are engaged and quiet hardware when you are not—especially on battery—without pretending every workload beats a full IDE shell. See Roadmap — Principles: energy and experience (desktop).
What drives usage
dev:desktop / dev:desktop:watch run API + Vite + Electrobun (+ optional screenshot helper). Why: each process has its own baseline CPU wakeups; this is a dev convenience, not the same as a minimal shipped shell.GET /api/dev/cursor-screenshot path uses full-screen capture when agents poll it. Why: screencapture and compositor work are noticeable if something hits that endpoint often — turn it off when you do not need it (ELIZA_DESKTOP_SCREENSHOT_SERVER=0); see Desktop local development.What Eliza already does
VrmEngine.setPaused / VrmViewer), e.g. when you leave companion mode for native tabs (settings, chat shell) so the 3D loop is not running in the background for those routes. Why: requestAnimationFrame / setAnimationLoop at display refresh is the main avoidable steady-state cost.VrmViewer also pauses when document.visibilityState !== "visible" (background tab / hidden document). Why: WKWebView can keep scheduling frames for a visible canvas; aligning with visibility avoids burning GPU when the user is not looking at Eliza.useIntervalWhenDocumentVisible (or equivalent) so 5s / 3s refresh timers do not hit the API while the document is hidden. Eliza Cloud credits polling (60s) skips work when hidden. Why: same battery/thermal story as the VRM loop, for network + React wakeups.requestAnimationFrame loop stops while the tab is hidden and resumes on visible. Why: second WebGL context should not animate off-screen.desktop:getPowerState on a 60s timer, when the renderer becomes ready, and when document.visibilityState returns to visible (so plugging in is noticed without waiting for the next interval). When onBattery is true, VrmEngine.setLowPowerRenderMode caps effective DPR at 1× on top of the usual MAX_RENDERER_PIXEL_RATIO clamp. Why: fewer shaded pixels when unplugged (e.g. HiDPI laptops). The main process resolves AC vs battery using pmset on macOS, /sys/class/power_supply (Battery + Discharging) on Linux, and SystemInformation.PowerStatus.PowerLineStatus on Windows (Offline = on battery). Opt-out: set localStorage eliza.vrmBatteryPixelCap to "0" to keep full resolution on battery (user Companion efficiency in Settings → Media can still request low-power on AC).eliza:companion-vrm-power is quality (never battery low-power), balanced (low-power on battery when the cap is on), or efficiency (always low-power). Legacy boolean keys migrate once.eliza:companion-animate-when-hidden): when the window or tab is hidden but companion is still the active scene, the engine stays unpaused and only the world + Spark are hidden so the VRM can idle with lower cost than drawing the full splat scene.setLowPowerRenderMode also disables directional shadow maps on the avatar key light and applies tighter Spark splat limits (maxPixelRadius, minAlpha, sort distance, etc.). Why: shadows and splat sorting are a large share of GPU time in companion/world mode.VrmEngine.setHalfFramerateMode skips every other main-loop tick (skipped ticks do not advance Clock, so the next tick’s delta is doubled). setLowPowerRenderMode is separate (DPR / shadows / Spark). Default policy ties half-FPS to “saving power” moments; Settings → Media can set full speed, when saving power, or always half.starting / onboarding is loading. Why: avoids paying WebGL/WebGPU init during the boot path when the UI only needs status and loaders.MAX_RENDERER_PIXEL_RATIO in VrmEngine) so Retina does not always mean 2× shader cost at 3× physical pixels.What you can do today
companionSceneActive stays tied to shell/tab state, so the heavy scene is off when you are not in companion or character flows.eliza.avatarRenderer to webgl (or webgpu to experiment the other way). Why: path differs by machine and OS version; the desktop webview defaults WebGPU in the Electrobun runtime — sometimes the fallback is kinder to thermals.ELIZA_DESKTOP_SCREENSHOT_SERVER, ELIZA_DESKTOP_DEV_LOG).Code: eliza/packages/app-core/src/hooks/useDocumentVisibility.ts, VectorBrowserView.tsx (3D graph), ElizaCloudDashboard.tsx, StreamView.tsx, stream/StreamVoiceConfig.tsx, GameView.tsx, ChatView.tsx (game-modal carryover timer), FineTuningView.tsx, state/AppContext.tsx (cloud credits interval), VrmViewer.tsx, VrmEngine.ts, vrm-desktop-energy.ts.
Electrobun is a native shell, not a separate runtime architecture. Desktop, VPS, sandboxed, and CLI/server deployments all use the same Eliza runtime entrypoint. The shell chooses one of three runtime modes at startup:
| Mode | Behavior |
|---|---|
local | Spawn the canonical Eliza runtime locally as a child Bun process |
external | Do not spawn a local runtime; point the renderer at an explicit API base |
disabled | Do not auto-start a local runtime; still point the renderer at the expected local API base for a manually managed server |
On startup, the Electrobun shell and AgentManager coordinate these steps:
dist/ bundle. In packaged builds, the runtime is copied into Resources/app/eliza-dist/.local, external, or disabled runtime mode.window.__ELIZA_API_BASE__ into index.html before React mounts so the UI never falls back to the static server for /api/* requests.local, spawn the canonical runtime -- Electrobun launches bun run entry.js start as a child process, waits for /api/health, and then pushes the actual bound port to the renderer.external, connect only -- Electrobun does not start a child runtime. The renderer uses the normalized external API base and optional API token.disabled, wait for a manually managed local runtime -- Electrobun does not auto-start the child runtime, but the renderer still targets the expected local API base so a separately managed server can satisfy requests.Embedded local mode (packaged or dev without external API): the Electrobun main process chooses a listen port for the child eliza start process as follows:
ELIZA_PORT (default 2138). The shell probes 127.0.0.1 and, if that port is busy, uses the next free port (same idea as dev-platform, implemented in loopback-port.ts). Why: two Eliza instances or another service may legitimately hold 2138; we should not SIGKILL unrelated processes by default (see ELIZA_AGENT_RECLAIM_STALE_PORT in Desktop local development to opt back into reclaim).ELIZA_PORT so entry.js start binds there when possible.waitForHealthy follow the actual port before marking the agent running.pushApiBaseToRenderer / injectApiBase use AgentManager’s resolved port; status listeners refresh main and detached windows. Why: the dashboard must not keep using a stale loopback URL after a dynamic bind.external mode: no embedded child; the UI uses ELIZA_DESKTOP_API_BASE / related env (e.g. dev-platform sets this to http://127.0.0.1:<resolved API port>). Why: the API may already be running under bun run dev with its own port policy.
disabled mode: no auto-start; the renderer still targets the expected local API base for a process you manage yourself—set ELIZA_PORT / ELIZA_API_PORT to match that server.
CLI eliza start (non-Electrobun): after startApiServer returns, Eliza syncs ELIZA_PORT and ELIZA_API_PORT to the actual bound port. Why: if the HTTP stack falls forward to another port, shells and scripts reading env see the same port as /api/health.
The OS menu bar template is built in apps/app/electrobun/src/application-menu.ts and wired in index.ts (application-menu-clicked). Why a data file: the same structure is validated by tests and stays free of platform branches scattered through the main process.
| Item (example) | Action id | Behavior |
|---|---|---|
| Reset Eliza… | reset-eliza | Main process: shows the window, native confirm, then POST /api/agent/reset, embedded restart or POST /api/agent/restart, poll /api/status, and pushes desktopTrayMenuClick with itemId: "menu-reset-eliza-applied" + agentStatus. Renderer: handleResetAppliedFromMain runs the same local UI wipe as the end of Settings handleReset (completeResetLocalStateAfterServerWipe). Why main owns HTTP: after native dialogs, WKWebView can defer renderer fetch/bridge work, so reset looked hung; why renderer still wipes UI: one place for onboarding, ElizaClient base URL, cloud flags, and conversation lists so the menu cannot drift from Settings. |
Settings still uses handleReset (webview confirm + full flow). Legacy: tray may still emit menu-reset-eliza for older paths; see Desktop main-process reset for sequence, probes, and tests.
The embedded agent reports its state to the UI via IPC:
| State | Meaning |
|---|---|
not_started | Agent has not been started yet |
starting | Agent is initializing (API server may already be available) |
running | Agent is active and accepting requests |
stopped | Agent has been shut down |
error | Agent encountered a fatal error |
For testing, remote connectivity, or locally managed runtime workflows:
| Environment Variable | Effect |
|---|---|
ELIZA_DESKTOP_TEST_API_BASE | Use this API base and switch to external mode |
ELIZA_DESKTOP_API_BASE | Use this API base and switch to external mode |
ELIZA_API_BASE_URL / ELIZA_API_BASE | Generic API-base fallback vars; also switch to external mode |
ELIZA_DESKTOP_SKIP_EMBEDDED_AGENT=1 | Switch to disabled mode; do not auto-start the child runtime |
ELIZA_API_TOKEN | Inject an API authentication token into the renderer |
The desktop app registers 10 native modules via IPC, each providing platform-specific capabilities. All modules are initialized in initializeNativeModules() and their IPC handlers are registered in registerAllIPC(). Every module follows a singleton pattern with a dedicated manager class.
Local embedded runtime management via the AgentManager class.
| IPC Channel | Description |
|---|---|
agent:start | Start the local child runtime when desktop mode is local |
agent:stop | Stop the local child runtime |
agent:restart | Stop and restart the runtime, picking up config changes |
agent:status | Get the current AgentStatus object |
In external and disabled mode, agent:start rejects instead of spawning the embedded runtime. The agent also emits agent:status events to the renderer whenever local-runtime state changes.
Core native desktop features via the DesktopManager class. This is the largest module, covering eight subsystems:
System Tray -- Create, update, and destroy tray icons with context menus. Supports tooltip, title (macOS), icons for menu items, and submenus. Tray events (click, double-click, right-click) are forwarded to the renderer with modifier key state and cursor coordinates.
Global Keyboard Shortcuts -- Register system-wide hotkeys that work even when the app is not focused. Each shortcut has a unique ID and an desktop accelerator string. When pressed, a desktop:shortcutPressed event is sent to the renderer.
| IPC Channel | Description |
|---|---|
desktop:registerShortcut | Register a global shortcut by ID and accelerator |
desktop:unregisterShortcut | Unregister a shortcut by ID |
desktop:unregisterAllShortcuts | Remove all registered shortcuts |
desktop:isShortcutRegistered | Check if an accelerator is currently registered |
Auto-Launch -- Configure the app to start on system login, optionally hidden, via desktop:setAutoLaunch and desktop:getAutoLaunchStatus.
Window Management -- Programmatic control over the main window. Supports size, position, min/max dimensions, resizability, always-on-top, fullscreen, opacity, vibrancy (macOS), background color, and more. Window events (focus, blur, maximize, minimize, restore, close) are forwarded to the renderer.
Native Notifications -- Rich notifications with actions, reply support, urgency levels, and click handling. Each notification gets a unique auto-incremented ID. Supports click, action, reply, and close event callbacks forwarded to the renderer.
Power Monitoring -- Battery state, idle time detection, and suspend/resume events. Emits desktop:powerSuspend, desktop:powerResume, desktop:powerOnAC, and desktop:powerOnBattery events.
Clipboard Operations -- Read and write text, HTML, RTF, and images to the system clipboard.
Shell Operations -- Open external URLs in the default browser, reveal files in Finder/Explorer, and trigger system beeps.
Network discovery for finding Eliza gateway servers on the local network via the GatewayDiscovery class. Uses mDNS/Bonjour for service discovery with the _eliza._tcp service type.
The module dynamically loads discovery libraries in priority order:
Discovered gateways include metadata from TXT records: stable ID, TLS configuration, gateway port, canvas port, and Tailnet DNS name. Events (found, updated, lost) are forwarded to the renderer via gateway:discovery.
| IPC Channel | Description |
|---|---|
gateway:startDiscovery | Begin scanning with optional service type and timeout |
gateway:stopDiscovery | Stop active discovery |
gateway:getDiscoveredGateways | List all currently known gateways |
gateway:isDiscovering | Check if discovery is active |
Full conversation mode via the TalkModeManager class, integrating speech-to-text (STT) and text-to-speech (TTS).
STT Engines:
@elizaos/plugin-local-inference (Qwen3-ASR via libelizainference / llama.cpp). Supports streaming transcription.TTS Engines:
eleven_v3), stability, similarity boost, and speed. Audio chunks are streamed to the renderer as base64-encoded data.Voice Activity Detection (VAD): Configurable silence threshold and duration for automatic speech segmentation.
| State | Meaning |
|---|---|
idle | Talk mode is off |
listening | Actively capturing and transcribing audio |
processing | Processing captured speech |
speaking | TTS is playing audio |
error | An error occurred |
Audio data flows from the renderer to the main process via talkmode:audioChunk IPC messages as Float32Array samples.
Wake word detection for hands-free activation via the SwabbleManager class. Uses Whisper for continuous speech transcription combined with a WakeWordGate that performs timing-based wake word matching.
Configuration:
triggers -- Array of wake word phrases (e.g., ["eliza", "hey eliza"])minPostTriggerGap -- Minimum pause (seconds) after the wake word before the command starts (default: 0.45s)minCommandLength -- Minimum number of words in the command after the wake word (default: 1)modelSize -- Whisper model size to useThe wake word gate includes fuzzy matching for common transcription variations (e.g., "melody" matches "eliza", "okay" matches "ok").
When a wake word is detected, a swabble:wakeWord event is sent to the renderer containing the matched trigger, extracted command, full transcript, and the post-trigger gap measurement.
Native screenshot and screen recording via the ScreenCaptureManager class.
Screenshots: Capture the primary screen, a specific source, or the main window. Supports PNG and JPEG formats with configurable quality. Screenshots can be saved to the user's Pictures directory.
Screen Recording: Uses a hidden BrowserWindow renderer for MediaRecorder-based recording (since MediaRecorder requires a renderer context). Supports configurable quality presets, FPS, bitrate, system audio, and max duration auto-stop. Recordings are saved as WebM (VP9 preferred) to the system temp directory.
| Quality | Bitrate |
|---|---|
low | 1 Mbps |
medium | 4 Mbps |
high | 8 Mbps |
highest | 16 Mbps |
Recording supports pause/resume and provides real-time state updates including duration and file size.
Camera capture for photo and video via the CameraManager class. Like screen recording, this uses a hidden BrowserWindow renderer for getUserMedia / MediaRecorder access.
Features:
| Quality | Video Bitrate |
|---|---|
low | 1 Mbps |
medium | 2.5 Mbps |
high | 5 Mbps |
highest | 8 Mbps |
Auxiliary BrowserWindow management via the CanvasManager class. Each canvas is a separate window used for web navigation, JavaScript evaluation, page snapshots, and A2UI (Agent-to-UI) message injection.
| IPC Channel | Description |
|---|---|
canvas:createWindow | Create a new canvas window (default 1280x720, hidden) |
canvas:destroyWindow | Close and dispose a canvas window |
canvas:navigate | Navigate a canvas to a URL |
canvas:eval | Execute JavaScript in the canvas page |
canvas:snapshot | Capture a screenshot (supports sub-rectangles) |
canvas:a2uiPush | Inject an A2UI message payload |
canvas:a2uiReset | Reset A2UI state on the page |
canvas:show / canvas:hide | Toggle visibility |
canvas:resize | Resize with optional animation |
canvas:listWindows | List all active canvas windows |
Canvas windows emit canvas:didFinishLoad, canvas:didFailLoad, and canvas:windowClosed events to the main renderer.
GPS and geolocation services via the LocationManager class using IP-based geolocation.
The module queries multiple IP geolocation services as fallbacks: ip-api.com, ipapi.co, and freegeoip.app. It supports single position queries, position watching (polling at configurable intervals), and caching of the last known location.
System permission management via the PermissionManager class with platform-specific implementations for macOS, Windows, and Linux.
Managed permissions:
| Permission ID | Name | Platforms | Required For |
|---|---|---|---|
accessibility | Accessibility | macOS | Computer use, browser control |
screen-recording | Screen Recording | macOS | Computer use, vision |
microphone | Microphone | All | Talk mode, voice |
camera | Camera | All | Camera, vision |
shell | Shell Access | All | Shell/terminal commands |
Permission states are cached for 30 seconds (configurable). The shell permission includes a soft toggle -- it can be disabled in the UI without affecting the OS-level permission.
IPC channels include permissions:getAll, permissions:check, permissions:request, permissions:openSettings, permissions:checkFeature, and permissions:setShellEnabled.
The desktop app registers these global keyboard shortcuts:
| Shortcut | Action |
|---|---|
Cmd/Ctrl+K | Open the Command Palette |
Cmd/Ctrl+E | Open the Emote Picker |
These shortcuts work system-wide when the app is running. Additional shortcuts can be registered dynamically via the desktop:registerShortcut IPC channel.
The desktop app supports the eliza:// custom URL protocol for deep linking. The protocol is registered via the Electrobun deep linking integration.
The eliza://share URL scheme allows external applications to share content with your agent:
eliza://share?title=Hello&text=Check+this+out&url=https://example.com
Parameters:
title -- optional title for the shared content.text -- optional text body.url -- optional URL to share.file -- one or more file paths (can be repeated).File drag-and-drop from the OS is also supported via the desktop runtime open-file event. Share payloads are queued if the main window is not yet ready and flushed once the renderer finishes loading. Events are dispatched as eliza:share-target custom DOM events.
The desktop app checks for updates on launch via the Electrobun updater, publishing to GitHub releases under the elizaOS/eliza repository.
In development mode:
localhost and devtools://* origins are allowed for scripts.BrowserWindow instances use contextIsolation: true and nodeIntegration: false.