examples/electron/RunAnywhereAI/index.html
CPUGPU · needs build starting…
On-device models — download, load, and manage storage. Nothing leaves your machine.
Add any model
Load any compatible model by HuggingFace repo, direct URL, or local file path.
Source
Type Language (chat)Vision-languageEmbeddingsSpeech-to-textText-to-speech Add
Examples: unsloth/gemma-3-1b-it-GGUF · https://huggingface.co/owner/repo (paste the page URL) · https://…/model.gguf · C:\models\model.gguf. GGUF + mmproj are auto-resolved for vision models.
Generation defaults and secure credentials.
System prompt
Temperature · 0.7
Max tokens
Reasoning mode — ask the model to think step by step; its reasoning shows in a collapsible block, separate from the answer. Save settings
API key — stored encrypted (Windows DPAPI)
Save key
Compute device
Inference runs on the CPU in this build. A CUDA (NVIDIA GPU) build is available — launch with run-demo-gpu.cmd.
Extract typed JSON — decoding is grammar-constrained, so the output always parses.
Ada Lovelace was a 36 year old English mathematician who loved poetry. Extract
The model picks a tool and fills its arguments.
What is the weather in Tokyo in celsius? Choose tool
Describe an image with the vision-language model.
Choose imageNo image selected
Caption
Semantic similarity — cosine distance between two sentence embeddings.
Compare
Retrieval-augmented generation — add documents, then ask questions grounded in them. Embeddings (MiniLM) index your text; the LLM answers from the retrieved chunks. First run downloads MiniLM + the chat model.
Add a document
Add to knowledgeClear all
Ask a question
Ask
Hold to talk — mic → speech-to-text → LLM → text-to-speech → speaker.
Hold to talk Heard
Reply
Built-in energy VAD. Hold and speak — the first ~2s calibrate ambient noise.
Threshold · 0.015
Hold + speak
Detection
—