website/layouts/index.html
Open source · MIT · v4.8.0
Text, voice, vision, images, video, 3D and agents, from one open runtime. It works on the laptop you already own, and scales to a room full of GPUs when you have one. When the engine we need is too heavy, too closed, or does not exist, we write it.
Install LocalAI →Read the docs★ Star on GitHub
0 GitHub stars
0 Backends
0 Engines we wrote
0 Models, one click
localai · chat CPU only, no GPU
The runtime
One binary with an OpenAI-compatible API in front of it. Point an existing client at it and the calls keep working, except now the model is on your machine. It also speaks the Anthropic, Ollama and ElevenLabs APIs, so most tools need a URL change and nothing else.
The engine behind that API is swappable. One model can run on llama.cpp while the next loads on vLLM, SGLang or MLX, and the client never notices: same endpoint, same request, different engine underneath. Switching is one line in the model's config.
A small core pulls each engine in as a separate backend, only when a model asks for it. That is why one install covers this much ground without becoming a 9 GB download.
OpenAI APIAnthropic APIOllama APIElevenLabs APIRealtime over WebRTC
ReasonLanguage models, tool calling, structured outputllama.cpp · vLLM · MLXListenRealtime voice, transcription, diarizationparakeet · whisperSpeakSpeech synthesis and voice cloningmoss-tts · piperSeeVision, detection, recognition, depth, 3Drf-detr · depth-anythingCreateImages, video, music and sounddiffusers · ace-stepActAgents, MCP, skills, RAG, interactive toolsagents · MCP apps
localai · model gallery 1,585 models
Why the project exists
01 / HARDWARE
That path is tested in CI, on the hardware most people already have, and it is not a degraded fallback. A GPU makes it faster but is not required.
x86_64 · ARM64 · CUDA · ROCm · SYCL · Metal · Vulkan
02 / REALTIME
Speech in, tool calls in the middle, speech out over WebRTC, fast enough to feel like a conversation. Transcription, diarization and speech synthesis all run without a GPU.
Realtime API · WebRTC · streaming ASR · TTS · VAD
03 / DISTRIBUTED
Routing, VRAM-aware placement, prefix-cache affinity and failover are the runtime's problem. You add hardware, the cluster works out what to do with it.
Smart routing · autoscaling · P2P · NATS · federation
Senses
A chat model can only work with what somebody types at it. Our engines change what it has access to: what is happening in the room, who walked into it, where things are in space, and how to answer out loud. All of it runs locally, most of it on a CPU.
Hear words
Live transcription with speaker labels and timestamps, fast enough to keep up with a meeting while it is still happening.
parakeet.cppmoss-transcribe.cpp
Hear the room
527 kinds of sound event: a door, a dog, breaking glass, a smoke alarm. The model notices things nobody thought to type.
ced.cpp
Know who
Recognise a voice, recognise a face, and tell a live person from a photo held up to the camera.
voice-detect.cppface-detect.cpp
See things
Ask for "the red mug on the left" in plain language and get back coordinates, not a caption.
locate-anything.cpprf-detr.cpp
See space
Distance in metres from one ordinary photo, and a full 3D reconstruction from a handful of them. No rig, no camera poses, no GPU.
depth-anything.cppfree-splatter.cpptrellis2.cpp
Speak
Long-form speech in a cloned voice, across dozens of languages, up to 48 kHz.
moss-tts.cppmagpie-tts.cppvibevoice.cppvoxtral-tts.c
Hear clearly
Echo cancellation, noise suppression and dereverberation, so a voice loop survives a real room with a real speaker in it.
LocalVQE
Forget on purpose
Names, addresses and card numbers get caught and redacted on the machine, before anything is sent anywhere.
privacy-filter.cpp
One session can do all of it at once: hear the room, work out who is talking, read what is on the desk, call a tool, and answer out loud. One API, one machine, nothing leaves the building.
Made with LocalAI
Sound on
Play with sound
Every part of this clip came out of LocalAI, and the agent in the binary drove the machine that made it. It opened the app, ran the demo and captured the screen, while a local model wrote the script, a cloned voice read it, and the video endpoint generated and lip-synced the presenter. No human touched the keyboard, and nothing left the building.
Direction The agent in LocalAI drove the machine end to end
Script Written by a local language model
Voice Cloned from a few seconds of reference audio
Presenter Generated and lip-synced through the video endpoint
How the voice pipeline works →
Engines we build
{{/* Names the link back to the runtime section explicitly. Readers were arriving here and taking these for unrelated side projects, because nothing on the page said they are the backends the core loads. */}}
Those backends the core pulls in on demand are mostly ours. Most projects wrap somebody else's engine. We wrote ours, because the thing we needed came as a 9 GB Python install, or was closed, or did not exist. Each one is a binary and a GGUF file, checked against the reference implementation in CI.
Twenty-seven times faster than whisper.cpp, on a CPU.
NVIDIA NeMo Parakeet, ported to C++ and ggml. Ten checkpoints, all of them verified at WER 0 against NeMo, which means the transcript is identical to NeMo's while finishing first. Cache-aware streaming with end-of-utterance detection handles live audio, and the multilingual streaming model covers 40 or more locales.
27x vs whisper.cpp, CPU
1.40x vs NeMo, CPU median
WER 0 Parity with NeMo
37% Size at q8_0
Same audio, same words, ours finishes first
CPU speedup over NeMo, by dtype
Beats PyTorch on CPU, in half the memory.
Depth Anything 3 as a 99 MB file. It gives you metric depth, per-pixel confidence, camera intrinsics and extrinsics, and a back-projected point cloud you can export to glb or COLMAP. Output correlates 1.0 with the reference implementation, component by component, and there is no Python or CUDA toolkit anywhere at inference.
1.31x vs PyTorch, CPU
363 MB Peak RAM, q8_0
6.7x Faster to load
99 MB Smallest build
depth-anything.cpp on GitHub ↗
One photo in, distance in metres out, ahead of PyTorch on the same CPU
Every clip comes out of the benchmark suite that runs in CI on that engine. Where you see two panes, ours is racing the reference implementation on the same machine, on the same input.
[
parakeet.cpp long-form transcription vs NeMo ](https://github.com/mudler/parakeet.cpp)[
ced.cpp sound events, live 527 classes ](https://github.com/localai-org/ced.cpp)[
face-detect.cpp detect, landmark, recognise no Python ](https://github.com/mudler/face-detect.cpp)[
face-detect.cpp same person, different photo 1 to N ](https://github.com/mudler/face-detect.cpp)[
voice-detect.cpp who is speaking vs reference ](https://github.com/localai-org/voice-detect.cpp)[
depth-anything.cpp metric depth vs PyTorch, CPU ](https://github.com/mudler/depth-anything.cpp)[
locate-anything.cpp say it, find it open vocabulary ](https://github.com/mudler/locate-anything.cpp)[
moss-tts.cpp 48 kHz voice cloning vs reference ](https://github.com/mudler/moss-tts.cpp)[
magpie-tts.cpp 9 languages, 5 voices vs reference ](https://github.com/mudler/magpie-tts.cpp)
vllm.cppparakeet.cppmoss-transcribe.cppmoss-tts.cppmagpie-tts.cppced.cppvoice-detect.cppvoxtral-tts.cvibevoice.cpprf-detr.cpplocate-anything.cppdepth-anything.cppface-detect.cppfree-splatter.cpptrellis2.cppprivacy-filter.cppLocalVQElocal-storeapex-quantvllm.cppparakeet.cppmoss-transcribe.cppmoss-tts.cppmagpie-tts.cppced.cppvoice-detect.cppvoxtral-tts.cvibevoice.cpprf-detr.cpplocate-anything.cppdepth-anything.cppface-detect.cppfree-splatter.cpptrellis2.cppprivacy-filter.cppLocalVQElocal-storeapex-quant
In development, not announced yet
vLLM ported to C++20, with paged attention, continuous batching and prefix caching, on CPU, CUDA, Metal and Vulkan. It installs as 66 MB instead of a 9.1 GB virtualenv, and it stays ahead of vLLM at every concurrency level we have measured so far. Still being finished, so treat the numbers as provisional.
All {{ len .Site.Data.engines.engines }} engines →
APEX quantization
{{/* APEX was being used as a known term on first appearance, in a section that opened straight onto a benchmark table. The first two sentences say what it is and why it follows the engines. */}}
The engine decides how fast a model runs. The weights decide whether it runs at all, so we build those too. APEX assigns a different precision to every tensor and every layer: a 35B mixture-of-experts model goes from 64.6 GB, out of reach of any consumer GPU, to 12.2 GB at 74 tokens a second. That is more than twice the speed of the original, and quality barely moves. The file is an ordinary GGUF, so stock llama.cpp opens it with no patches, and 201 of them are already sitting in the LocalAI gallery.
F16 · 64.6 GB30.4 t/s
Q8_0 · 34.4 GB52.5 t/s
APEX Quality · 21.3 GB62.3 t/s
APEX Mini · 12.2 GB74.4 t/s
| Build | Size | Perplexity | HellaSwag | MMLU | tg128 t/s |
|---|---|---|---|---|---|
| F16 | 64.6 GB | 6.537 | 82.5% | 41.5% | 30.4 |
| Q8_0 | 34.4 GB | 6.533 | 83.0% | 41.2% | 52.5 |
| Unsloth UD-Q8_K_XL | 45.3 GB | 6.536 | 82.5% | 41.3% | 36.4 |
| APEX Quality | 21.3 GB | 6.527 | 83.0% | 41.2% | 62.3 |
| APEX I-Quality | 21.3 GB | 6.552 | 83.5% | 41.4% | 63.1 |
| APEX Compact | 16.1 GB | 6.783 | 82.5% | 40.9% | 69.8 |
| APEX Mini | 12.2 GB | 7.088 | 81.0% | 41.3% | 74.4 |
Qwen3.5-35B-A3B on an NVIDIA DGX Spark (GB10). Perplexity on wikitext-2-raw at context 2048. Full methodology in the technical report.
APEX models on Hugging Face ↗Technical report ↗
Distributed
01
One command on any box. It reports what hardware it has and which backends it can run, then joins the pool.
02
Requests land on the replica that already holds the model and the matching prefix cache, sized against real free VRAM rather than a guess.
03
In-flight work reschedules, the model loads somewhere else, and the client never finds out.
node-01 24 GB · CUDA
node-02 16 GB · ROCm
node-03 CPU · 64 GB
router prefix affinity · VRAM aware · autoscaling
The gallery
[
Quantizations
Every tier of every model we quantize, ranked against the hardware you actually have and installed with one click.
Browse the gallery →](/docs/getting-started/models/)[
Speech
Italian speech recognition trained and published by the team, streaming on a CPU through parakeet.cpp.
Model card →](#)[
Voices
Forty-two languages of text to speech, small enough to run on a Raspberry Pi, installed from the web interface.
Voice catalogue →](/docs/features/text-to-audio/)
the agent
Run local-ai chat and you are talking to an agent that already knows where your models are. It runs shell commands behind an approval gate you control, delegates to sub-agents, and loads MCP servers, plugins and skills. Claude Code plugins load as they are.
It is also nib, a single ~20 MB Go binary with no runtime and no daemon, so you can drop the same agent on any box you SSH into and press Ctrl+Space.
local-ai chatMCPpluginsskillssub-agentszero dependencies
Read the docs →nib on GitHub ↗
local-ai chat the agent, in the binary
Since March 2023
LocalAI has been trending on GitHub repeatedly since it launched, and it is one of the most starred self-hosted AI projects there is. {{ lang.FormatNumberCustom 0 .Site.Data.stats.contributors }} people have contributed code, {{ lang.FormatNumberCustom 0 .Site.Data.stats.discord }} are in the Discord, and the README is kept translated into eight languages because the users are everywhere.
0 Stars
0 Forks
0 Contributors
0 Releases
0 In Discord
0 Shipping since
MAR 2023
An OpenAI-compatible API in front of llama.cpp, so a laptop could answer the same calls as the cloud.
JUL 2025
Every backend moves out of the binary. You install only the engines your models need.
MAR 2026
Native agentic orchestration, a full React rewrite with canvas mode, WebRTC realtime audio.
APR 2026
Distributed mode with VRAM-aware routing, autoscaling, multi-user auth and per-user quotas.
MAY 2026
Voice recognition, face recognition with liveness, diarization, video generation, drop-in Ollama API.
JUL 2026
The native C and C++ ports take over the heavy Python backends, one modality at a time.
Where it runs
NVIDIA CUDAAMD ROCmIntel SYCLApple MetalVulkanJetsonRaspberry Pix86_64ARM64KubernetesDocker
DeutschEspañolfrançais日本語한국어PortuguêsРусский中文 {{/* The strongest sentence in this section is somebody else's, so it opens the section rather than closing it. Everything below is supporting evidence for what these three people already said. */}}
What other people say
[
“Some cool ggml-based work by @mudler_it recently, make sure to check it out.”
@ggerganov Georgi Gerganov, author of llama.cpp and ggml
On X, 2026 ↗](https://x.com/ggerganov/status/2065447087311917459)[
“What a wonderful project: parakeet.cpp. A ggml based parakeet inference pipeline that is 2x faster than my ONNX parakeet pipeline on Apple Silicon.”
@badlogicgames Mario Zechner, author of pi.agent
On X, 2026 ↗](https://x.com/badlogicgames/status/2061201400059531729)[
“Build something good enough that the community chooses to carry it beyond your reach. This week, Ettore Di Giacinto brought NVIDIA Parakeet to the CPU.”
Adi Margolin On LinkedIn
Read the post ↗](https://www.linkedin.com/posts/adimargolink_spinoza-saw-all-things-straining-to-become-activity-7468135820076634113-u3Hc) {{/* Names run as a sentence rather than a grid of pills. A pill wall of employers reads as a customer logo wall, which is a claim we are not making; a sentence keeps it about the people, which is the true one. */}}
Who shows up
{{- $co := slice }} {{- range .Site.Data.ecosystem.contributors.companies }}{{ $co = $co | append (printf " %s" .name) }}{{ end }} {{- $ac := slice }} {{- range .Site.Data.ecosystem.contributors.academia }}{{ $ac = $ac | append (printf " %s" .) }}{{ end }}
Some of them do it from a desk at {{ delimit $co ", " " and " | safeHTML }}. Others from labs at {{ delimit $ac ", " " and " | safeHTML }}.
That is where they work, not who sent them. Check out the full contributor list, 200+ and growing ↗
{{/* The list runs as a marquee because the count is the argument: any one of these is a weak signal, and the whole moving line is the strong one. It pauses on hover so the links stay usable. */}}
Click any name to see where they say so.
{{- range .Site.Data.ecosystem.integrations.projects }} {{ .name }} {{- end }} {{- range .Site.Data.ecosystem.integrations.projects }} {{ .name }} {{- end }}
Using LocalAI at work? Add your organisation to ADOPTERS.md.
{{- range .Site.Data.ecosystem.press.articles }} {{ .outlet }} {{ .title }}{{ .what }} {{- end }}
Built on, integrated with, written about
[
@sozercan builds on
AIKit now offers an extensible solution for finetuning LLMs! Thanks to @UnslothAI, you can finetune fast and efficiently. Then, deploy seamlessly with AIKit using @LocalAI_API for an end-to-end solution!
On X, 2024 ↗](https://x.com/sozercan/status/1769769695081546236)[
@ivanfioravanti benchmarks
An M5 Max with 40 GPU cores just beat an M3 Ultra with 80 on parakeet.cpp. Every model. ~1.7x faster on average, up to ~2x. Half the cores.
On X, 2026 ↗](https://x.com/ivanfioravanti/status/2062526685484851440)[
@PulumiCorp integration
Explore how to build and deploy a LLM app using @FlowiseAI and @LocalAI_API with AWS EKS, Pulumi, and TypeScript! Run your models locally or on-prem.
On X, 2024 ↗](https://x.com/PulumiCorp/status/1794038185061663083)[
@enricoros ecosystem
Congrats to @LocalAI_API for launching LocalAGI (Agents), and LocalRecall (Memory). The Local stack is well designed and expanding.
On X, 2025 ↗](https://x.com/enricoros/status/1912401037794898354)[
@UniverseAdam in print
My hardcopies of the official @Raspberry_Pi magazine @TheMagPi have arrived! And they have my Automatic Speech Recognition project based around @NordVPN's Meshnet and a self-hosted @LocalAI_API language model inside.
On X, 2024 ↗](https://x.com/UniverseAdam/status/1779854715519459432)[
@ivanfioravanti community
LocalAI is becoming stronger and better release, after release! Keep pushing @mudler_it and @LocalAI_API
On X, 2026 ↗](https://x.com/ivanfioravanti/status/2038141571678212580)[
@alepiad early days
I'm really excited about the prospect of open-source LLMs. In that respect, take a look at @LocalAI_API, a drop-in replacement for OpenAI's API but serving GGML models right on your own infrastructure.
On X, 2023 ↗](https://x.com/alepiad/status/1654502947697442816)[
@mattapperson builds on
Oh, high there @LocalAI_API, nice to see a terminal based UI for ya! (it's a WIP, but just wanted something cleaner then CURL calls)
On X, 2023 ↗](https://x.com/mattapperson/status/1727390465543041423)
From the team
APEX takes a 35B model from 64.6 GB to 12.2 GB, and from 30.4 to 74.4 tokens a second. Perplexity goes from 6.537 to 7.088. That trade is in the post too, with the command that produced it.
{{/* Pulled from the posts themselves. The cards used to be hand-written, which is how one of them ended up advertising a post that did not exist, and how all three linked to the index instead of the article. */}} {{- range first 3 (where .Site.RegularPages "Section" "blog") }} [
{{ .Date.Format "2 January 2006" }}
{{ .Params.summary | truncate 155 }}
Read the post →]({{%20.RelPermalink%20}}) {{- end }}
Get started
A container on any platform, a DMG on macOS, a binary on Linux, or a chart on Kubernetes. Backends download themselves the first time a model asks for one, so the base install stays small.
Installation guide →Join the Discord
ScriptDockerKubernetes
bash copy
$curl -sSL https://localai.io/install.sh | shdetecting platformlinux/amd64fetching latest release binaryinstalled to/usr/local/bin/local-ai$local-ai run qwen3.5-35b-a3b-apexpulling backend llama-cppAPI ready onhttp://localhost:8080model ready ·74.4 tok/s
$docker run -p 8080:8080 --name local-ai -ti localai/localai:lateststarting local-aidetected: CPU (AVX-512), 32 GB RAMAPI ready onhttp://localhost:8080# Podman works the same way$podman run -p 8080:8080 --name local-ai -ti localai/localai:latest
$kubectl apply -f https://localai.io/install/kubernetes.yamldeployment.apps/local-ai createdservice/local-ai created# or with Helm$helm install local-ai go-skynet/local-ai
macOS: DMGLinux: binaryPodmanHelm chartBuild from source