ContextQMD
Back to Localai

Linux Installation

docs/content/getting-started/linux.md

4.8.02.9 KB
Original Source

Manual Installation

Download Binary

You can manually download the appropriate binary for your system from the releases page:

  1. Go to GitHub Releases
  2. Download the binary for your architecture (amd64, arm64, etc.)
  3. Make it executable:
bash
chmod +x local-ai-*
  1. Run LocalAI:
bash
./local-ai-*

Run your first model

Starting the binary on its own gives you an empty server. To get a working chat right away, run LocalAI with a model name and it will download and serve it from the gallery:

bash
./local-ai-* run qwen3-4b

Once it is ready, open the WebUI at http://localhost:8080 or send a request to the API:

bash
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "qwen3-4b",
  "messages": [{"role": "user", "content": "Hello!"}]
}'

System Requirements

Hardware requirements vary based on:

  • Model size
  • Quantization method
  • Backend used

For performance benchmarks with different backends like llama.cpp, visit this link.

Configuration

After installation, you can:

  • Access the WebUI at http://localhost:8080
  • Configure models in the models directory
  • Customize settings via environment variables or config files

Start LocalAI on demand with systemd

LocalAI accepts a single TCP listener passed through the systemd socket activation protocol. This lets systemd listen on the public port and start LocalAI only when the first client connects.

Create /etc/systemd/system/local-ai.socket:

ini
[Unit]
Description=LocalAI API socket

[Socket]
ListenStream=8080
NoDelay=true

[Install]
WantedBy=sockets.target

Create the matching /etc/systemd/system/local-ai.service:

ini
[Unit]
Description=LocalAI

[Service]
Type=simple
User=localai
Group=localai
ExecStart=/usr/local/bin/local-ai run
WorkingDirectory=/var/lib/local-ai

Adjust the user, binary path, working directory, and model configuration for your installation. Then enable the socket, not the service:

bash
sudo systemctl daemon-reload
sudo systemctl enable --now local-ai.socket

The first connection to port 8080 starts local-ai.service; systemd holds that connection until LocalAI is ready to accept it. LOCALAI_ADDRESS and --address are ignored while an inherited listener is present. LocalAI rejects activation with multiple stream listeners so it cannot silently choose the wrong endpoint.

For a Podman-managed container, configure Podman to preserve and pass the systemd socket file descriptor into the container. The LocalAI process inside the container consumes the same activation protocol.

Next Steps