Localai Assistant - Localai

+++ disableToc = false title = "LocalAI Assistant" weight = 23 url = '/features/localai-assistant' +++

LocalAI Assistant is an admin-only chat modality. When enabled on a chat session, the conversation is wired to an in-process MCP server that exposes LocalAI's own admin/management surface as tools. You can install models, manage backends, edit model configs and check system status by chatting, with no REST calls or YAML edits.

The same MCP server is published as a Go package and can also be served over stdio to control a remote LocalAI instance from outside (e.g. from a desktop MCP host, Cursor, or mcphost).

Enabling the assistant in chat

Open the chat UI as an admin user and pick a chat-capable model in the model selector. The header shows a Manage toggle - flip it on, and a Manage mode badge appears next to the chat title. Starter chips ("What is installed?", "Install a chat model", "Show system status", "Update a backend") help you get going.

The home page also exposes a Manage by chat CTA that opens a fresh chat already in Manage mode.

Once on, try:

Install Qwen 3 chat

The assistant searches the gallery, lists candidates, asks you to pick, summarises the install, and waits for your confirmation before calling install_model. While the install runs, it polls progress and reports the outcome.

Disabling the feature

Either toggle it off in Settings → LocalAI Assistant (takes effect without restart), or hard-disable at startup:

bash

LOCALAI_DISABLE_ASSISTANT=true local-ai run

When disabled, the chat handler refuses requests with metadata.localai_assistant=true and returns 503. The Manage toggle is hidden in the UI.

Security model

The chat toggle is hidden for non-admin users.
The chat handler re-checks admin role at request time even when auth is configured to skip the assistant feature gate (defense in depth).
The MCP server itself is in-process - there is no localhost loopback, no synthetic API key, and no extra TCP socket.
Mutating tools (install_model, delete_model, edit_model_config, upgrade_backend, …) are guarded by a system-prompt rule that requires the LLM to confirm the action with the user before calling them. There is no separate code-side preview/apply step.

Standalone stdio MCP server

You can run the same admin tool surface as a stdio MCP server pointed at any LocalAI HTTP API:

bash

local-ai mcp-server --target http://remote.localai:8080 --api-key <admin-key>
# read-only mode - skips registration of every mutating tool
local-ai mcp-server --target http://remote.localai:8080 --read-only

Useful for hooking LocalAI admin into Claude Desktop, Cursor, or any MCP host. The tool catalog is identical to the in-process variant.

Tool catalog

Read-only: gallery_search, list_installed_models, list_galleries, list_backends, list_known_backends, get_job_status, get_model_config, vram_estimate, system_info, list_nodes.

Mutating (require user confirmation per the assistant's safety prompt): install_model, import_model_uri, delete_model, install_backend, upgrade_backend, edit_model_config, reload_models, toggle_model_state, toggle_model_pinned.

Adding new tools or skills

The MCP server lives at pkg/mcp/localaitools/. Tools are registered in tools_*.go; skill prompts (the markdown the LLM sees) are embedded from prompts/. To add a new admin tool, add a method to the LocalAIClient interface plus implementations in inproc/ and httpapi/, then register the tool and add or update a skill prompt. See .agents/localai-assistant-mcp.md for the full contributor checklist.