Runtime Provider Integration

How llmfit detects and talks to Ollama, llama.cpp, Docker Model Runner, LM Studio, and remote instances.

Runtime provider integration

llmfit supports multiple local runtime providers:

Ollama (daemon/API based pulls)
llama.cpp (direct GGUF downloads from Hugging Face + local cache detection)
MLX (Apple Silicon / mlx-community model cache + optional server) — MLX downloads map to mlx-community/* repos on HuggingFace, not the original model publisher
Docker Model Runner (Docker Desktop's built-in model serving)
LM Studio (local model server with REST API for model management + downloads)

When more than one compatible provider is available for a model, pressing d in the TUI opens a provider picker modal.

Ollama integration

llmfit integrates with Ollama to detect which models you already have installed and to download new ones directly from the TUI.

Requirements

Ollama must be installed and running (ollama serve or the Ollama desktop app)
llmfit connects to http://localhost:11434 (Ollama's default API port)
No configuration needed — if Ollama is running, llmfit detects it automatically

Remote Ollama instances

To connect to Ollama running on a different machine or port, set the OLLAMA_HOST environment variable:

# Connect to Ollama on a specific IP and port
OLLAMA_HOST="http://192.168.1.100:11434" llmfit

# Connect via hostname  
OLLAMA_HOST="http://ollama-server:666" llmfit

# Works with all TUI and CLI commands
OLLAMA_HOST="http://192.168.1.100:11434" llmfit --cli
OLLAMA_HOST="http://192.168.1.100:11434" llmfit fit --perfect -n 5

This is useful for:

Running llmfit on one machine while Ollama serves from another (e.g., GPU server + laptop client)
Connecting to Ollama running in Docker containers with custom ports
Using Ollama behind reverse proxies or load balancers

How it works

On startup, llmfit queries GET /api/tags to list your installed Ollama models. Each installed model gets a green ✓ in the Inst column of the TUI. The system bar shows Ollama: ✓ (N installed).

When you press d on a model, llmfit sends POST /api/pull to Ollama to download it. The row highlights with an animated progress indicator showing download progress in real-time. Once complete, the model is immediately available for use with Ollama.

If Ollama is not running, Ollama-specific operations are skipped; the TUI still supports other providers like llama.cpp where available.

llama.cpp integration

llmfit integrates with llama.cpp as a runtime/download provider in both TUI and CLI.

Requirements:

llama-cli or llama-server available in PATH (for runtime detection)
network access to Hugging Face for GGUF downloads

How it works:

llmfit maps HF models to known GGUF repos (with heuristic fallbacks)
downloads GGUF files into the local llama.cpp model cache
marks models installed when matching GGUF files are present locally

Environment variables

Variable	Default	Description
`LLAMA_CPP_PATH`	(none)	Directory containing llama.cpp binaries (`llama-cli`, `llama-server`). Checked before `PATH` lookup.
`LLAMA_SERVER_PORT`	`8080`	Port used when probing a running `llama-server` health endpoint for runtime detection.

If llama.cpp is installed in a non-standard location, set LLAMA_CPP_PATH so llmfit can find it without requiring it in your PATH.

Docker Model Runner integration

llmfit integrates with Docker Model Runner, Docker Desktop's built-in model serving feature.

Requirements:

Docker Desktop with Model Runner enabled
Default endpoint: http://localhost:12434

How it works:

llmfit queries GET /engines to list models available in Docker Model Runner
models are matched to the HF database using Ollama-style tag mapping (Docker Model Runner uses ai/<tag> naming)
pressing d in the TUI pulls via docker model pull

Remote Docker Model Runner instances

To connect to Docker Model Runner on a different host or port, set the DOCKER_MODEL_RUNNER_HOST environment variable:

DOCKER_MODEL_RUNNER_HOST="http://192.168.1.100:12434" llmfit

LM Studio integration

llmfit integrates with LM Studio as a local model server with built-in model download capabilities.

Requirements:

LM Studio must be running with its local server enabled
Default endpoint: http://127.0.0.1:1234

How it works:

llmfit queries GET /v1/models to list models available in LM Studio
pressing d in the TUI triggers a download via POST /api/v1/models/download
download progress is tracked by polling GET /api/v1/models/download-status
LM Studio accepts HuggingFace model names directly, so no name mapping is needed

Remote LM Studio instances

To connect to LM Studio on a different host or port, set the LMSTUDIO_HOST environment variable:

LMSTUDIO_HOST="http://192.168.1.100:1234" llmfit

API authentication

If your LM Studio instance has Require API Key enabled (required for MCP server access), set the LMSTUDIO_API_KEY environment variable to provide a Bearer token with all requests:

export LMSTUDIO_API_KEY="your-api-key-here"
llmfit

Model name mapping

llmfit's database uses HuggingFace model names (e.g. Qwen/Qwen2.5-Coder-14B-Instruct) while Ollama uses its own naming scheme (e.g. qwen2.5-coder:14b). llmfit maintains an accurate mapping table between the two so that install detection and pulls resolve to the correct model. Each mapping is exact — qwen2.5-coder:14b maps to the Coder model, not the base qwen2.5:14b.