docs/development.md
Install prerequisites:
PATH is recommended, especially on WindowsFor pure Go iteration against an existing native payload, run Ollama from the repository root:
go run . serve
[!NOTE] Ollama includes native code compiled with CGO. From time to time these data structures can change and CGO can get out of sync resulting in unexpected crashes. You can force a full build of the native code by running
go clean -cachefirst.
For a fresh checkout, or after changing native code, build from the repository root. On macOS arm64, this builds Metal inference. On all other platforms this builds CPU-only inference. It builds the Go binary at the repository root and installs the native runtime payload under build/lib/ollama.
cmake -B build .
cmake --build build --parallel 8
./ollama serve
To install into a standard prefix layout:
cmake --install build --prefix /path/to/install
On all platforms except macOS arm64, to build GPU backends select the backends explicitly:
cmake -B build . -DOLLAMA_LLAMA_BACKENDS="cuda_v13;vulkan"
cmake --build build --parallel 8
Supported backend values are cuda_v12, cuda_v13, rocm_v7_1, rocm_v7_2, vulkan, cuda_jetpack5, and cuda_jetpack6.
Use standard CMake architecture overrides to narrow GPU builds for local hardware:
# CUDA
cmake -B build . -DOLLAMA_LLAMA_BACKENDS=cuda_v13 -DCMAKE_CUDA_ARCHITECTURES=native
# ROCm / HIP
cmake -B build . -DOLLAMA_LLAMA_BACKENDS=rocm_v7_2 -DCMAKE_HIP_ARCHITECTURES=gfx1100
You can tune GGML build options by setting GGML_* values during configure. For example, to build CUDA v12 for Pascal without flash attention kernels:
cmake -B build . -DOLLAMA_LLAMA_BACKENDS=cuda_v12 -DCMAKE_CUDA_ARCHITECTURES=61 -DGGML_CUDA_FA=OFF
Additional prerequisites:
MLX Metal requires the Metal toolchain. Install Xcode first, then:
xcodebuild -downloadComponent MetalToolchain
Additional prerequisites:
For Ninja builds, run CMake from a Developer PowerShell/Command Prompt or another shell where the Visual Studio compiler is available.
Building for Vulkan requires VULKAN_SDK environment variable:
PowerShell
powershell$env:VULKAN_SDK="C:\VulkanSDK\<version>"CMD
cmdset VULKAN_SDK=C:\VulkanSDK\<version>
Windows ARM does not support additional acceleration libraries at this time.
Additional prerequisites:
sudo apt install vulkan-sdk (Ubuntu/Debian) or sudo dnf install vulkan-sdk (Fedora/CentOS)sudo apt install libopenblas-dev liblapack-dev liblapacke-dev (Ubuntu/Debian)[!IMPORTANT] Ensure prerequisites are in
PATHbefore running CMake.
The MLX engine enables running safetensor based models. On macOS arm64, MLX is enabled by default. On other platforms, MLX backends are selected with OLLAMA_MLX_BACKENDS.
Requires CUDA 13+ and cuDNN 9+.
cmake -B build . -DOLLAMA_MLX_BACKENDS=cuda_v13
cmake --build build --parallel 8
To build against a local checkout of MLX and/or MLX-C (useful for development), set environment variables before running CMake:
export OLLAMA_MLX_SOURCE=/path/to/mlx
export OLLAMA_MLX_C_SOURCE=/path/to/mlx-c
On macOS arm64:
OLLAMA_MLX_SOURCE=../mlx OLLAMA_MLX_C_SOURCE=../mlx-c cmake -B build .
cmake --build build --parallel 8
For CUDA:
$env:OLLAMA_MLX_SOURCE="../mlx"
$env:OLLAMA_MLX_C_SOURCE="../mlx-c"
cmake -B build . -DOLLAMA_MLX_BACKENDS=cuda_v13
cmake --build build --parallel 8
docker build .
docker build --build-arg FLAVOR=rocm .
To run tests, use go test:
go test ./...
Ollama looks for native helper binaries and acceleration libraries in installed and local development layouts:
../lib/ollama for standard installs where ollama is under bin/./lib/ollama for Windows release-style payloads and local dist output. for macOS release artifacts that colocate helpers with ollamabuild/lib/ollama and dist/<platform>/lib/ollama for local development buildsIf the libraries are not found, Ollama will not run with any acceleration libraries.