docs/09 - Docker.md
Docker Compose is a way of installing and launching the web UI in an isolated Ubuntu image using only a few commands.
You need Docker Compose v2.17 or higher:
~$ docker compose version
Docker Compose version v2.21.0
Installation instructions: https://docs.docker.com/engine/install/
For NVIDIA GPUs, you also need the NVIDIA Container Toolkit.
There are four Docker variants available under docker/:
| Directory | GPU | Notes |
|---|---|---|
docker/nvidia | NVIDIA | Requires NVIDIA Container Toolkit |
docker/amd | AMD | Requires ROCm-compatible GPU |
docker/intel | Intel Arc | Beta support |
docker/cpu | None | CPU-only inference |
To launch (using NVIDIA as an example):
cd textgen/docker/nvidia
cp ../.env.example .env
# Optionally edit .env to customize ports, TORCH_CUDA_ARCH_LIST, etc.
docker compose up --build
The web UI will be available at http://localhost:7860.
Create a user_data/ directory next to the docker-compose.yml to persist your models, characters, presets, and settings between container rebuilds:
mkdir -p user_data
This directory is mounted into the container at runtime. You can place a CMD_FLAGS.txt inside it to pass persistent flags to the web UI (e.g., --api).
Models can be downloaded through the web UI's “Model” tab once it's running, and they will be saved to user_data/models/.
An external repository maintains a docker wrapper for this project as well as several pre-configured 'one-click' docker compose variants. It can be found at: Atinoda/text-generation-webui-docker.