Back to Docker

docker model start-runner

_vendor/github.com/docker/model-runner/cmd/cli/docs/reference/model_start-runner.md

18.09-release2.3 KB
Original Source

docker model start-runner

<!---MARKER_GEN_START-->

Start Docker Model Runner (Docker Engine only)

Options

NameTypeDefaultDescription
--backendstringSpecify backend (llama.cpp|vllm|diffusers). Default: llama.cpp
--debugboolEnable debug logging
--do-not-trackboolDo not track models usage in Docker Model Runner
--gpustringautoSpecify GPU support (none|auto|cuda|rocm|musa|cann)
--hoststring127.0.0.1Host address to bind Docker Model Runner
--portuint160Docker container port for Docker Model Runner (default: 12434 for Docker Engine, 12435 for Cloud mode)
--proxy-certstringPath to a CA certificate file for proxy SSL inspection
--tlsboolEnable TLS/HTTPS for Docker Model Runner API
--tls-certstringPath to TLS certificate file (auto-generated if not provided)
--tls-keystringPath to TLS private key file (auto-generated if not provided)
--tls-portuint160TLS port for Docker Model Runner (default: 12444 for Docker Engine, 12445 for Cloud mode)
<!---MARKER_GEN_END-->

Description

This command starts the Docker Model Runner without pulling container images. Use this command to start the runner when you already have the required images locally.

For the first-time setup or to ensure you have the latest images, use docker model install-runner instead.