Gpu.Xpu.Inc - Vllm — ContextQMD

--8<-- [start:installation]

vLLM initially supports basic model inference and serving on Intel GPU platform.

--8<-- [end:installation] --8<-- [start:requirements]

Supported Hardware: Intel Data Center GPU, Intel ARC GPU
Dependency: vllm-xpu-kernels: a package provide all necessary vllm custom kernel when running vLLM on Intel GPU platform,
Python: 3.12 !!! warning The provided vllm-xpu-kernels whl is Python3.12 specific so this version is a MUST.

--8<-- [end:requirements] --8<-- [start:set-up-using-python]

There is no extra information on creating a new Python environment for this device.

--8<-- [end:set-up-using-python] --8<-- [start:pre-built-wheels]

Currently, there are no pre-built XPU wheels.

--8<-- [end:pre-built-wheels] --8<-- [start:build-wheel-from-source]

First, install required driver.
Second, install Python packages for vLLM XPU backend building (Intel OneAPI dependencies are installed automatically as part of torch-xpu, see PyTorch XPU get started):
Start from vllm-xpu-kernels v0.1.10, we recommend user upgrade driver to compute runtime 26.18 release, to avoid potential compatibility issue.

bash

git clone https://github.com/vllm-project/vllm.git
cd vllm
pip install --upgrade pip
pip install -v -r requirements/xpu.txt

Then, install the correct Triton package for Intel XPU.

The default triton package (for NVIDIA GPUs) may be installed as a transitive dependency (e.g., via xgrammar). For Intel XPU, you must replace it with triton-xpu:
bash
```
pip uninstall -y triton triton-xpu
pip install triton-xpu==3.7.2 --extra-index-url https://download.pytorch.org/whl/xpu
```
!!! note - triton (without suffix) is for NVIDIA GPUs only. On XPU, using it instead of triton-xpu can cause correctness or runtime issues. - For torch 2.13 (the version used in requirements/xpu.txt), the matching package is triton-xpu==3.7.2. If you use a different version of torch, check the corresponding triton-xpu version in docker/Dockerfile.xpu.
Finally, build and install vLLM XPU backend:

bash

VLLM_TARGET_DEVICE=xpu pip install --no-build-isolation -e . -v

--8<-- [end:build-wheel-from-source] --8<-- [start:pre-built-images]

vLLM offers official Docker images for deployment. The images can be used to run OpenAI compatible server and are available on Docker Hub as vllm/vllm-openai-xpu.

vllm/vllm-openai-xpu:latest — stable release, available starting from v0.26.0
vllm/vllm-openai-xpu:nightly — preview build from the latest development branch, use this if you want the latest features and fixes

bash

docker run --rm \
    --network=host \
    --device /dev/dri:/dev/dri \
    -v /dev/dri/by-path:/dev/dri/by-path \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=$HF_TOKEN" \
    --ipc=host \
    --privileged \
    vllm/vllm-openai-xpu:<tag> \
    --model Qwen/Qwen3-0.6B

To use the docker image as base for development, you can launch it in interactive session through overriding the entrypoint.

???+ console "Commands" bash docker run --rm -it \ --network=host \ --device /dev/dri:/dev/dri \ -v /dev/dri/by-path:/dev/dri/by-path \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=$HF_TOKEN" \ --ipc=host \ --privileged \ --entrypoint /bin/bash \ vllm/vllm-openai-xpu:<tag>

--8<-- [end:pre-built-images] --8<-- [start:build-image-from-source]

bash

docker build -f docker/Dockerfile.xpu -t vllm-xpu-env --shm-size=4g .
docker run -it \
             --rm \
             --network=host \
             --device /dev/dri:/dev/dri \
             -v /dev/dri/by-path:/dev/dri/by-path \
             --ipc=host \
             --privileged \
             vllm-xpu-env

--8<-- [end:build-image-from-source] --8<-- [start:supported-features]

XPU platform supports tensor parallel inference/serving and also supports pipeline parallel as a beta feature for online serving. For pipeline parallel, we support it on single node with mp as the backend. For example, a reference execution like following:

bash

vllm serve facebook/opt-13b \
     --dtype=bfloat16 \
     --max_model_len=1024 \
     --distributed-executor-backend=mp \
     --pipeline-parallel-size=2 \
     -tp=8

By default, a ray instance will be launched automatically if no existing one is detected in the system, with num-gpus equals to parallel_config.world_size. We recommend properly starting a ray cluster before execution, referring to the examples/ray_serving/run_cluster.sh helper script.

--8<-- [end:supported-features] --8<-- [start:distributed-backend]

XPU platform uses torch-ccl for torch<2.8 and xccl for torch>=2.8 as distributed backend, since torch 2.8 supports xccl as built-in backend for XPU.

--8<-- [end:distributed-backend]