FP8 ViT Encoder Attention

For visual understanding workloads with large images (e.g. QHD, 4K) and relatively short text prompts/generation, the ViT encoder attention can become a significant bottleneck, especially when the text model is quantized (e.g. NVFP4). vLLM supports optional FP8 quantization for ViT encoder attention via FlashInfer cuDNN on NVIDIA GPUs and AITER on AMD GPUs. Q/K/V are quantized on-the-fly to FP8 before the attention call.

!!! note - Currently supports Qwen3-VL family models only (qwen3_vl, qwen3_vl_moe, qwen3_5, qwen3_5_moe, and other models using Qwen3 ViT). - Dynamic scaling is not compatible with ViT full CUDA graphs. - Performance gains are mostly visible at QHD/4K resolutions or multi-image requests. Smaller images may see no speedup due to quantization overhead (3 quantization kernel launches + un-padding). - FP8 tensor-core speedup is more pronounced on GB300 than GB200. - On ROCm, packed variable-length image and video batches are supported by AITER's native varlen FP8 attention.

Requirements

• NVIDIA: FlashInfer cuDNN backend with cuDNN >= 9.17.1.
• AMD: AITER with flash_attn_varlen_fp8_pertensor_func support on gfx942 (MI300 series) or gfx950 (MI350 series).

Usage

Enable FP8 ViT attention by passing --mm-encoder-attn-dtype fp8 and selecting the backend for the current platform:

vllm serve $MODEL \
    --mm-encoder-attn-backend FLASHINFER \
    --mm-encoder-attn-dtype fp8

# AMD ROCm
vllm serve $MODEL \
    --mm-encoder-attn-backend ROCM_AITER_FA \
    --mm-encoder-attn-dtype fp8

By default (no scale file), dynamic scaling is used: a 16-entry circular buffer of observed Q/K/V amax values drives per-forward scale updates. This matches BF16 accuracy without any calibration but adds a small per-forward overhead.

Calibrate-Once, Reuse Workflow (Recommended)

For production, calibrate static scales on a representative dataset once and reuse them to avoid the dynamic overhead:

# Step 1: calibrate and save scales (runs dynamic scaling for 16 passes,
# then dumps the learned scales to JSON).
vllm bench mm-processor \
    --model $MODEL --mm-encoder-attn-backend $MM_ATTN_BACKEND \
    --mm-encoder-attn-dtype fp8 \
    --mm-encoder-fp8-scale-save-path /path/to/scales.json \
    --dataset-name hf --dataset-path lmarena-ai/VisionArena-Chat \
    --num-prompts 100

# Step 2: serve with static scales (no dynamic overhead).
vllm serve $MODEL \
    --mm-encoder-attn-backend $MM_ATTN_BACKEND \
    --mm-encoder-attn-dtype fp8 \
    --mm-encoder-fp8-scale-path /path/to/scales.json

Saved scales are multiplied by --mm-encoder-fp8-scale-save-margin (default 1.5) to leave headroom against activation outliers not present in the calibration set. The default has been validated to generalize across datasets (e.g. VisionArena-Chat calibration maintains BF16 accuracy on ChartQA).

Scale File Format

{
    "visual.blocks.0.attn.attn": {"q": 224.0, "k": 198.0, "v": 210.0},
    "visual.blocks.1.attn.attn": {"q": 218.0, "k": 195.0, "v": 207.0}
}

Keys q_scale / k_scale / v_scale are accepted as aliases.

Performance

Core cuDNN attention kernel (PyTorch profiler, cudnn_generated_fort_native_sdpa_sm100_flash_fprop, head_dim=128, seq_len=8192):

End-to-end encoder forward time (Qwen3-VL-30B-A3B-Instruct on GB200, 3 images/request):

Crossover is around FullHD with 3 images/request. At QHD and above, FP8 wins.

Complete AITER attention call on MI300X (BF16 input, 16 heads, head_dim=72; FP8 includes Q/K/V quantization):

Accuracy

ChartQA, Qwen3-VL-8B-Instruct, 500 samples. FP8 static uses scales calibrated on VisionArena-Chat (with default 1.5x margin):

All three configurations match within statistical noise, confirming that static scales calibrated on one dataset generalize to another.