PaddleOCR-VL

Deployment

For all methods and hardware platforms, see the official SGLang installation guide. The two paths below match the Python / Docker toggle in the command panel.

pip install --upgrade pip
pip install uv
uv pip install --prerelease=allow sglang

Then run the Python output of the command panel below in that environment.

docker pull lmsysorg/sglang:dev

For how to launch the image, see Install → Method 3: Using Docker. Substitute the inner sglang serve ... with what the command generator below produces.

Pick a release and your hardware to generate the launch command. The model is 0.9B and single-GPU, so there is one serving recipe per platform; the axis that actually moves cost is Page Resolution, which caps how many image tokens one page is worth.

import { Deployment } from "/src/snippets/_deployment.jsx"; import { config } from "/src/snippets/configs/PaddlePaddle/paddleocr-vl.jsx";

Playground

Use the Playground to layer tensor parallelism on top of the selected deployment cell. At this size TP is a latency knob, not a capacity one — the weights fit on one GPU.

import { Playground } from "/src/snippets/_playground.jsx";

1. Model Introduction

PaddleOCR-VL is Baidu's compact document-parsing vision-language model: a NaViT-style dynamic-resolution SigLIP vision encoder feeding an ERNIE-4.5-0.3B language backbone, 0.9B parameters in total, released under Apache 2.0. It recognizes text, tables, formulas, charts and seals across 109 languages, and is small enough that a single GPU serves it comfortably.

SGLang serves this model, which is the recognition stage of PaddleOCR's document-parsing pipeline — not the pipeline itself. Layout and region detection, cropping, reading order and the assembly of markdown or JSON stay in PaddleOCR, which calls a model endpoint once per region. That split is why the model is prompt-conditioned rather than instruction-following: the caller decides whether a crop is a table, a formula or a chart (§3.1), and SGLang is the backend that answers.

All three releases share an identical config.json (same PaddleOCRVLForConditionalGeneration architecture, same tower and backbone dimensions), so one SGLang recipe serves every variant and only the model path changes.

Recommended generation: greedy decoding (temperature=0) with a per-page max_tokens budget — the model card uses 512 for a single region and the reference server allows more for a full page. These are informational; do not hardcode them in library code.

Resources: Hugging Face · PaddleOCR on GitHub

2. Configuration Tips

• Page resolution is the main cost knob. The vision tower and the prefill both scale with the patch count of a page. max_pixels is expressed in 28x28 units (patch size 14 with a 2x2 merge), so max_pixels / 784 is the image-token budget per page. The checkpoint's own default is 1280 tokens; the Page Resolution selector in the Deploy panel emits the corresponding --mm-process-config value. Lower it for clean born-digital PDFs, raise it for dense scans and small print.
• Prompt selects the task. PaddleOCR-VL is prompt-conditioned rather than instruction-following — use the exact task strings in §3.1. A free-form question will not behave like a chat model.
• Leave --trust-remote-code off. The checkpoints ship their own configuration_paddleocr_vl.py / processing_paddleocr_vl.py, but transformers 5.12 supports paddleocr_vl natively — and the bundled remote image processor is the slower of the two implementations (measured 87.4 ms vs 39.1 ms per 1080p page). Passing the flag pins SGLang to the remote copy. Serving without it produced byte-identical OCR output on every page we checked and about 5% more requests per second at 32-way concurrency.
• Preprocessing is parallelized for you. A full-resolution page costs tens of milliseconds of CPU to resize, normalize and patchify, which caps throughput long before the GPU saturates, so this model runs the image processor across several workers by default. --mm-processor-worker-num overrides the count; raising it past the default did not help in our measurements.
• Keep the radix cache on for repeated pages. Unlike whole-document batch OCR over unique scans, a workload that re-asks about the same page (different task prompts on one image) reuses the image prefix. Add --disable-radix-cache only if every request carries a different page.
• The saturated-throughput flags earn their place. A page is ~2700 tokens, so the default 8192-token prefill budget packs only three of them into a forward. Raising it to 16384 and letting decode ride along in the same batch (--enable-mixed-chunk, --num-continuous-decode-steps 2) measured +11% requests per second at 32-way concurrency and cut queued TTFT by 23%, with single-stream latency unchanged. Measured on an H200; on a smaller card lower --chunked-prefill-size until it fits.
• Prefill CUDA graph is on for this model. SGLang normally switches the breakable prefill graph off for every multimodal architecture; PaddleOCR-VL is allowlisted back in, which is worth 16.1 ms → 11.5 ms of single-stream TTFT on text-only prompts. Image-carrying batches are rejected at graph replay and run eager, so this helps mixed and text traffic, not pure page parsing. No flag needed.
• Tensor parallelism is optional. The weights are under 2 GB in BF16; TP>1 only shortens the vision-encoder and prefill critical path, at the cost of a collective per layer. Measure before adopting it.
• Context length. The backbone advertises 131072 positions, but a parsed page rarely needs more than a few thousand tokens. The recipe pins --context-length 16384 so the KV pool stays small and concurrency stays high; raise it only if you batch many pages into one request.

Measured on one H200

One 1080p page (~2700 image tokens) in, 128 tokens out, prefix cache disabled, median TTFT:

Throughput at saturation is bound by the vision tower, which runs full attention over every patch of the page — so the Page Resolution selector is the lever that moves it, not tensor parallelism.

3. Advanced Usage

3.1 Task prompts

PaddleOCR-VL exposes its capabilities through a small set of fixed prompts. Send the prompt as the text part and the page as the image part of the same user turn.

Structured tasks answer in the model's own markup, not HTML: Table Recognition: returns OTSL-style cell tokens (<fcel> per cell, <nl> per row), so a caller that wants HTML or markdown converts them itself.

<fcel>Methods<fcel>R<fcel>P<fcel>F<fcel>FPS<nl><fcel>SegLink [26]<fcel>70.0<fcel>86.0<fcel>77.0<fcel>8.9<nl><fcel>PixelLink [4]<fcel>73.2<fcel>83.0<fcel>77.8<fcel>-<nl><fcel>TextSnake [18]<fcel>73.9<fcel>83.2<fcel>78.3<fcel>1.1<nl>
... (one <fcel> per cell, one <nl> per row, to the end of the table)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="PaddlePaddle/PaddleOCR-VL-1.6",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "OCR:"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://raw.githubusercontent.com/PaddlePaddle/PaddleOCR/release/2.9/doc/imgs_en/img_12.jpg"
                    },
                },
            ],
        }
    ],
    max_tokens=2048,
)

print(response.choices[0].message.content)

ACKNOWLEDGEMENTS
We would like to thank all the designers and contributors who have been involved in the
production of this book; their contributions have been indispensable to its creation. We would
also like to express our gratitude to all the producers for their invaluable opinions and
assistance throughout this project. And to the many others whose names are not credited but
have made specific input in this book, we thank you for your continuous support.

3.2 Parsing a multi-page document

The model parses one page per request. Render each page to an image, then fan the pages out concurrently — SGLang batches the vision encoders of in-flight requests into a single forward, so concurrency is what keeps the GPU busy on a model this small.

import base64
from concurrent.futures import ThreadPoolExecutor

import pymupdf
from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")


def render(page, dpi=200):
    pixmap = page.get_pixmap(dpi=dpi)
    return base64.b64encode(pixmap.tobytes("png")).decode("ascii")


def parse(page_png_b64):
    response = client.chat.completions.create(
        model="PaddlePaddle/PaddleOCR-VL-1.6",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "OCR:"},
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": f"data:image/png;base64,{page_png_b64}"
                        },
                    },
                ],
            }
        ],
        max_tokens=2048,
    )
    return response.choices[0].message.content


document = pymupdf.open("your_document.pdf")
pages = [render(page) for page in document]

with ThreadPoolExecutor(max_workers=16) as pool:
    for index, text in enumerate(pool.map(parse, pages)):
        print(f"--- page {index + 1} ---")
        print(text)

--- page 1 ---
(a) Total-Text
(b) Total-Text
(c) CTW1500
(d) CTW1500

Figure 8. Visual experimental results. The blue contours are boundary proposals, and the
green contours are final detection boundaries.
Table 6. Experimental results on CTW-1500.
Methods
Ext
R
P
F
FPS
TextSnake [18]
Syn
85.3
67.9
75.6
-
... (page continues)
--- page 2 ---
... (one block per page, in page order)