LongCat-2.0

Deployment

For all methods and hardware platforms, see the official SGLang installation guide. LongCat-2.0 support is on SGLang main; use a nightly wheel or rolling nightly Docker image until the next tagged release includes it. The two paths below match the Python / Docker toggle in the command panel.

pip install --upgrade pip
pip install uv

# Choose the nightly wheel index for your CUDA runtime.
SGLANG_WHL_INDEX=https://docs.sglang.ai/whl/cu130  # B300 / CUDA 13
# SGLANG_WHL_INDEX=https://docs.sglang.ai/whl/cu129  # CUDA 12.9

uv pip install --prerelease=allow --extra-index-url "${SGLANG_WHL_INDEX}" "sglang[all]"

Then run the Python output of the command panel below in that environment.

# Choose the rolling nightly image for your hardware.
SGLANG_DOCKER_IMAGE=lmsysorg/sglang:dev-cu13  # B300 / CUDA 13
# SGLANG_DOCKER_IMAGE=lmsysorg/sglang:dev  # Other supported hardware

docker pull "${SGLANG_DOCKER_IMAGE}"

For how to launch the image, see Install -> Method 3: Using Docker. Substitute the inner sglang serve ... with what the command generator below produces.

Pick your hardware + recipe to generate the launch command. LongCat-2.0 currently exposes one model-card-aligned serving strategy:

• Balanced - the validated B300 recipe and the 2-node H200/B200/H20 topology use TP/EP parallelism with LongCat sparse attention prefill.

import { Deployment } from "/src/snippets/_deployment.jsx"; import { config } from "/src/snippets/configs/meituan-longcat/longcat-2.0.jsx"; import { benchmarks } from "/src/snippets/configs/meituan-longcat/longcat-2.0-benchmarks.jsx";

Playground

The Playground is where you experiment with SGLang features beyond the verified matrix. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing.

import { Playground } from "/src/snippets/_playground.jsx";

1. Model Introduction

LongCat-2.0-FP8 is the FP8 checkpoint of Meituan LongCat-2.0, a large sparse Mixture-of-Experts language model with 1.6T total parameters and about 48B activated parameters per token. It combines LongCat Sparse Attention (LSA), expert parallel MoE layers, and an n-gram/token-table embedding path for serving long-context workloads efficiently.

Resources: LongCat-2.0-FP8.

2. Configuration Tips

• Remote code. Use --trust-remote-code for the Hugging Face checkpoint.
• Topology. The 8x B300 recipe uses TP=8 and EP=8. H200, B200, and H20 use a 2-node 16 GPU layout with TP=16 and EP=16; the command panel injects the multi-node rank flags for you.
• LongCat sparse attention. Keep --nsa-prefill-backend fa3 with --chunked-prefill-size 2048 for the model-card-aligned prefill path.
• Memory. The recipe uses --kv-cache-dtype bfloat16 and starts at --mem-fraction-static 0.92. Tune memory only after the generated command launches cleanly on your cluster.
• Weight loading. --model-loader-extra-config '{"enable_multithread_load":true,"num_threads":12}' loads checkpoint shards in parallel and reduces startup time.
• FP8 backend selection. Do not pass --fp8-gemm-runner-backend manually. SGLang selects the correct backend for the LongCat FP8 scale layout.
• Host, port, and ranks. Use the command panel environment fields for HOST_IP, PORT, NODE0_IP, and NODE_RANK instead of hardcoding them in the recipe.

3. Advanced Usage

3.1 Test the deployment

curl http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meituan-longcat/LongCat-2.0-FP8",
    "messages": [
      {"role": "user", "content": "A shop has 17 apples and sells 8. Then it buys 6 more. How many apples are there? Answer with only the final number."}
    ],
    "max_tokens": 32,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

15

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="meituan-longcat/LongCat-2.0-FP8",
    messages=[
        {
            "role": "user",
            "content": "Solve: A shop has 17 apples and sells 8, then buys 6 more. Answer with only the final number.",
        }
    ],
    max_tokens=32,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)

print(response.choices[0].message.content)

15

4. Validation

The B300 recipe was validated with meituan-longcat/LongCat-2.0-FP8 on 8x B300 using the command generated above.

CUDA graph was enabled, and decode CUDA graph capture completed successfully during serving validation.