Back to Sglang

Ornith-1.0

docs_new/cookbook/autoregressive/DeepReinforce/Ornith-1.0.mdx

0.5.1610.6 KB
Original Source

Deployment

<a id="install" /> <Accordion title="Install SGLang">

Ornith-1.0 model cards recommend SGLang >=0.5.9. The Deploy panel below emits the base serve command; the reasoning and tool-call parsers from the model-card quickstarts (--reasoning-parser qwen3 for <think>...</think> traces, --tool-call-parser qwen3_coder for Qwen-style XML tool calls) are added on top via the Playground.

<Tabs> <Tab title="Python (pip / uv)">
bash
pip install --upgrade pip
pip install uv
uv pip install "sglang>=0.5.9"

Then run the Python output of the command panel below in that environment.

</Tab> <Tab title="Docker">
bash
docker pull lmsysorg/sglang:latest

For how to launch the image, see Install -> Method 3: Using Docker. Substitute the inner sglang serve ... with what the command generator below produces.

</Tab> </Tabs> </Accordion>

Pick an Ornith checkpoint to generate the launch command. The non-FP8 397B recipe requires an H200 single node in this matrix. The 397B-FP8 recipe is available on H100 and H200 with TP=8; H100 also supports the 35B and 9B checkpoints. The 35B recipes use tensor parallelism 2 in this matrix. The 9B checkpoint is dense and serves on a single GPU by default; the command panel makes that default explicit with --tp 1.

import { Deployment } from "/src/snippets/_deployment.jsx"; import { config } from "/src/snippets/configs/deepreinforce-ai/ornith-1.0.jsx";

<Deployment config={config} />

Playground

The Playground layers SGLang features on top of whichever cell the Deploy panel is showing — only your overrides change, and any change flips the badge to Not Verified until the new configuration is run end-to-end.

For Ornith-1.0 the knobs are the reasoning and tool-call parsers:

  • Reasoning Parser appends --reasoning-parser qwen3. Ornith emits <think>...</think> traces; with this on, SGLang surfaces them as message.reasoning_content instead of leaving the tags inline in content.
  • Tool Call Parser appends --tool-call-parser qwen3_coder, so Qwen-style XML tool calls are returned as OpenAI-compatible tool_calls.

import { Playground } from "/src/snippets/_playground.jsx";

<Playground config={config} />

1. Model Introduction

Ornith-1.0 is DeepReinforce's self-improving open-source model family for agentic coding. The model cards describe the family as post-trained on top of Gemma 4 and Qwen 3.5, and the collection currently includes 397B, 35B, and 9B repos plus FP8 and GGUF variants. The model cards report results on Terminal-Bench 2.1, SWE-Bench, NL2Repo, ClawEval, and SWE Atlas benchmarks.

Key Features:

  • Agentic coding specialization: the model cards describe Ornith-1.0 as specialized for agentic coding and report coding-agent benchmark results.
  • Self-improving training: the model cards state that Ornith-1.0 uses reinforcement learning to optimize both solution rollouts and the scaffold that drives those rollouts.
  • Reasoning model behavior: assistant responses begin with a <think>...</think> reasoning block before the final answer; enable the --reasoning-parser qwen3 toggle in the Playground to split it into reasoning_content.
  • Tool calling: emits Qwen-style XML tool calls; enable the --tool-call-parser qwen3_coder toggle in the Playground.
  • Long context: model-card recipes use --context-length 262144.
  • MIT license: the Hugging Face repos are released under MIT.

Available Models:

<table style={{width: "100%", borderCollapse: "collapse", tableLayout: "fixed"}}> <colgroup> <col style={{width: "32%"}} /> <col style={{width: "18%"}} /> <col style={{width: "18%"}} /> <col style={{width: "32%"}} /> </colgroup> <thead> <tr style={{borderBottom: "2px solid #d55816"}}> <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Model</th> <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Format</th> <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Deploy Panel</th> <th style={{textAlign: "left", padding: "10px 12px", fontWeight: 700}}>Notes</th> </tr> </thead> <tbody> <tr> <td style={{padding: "9px 12px", fontWeight: 500}}><a href="https://huggingface.co/deepreinforce-ai/Ornith-1.0-397B">deepreinforce-ai/Ornith-1.0-397B</a></td> <td style={{padding: "9px 12px"}}>BF16</td> <td style={{padding: "9px 12px"}}>H200 only</td> <td style={{padding: "9px 12px"}}>Flagship 397B MoE checkpoint; model-card baseline uses TP=8 on an H200 single node.</td> </tr> <tr> <td style={{padding: "9px 12px", fontWeight: 500}}><a href="https://huggingface.co/deepreinforce-ai/Ornith-1.0-397B-FP8">deepreinforce-ai/Ornith-1.0-397B-FP8</a></td> <td style={{padding: "9px 12px"}}>FP8</td> <td style={{padding: "9px 12px"}}>H100 / H200</td> <td style={{padding: "9px 12px"}}>FP8 repo in the collection; the deploy command uses this repo id with TP=8.</td> </tr> <tr> <td style={{padding: "9px 12px", fontWeight: 500}}><a href="https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B">deepreinforce-ai/Ornith-1.0-35B</a></td> <td style={{padding: "9px 12px"}}>BF16</td> <td style={{padding: "9px 12px"}}>H100 / H200</td> <td style={{padding: "9px 12px"}}>35B MoE checkpoint; the deploy command uses TP=2.</td> </tr> <tr> <td style={{padding: "9px 12px", fontWeight: 500}}><a href="https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-FP8">deepreinforce-ai/Ornith-1.0-35B-FP8</a></td> <td style={{padding: "9px 12px"}}>FP8</td> <td style={{padding: "9px 12px"}}>H100 / H200</td> <td style={{padding: "9px 12px"}}>FP8 repo in the collection; the deploy command uses this repo id with TP=2.</td> </tr> <tr> <td style={{padding: "9px 12px", fontWeight: 500}}><a href="https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B">deepreinforce-ai/Ornith-1.0-9B</a></td> <td style={{padding: "9px 12px"}}>BF16</td> <td style={{padding: "9px 12px"}}>H100 / H200</td> <td style={{padding: "9px 12px"}}>Dense 9B checkpoint; the model card describes it as designed for efficient single-GPU deployment.</td> </tr> <tr> <td style={{padding: "9px 12px", fontWeight: 500}}><a href="https://huggingface.co/deepreinforce-ai/Ornith-1.0-35B-GGUF">deepreinforce-ai/Ornith-1.0-35B-GGUF</a></td> <td style={{padding: "9px 12px"}}>GGUF</td> <td style={{padding: "9px 12px"}}>No</td> <td style={{padding: "9px 12px"}}>Listed for completeness; GGUF targets llama.cpp-style local inference, not the SGLang server recipe here.</td> </tr> <tr> <td style={{padding: "9px 12px", fontWeight: 500}}><a href="https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B-GGUF">deepreinforce-ai/Ornith-1.0-9B-GGUF</a></td> <td style={{padding: "9px 12px"}}>GGUF</td> <td style={{padding: "9px 12px"}}>No</td> <td style={{padding: "9px 12px"}}>Listed for completeness; the model card shows llama.cpp and Ollama examples for the GGUF build.</td> </tr> </tbody> </table>

License: MIT

Resources: Hugging Face collection · Ornith blog

2. Configuration Tips

  • Reasoning parser: Ornith responses include <think>...</think>. Enable the --reasoning-parser qwen3 toggle in the Playground so OpenAI-compatible responses expose the reasoning trace as message.reasoning_content.
  • Tool-call parser: enable the --tool-call-parser qwen3_coder toggle in the Playground so <tool_call> blocks are returned as OpenAI-compatible tool calls.
  • Context length: the model-card SGLang recipes use --context-length 262144. Lower it if you need more memory headroom.
  • Tensor parallelism: the 397B model-card recipes use --tp 8; in this single-node matrix, non-FP8 397B is H200-only, while 397B-FP8 is available on both H100 and H200. The 35B deploy commands use --tp 2. The 9B model-card recipe is single-GPU by default; the command panel makes that explicit with --tp 1. Adjust TP to match your node and memory budget.
  • Sampling: model cards recommend temperature=0.6, top_p=0.95, and top_k=20 for normal use. Their reported benchmark setup may use different task-specific sampling parameters.
  • Benchmarks: benchmark numbers in the model cards are reported by DeepReinforce. They are useful for context, but the command panel leaves recipes unverified until exact runs are signed off.

3. Usage Examples

3.1 Basic Chat Completion

message.reasoning_content is only populated when the server was launched with the --reasoning-parser qwen3 toggle (see the Playground); otherwise the <think>...</think> trace stays inline in message.content.

<Accordion title="Python client">
python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="Ornith-1.0-9B",
    messages=[
        {"role": "user", "content": "Write a compact Python function is_prime(n)."}
    ],
    temperature=0.6,
    top_p=0.95,
    max_tokens=1024,
    extra_body={"top_k": 20},
)

message = response.choices[0].message
print("=============== Reasoning ===============")
print(message.reasoning_content)
print("=============== Answer ==================")
print(message.content)
</Accordion>

3.2 Tool Calling

Enable the --tool-call-parser qwen3_coder toggle in the Playground and launch with the resulting command. Then use the standard OpenAI-compatible tools field:

<Accordion title="Tool-call request">
python
from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

tools = [{
    "type": "function",
    "function": {
        "name": "run_tests",
        "description": "Run the project's test suite.",
        "parameters": {
            "type": "object",
            "properties": {
                "target": {"type": "string", "description": "Test target or command"}
            },
            "required": ["target"],
        },
    },
}]

response = client.chat.completions.create(
    model="Ornith-1.0-9B",
    messages=[{"role": "user", "content": "Run the unit tests for the parser module."}],
    tools=tools,
    tool_choice="auto",
    temperature=0.6,
    top_p=0.95,
    max_tokens=2048,
)

print(response.choices[0].message.tool_calls)
</Accordion>