Mistral Small 4

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-4-119B-2603",
    messages=[
        {"role": "user", "content": "Solve step by step: what is 17 × 23 + 144 / 12?"},
    ],
    extra_body={"reasoning_effort": "high"},
)

print("Reasoning:", response.choices[0].message.reasoning_content)
print("Answer:", response.choices[0].message.content)

Reasoning: First, I'll break down the problem into two parts: the multiplication and
the division. According to the order of operations (PEMDAS/BODMAS), multiplication and
division are performed from left to right before addition.

17 × 23 = 17 × (20 + 3) = (17 × 20) + (17 × 3) = 340 + 51 = 391
144 / 12 = 12

Finally, add the results: 391 + 12 = 403

Answer: The solution to the problem is as follows:

1. First, perform the multiplication: 17 × 23.
   - 17 × 20 = 340
   - 17 × 3 = 51
   - 340 + 51 = 391

2. Then, perform the division: 144 / 12 = 12.

3. Finally, add the results:
   - 391 + 12 = 403

**Answer:** \boxed{403}

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-4-119B-2603",
    messages=[
        {"role": "user", "content": "Write a Python function to reverse a string."},
    ],
    extra_body={"reasoning_effort": "none"},
)

print(response.choices[0].message.content)

# Python Function to Reverse a String

Here are several ways to write a Python function to reverse a string:

## Method 1: Using String Slicing (Most Pythonic)
```python
def reverse_string(s):
    """Reverse a string using slicing."""
    return s[::-1]
```

## Method 2: Using a Loop
```python Example
def reverse_string(s):
    """Reverse a string using a loop."""
    reversed_str = ""
    for char in s:
        reversed_str = char + reversed_str
    return reversed_str
```

## Method 3: Using reversed() function
```python Example
def reverse_string(s):
    """Reverse a string using reversed() function."""
    return ''.join(reversed(s))
```

The first method using string slicing (`s[::-1]`) is generally the most efficient and
recommended approach in Python.

Example usage:
```python Example
original = "Hello, World!"
reversed_str = reverse_string(original)
print(reversed_str)  # Output: "!dlroW ,olleH"
```

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

stream = client.chat.completions.create(
    model="mistralai/Mistral-Small-4-119B-2603",
    messages=[
        {"role": "user", "content": "Explain the difference between async and threading in Python."},
    ],
    extra_body={"reasoning_effort": "high"},
    stream=True,
)

print("=== Reasoning ===")
for chunk in stream:
    delta = chunk.choices[0].delta
    if hasattr(delta, "reasoning_content") and delta.reasoning_content:
        print(delta.reasoning_content, end="", flush=True)
    elif delta.content:
        print("\n=== Response ===")
        print(delta.content, end="", flush=True)
print()

=== Reasoning ===
Okay, the user is asking about the difference between async and threading in Python.
I need to break this down clearly, covering the key aspects of both, like their
purposes, performance characteristics, and use cases...
=== Response ===
In Python, **`async`/`asyncio`** and **`threading`** are two different concurrency
models, each suited for specific use cases. Here's a breakdown of their key differences:

### 1. Model of Concurrency
- **Threading**: Based on preemptive multitasking using OS threads.
- **Async** (`asyncio`): Based on cooperative multitasking. Tasks voluntarily yield...

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-4-119B-2603",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools,
    tool_choice="auto",
)

tool_calls = response.choices[0].message.tool_calls
for tc in tool_calls:
    print(f"Tool: {tc.function.name}")
    print(f"Args: {tc.function.arguments}")

Tool: get_weather
Args: {"location": "Paris"}

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="mistralai/Mistral-Small-4-119B-2603",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe what you see in this image."},
                {
                    "type": "image_url",
                    "image_url": {"url": "https://raw.githubusercontent.com/sgl-project/sglang/main/assets/logo.png"},
                },
            ],
        }
    ],
)

print(response.choices[0].message.content)

The image is a copyright symbol, represented by a stylized version of the lowercase
letter "c" inside a circle. The "c" is depicted in a white or light-colored font, and
the circle is orange. The design is simple yet striking, using oval and elliptical
shapes to create a distinct symbol which signifies copyright protection.

python3 benchmark/gsm8k/bench_sglang.py --port 30000

TODO

python3 benchmark/mmlu/bench_sglang.py --port 30000

TODO

python3 -m sglang.bench_serving \
  --backend sglang \
  --num-prompts 10 \
  --max-concurrency 1 \
  --random-input-len 1024 \
  --random-output-len 512 \
  --port 30000

TODO

python3 -m sglang.bench_serving \
  --backend sglang \
  --num-prompts 1000 \
  --max-concurrency 100 \
  --random-input-len 1024 \
  --random-output-len 512 \
  --port 30000

TODO