LLM Sampling

import { VersionBadge } from "/snippets/version-badge.mdx";

Use this when a server asks your client to run an LLM completion on its behalf.

Sampling is how a server borrows your model. Rather than hold an API key of its own, the server describes the messages it wants completed and asks you to run them — you pick the model, and you pay for the tokens. Your side of that arrangement is one function, a sampling handler, registered when you create the client.

The handler receives the conversation the server wants completed, the parameters it asked for, and a request context carrying metadata about the call. Return the generated text as a string and FastMCP wraps it in the protocol's result for you; return a CreateMessageResult yourself when you want to report the real model name or hand back content that isn't text. If the handler raises, the client sends the error back in place of a completion and the server's tool decides what to do about it.

Handler Template

from fastmcp import Client
from fastmcp.client.sampling import SamplingMessage, SamplingParams, RequestContext
from mcp.types import TextContent


async def sampling_handler(
    messages: list[SamplingMessage],
    params: SamplingParams,
    context: RequestContext,
) -> str:
    """Run the server's messages against your LLM and return the completion."""
    conversation = [
        f"{message.role}: {message.content.text}"
        for message in messages
        if isinstance(message.content, TextContent)
    ]
    system_prompt = params.system_prompt or "You are a helpful assistant."

    # Call your LLM here with `conversation` and `system_prompt`.
    return "Generated response based on the messages"


client = Client("my_mcp_server.py", sampling_handler=sampling_handler)

The client answers with this handler however the server asks for a completion. The default mode="auto" negotiates whichever protocol era the server speaks, and one handler covers both of the routes an era can use — see Request Routes.

Handler Parameters

Everything the server sends arrives in the first two arguments. The messages are the conversation to complete; the parameters are how the server would like it completed. You decide how much of that to honor, since the client owns the model — a preference your provider cannot express is yours to ignore.

Built-in Handlers

Writing the provider call yourself is rarely worth it. FastMCP ships handlers for OpenAI, Anthropic, and Google Gemini that implement the full sampling API, tool use included, and translate the protocol's parameters into each provider's own. Give one a default model and pass it where your own handler would go. Write a custom handler when you need routing across providers, caching, or a provider FastMCP does not cover.

OpenAI Handler

from fastmcp import Client
from fastmcp.client.sampling.handlers.openai import OpenAISamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=OpenAISamplingHandler(default_model="gpt-4o"),
)

Point the handler at any OpenAI-compatible API, including a local model server, by passing your own provider client:

from fastmcp import Client
from fastmcp.client.sampling.handlers.openai import OpenAISamplingHandler
from openai import AsyncOpenAI

client = Client(
    "my_mcp_server.py",
    sampling_handler=OpenAISamplingHandler(
        default_model="llama-3.1-70b",
        client=AsyncOpenAI(base_url="http://localhost:8000/v1"),
    ),
)

Anthropic Handler

from fastmcp import Client
from fastmcp.client.sampling.handlers.anthropic import AnthropicSamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=AnthropicSamplingHandler(default_model="claude-sonnet-4-5"),
)

Google Gemini Handler

from fastmcp import Client
from fastmcp.client.sampling.handlers.google_genai import GoogleGenaiSamplingHandler

client = Client(
    "my_mcp_server.py",
    sampling_handler=GoogleGenaiSamplingHandler(default_model="gemini-2.0-flash"),
)

The source of these handlers is the best reference for writing your own.

Tool Use

A sampling request can carry tools. When it does, your handler passes them to the model and returns whatever comes back, tool calls included — the server executes the tools itself and sends a follow-up sampling request with the results if it needs another turn. Your handler never runs a tool.

Registering any sampling_handler advertises full sampling support, tools included. A handler that only generates text should say so, so servers know not to send tools it will drop:

from fastmcp import Client
from mcp.types import SamplingCapability


async def text_only_handler(messages, params, context) -> str:
    return "Generated response based on the messages"


client = Client(
    "my_mcp_server.py",
    sampling_handler=text_only_handler,
    sampling_capabilities=SamplingCapability(),
)

Request Routes

Servers reach your handler by two routes, and which one applies depends on the protocol era the connection negotiated. A handshake-era server pushes a sampling/createMessage request down the open session while a tool is running and waits for the reply. A modern (2026-07-28) connection has no such channel, so the tool ends its round by returning a request for a completion instead; the client answers from your handler and calls the tool again with the result attached.

One registration covers both, so this is rarely something you configure — it matters only when you pin an era, since mode="legacy" is the sole route that carries a pushed request. See protocol negotiation for how the era is chosen, and Sampling under Servers for how a server issues these requests.