HTTP Request Retries

Pydantic AI provides retry functionality for HTTP requests made by model providers through custom HTTP transports. This is particularly useful for handling transient failures like rate limits, network timeouts, or temporary server errors.

This is the lowest of the several layers that can retry in an agent run, and the only one the model never sees.

Overview

The retry functionality is built on top of the tenacity library and integrates seamlessly with httpx2 clients. You can configure retry behavior for providers whose SDK accepts a custom httpx2 client.

Installation

To use the retry transports, you need to install tenacity, which you can do via the retries dependency group:

bash

pip/uv-add 'pydantic-ai-slim[retries]'

Usage Example

Here's an example of adding retry functionality with smart retry handling:

python

from httpx2 import AsyncClient, ConnectError, HTTPStatusError
from tenacity import retry_if_exception_type, stop_after_attempt, wait_exponential

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from pydantic_ai.retries import (
    AsyncHTTPX2TenacityTransport,
    RetryConfig,
    wait_retry_after,
)


def create_retrying_client():
    """Create a client with smart retry handling for multiple error types."""

    def should_retry_status(response):
        """Raise exceptions for retryable HTTP status codes."""
        if response.status_code in (429, 502, 503, 504):
            response.raise_for_status()  # This will raise HTTPStatusError

    transport = AsyncHTTPX2TenacityTransport(
        config=RetryConfig(
            # Retry on HTTP errors and connection issues
            retry=retry_if_exception_type((HTTPStatusError, ConnectError)),
            # Smart waiting: respects Retry-After headers, falls back to exponential backoff
            wait=wait_retry_after(
                fallback_strategy=wait_exponential(multiplier=1, max=60),
                max_wait=300
            ),
            # Stop after 5 attempts
            stop=stop_after_attempt(5),
            # Re-raise the last exception if all retries fail
            reraise=True
        ),
        validate_response=should_retry_status
    )
    return AsyncClient(transport=transport)

# Use the retrying client with a model
client = create_retrying_client()
model = OpenAIChatModel('gpt-5.2', provider=OpenAIProvider(http_client=client))
agent = Agent(model)

Wait Strategies

wait_retry_after

The wait_retry_after function is a smart wait strategy that automatically respects HTTP Retry-After headers:

python

from tenacity import wait_exponential

from pydantic_ai.retries import wait_retry_after

# Basic usage - respects Retry-After headers, falls back to exponential backoff
wait_strategy_1 = wait_retry_after()

# Custom configuration
wait_strategy_2 = wait_retry_after(
    fallback_strategy=wait_exponential(multiplier=2, max=120),
    max_wait=600  # Never wait more than 10 minutes
)

This wait strategy:

Automatically parses Retry-After headers from HTTP 429 responses
Supports both seconds format ("30") and HTTP date format ("Wed, 21 Oct 2015 07:28:00 GMT")
Falls back to your chosen strategy when no header is present
Respects the max_wait limit to prevent excessive delays

Transport Classes

AsyncHTTPX2TenacityTransport

For asynchronous HTTP clients (recommended for most use cases):

python

from httpx2 import AsyncClient
from tenacity import stop_after_attempt

from pydantic_ai.retries import AsyncHTTPX2TenacityTransport, RetryConfig


def validator(response):
    """Treat responses with HTTP status 4xx/5xx as failures that need to be retried.
    Without a response validator, only network errors and timeouts will result in a retry.
    """
    response.raise_for_status()

# Create the transport
transport = AsyncHTTPX2TenacityTransport(
    config=RetryConfig(stop=stop_after_attempt(3), reraise=True),
    validate_response=validator
)

# Create a client using the transport:
client = AsyncClient(transport=transport)

HTTPX2TenacityTransport

For synchronous HTTP clients:

python

from httpx2 import Client
from tenacity import stop_after_attempt

from pydantic_ai.retries import HTTPX2TenacityTransport, RetryConfig


def validator(response):
    """Treat responses with HTTP status 4xx/5xx as failures that need to be retried.
    Without a response validator, only network errors and timeouts will result in a retry.
    """
    response.raise_for_status()

# Create the transport
transport = HTTPX2TenacityTransport(
    config=RetryConfig(stop=stop_after_attempt(3), reraise=True),
    validate_response=validator
)

# Create a client using the transport
client = Client(transport=transport)

Common Retry Patterns

Rate Limit Handling with Retry-After Support

python

from httpx2 import AsyncClient, HTTPStatusError
from tenacity import retry_if_exception_type, stop_after_attempt, wait_exponential

from pydantic_ai.retries import (
    AsyncHTTPX2TenacityTransport,
    RetryConfig,
    wait_retry_after,
)


def create_rate_limit_client():
    """Create a client that respects Retry-After headers from rate limiting responses."""
    transport = AsyncHTTPX2TenacityTransport(
        config=RetryConfig(
            retry=retry_if_exception_type(HTTPStatusError),
            wait=wait_retry_after(
                fallback_strategy=wait_exponential(multiplier=1, max=60),
                max_wait=300  # Don't wait more than 5 minutes
            ),
            stop=stop_after_attempt(10),
            reraise=True
        ),
        validate_response=lambda r: r.raise_for_status()  # Raises HTTPStatusError for 4xx/5xx
    )
    return AsyncClient(transport=transport)

# Example usage
client = create_rate_limit_client()
# Client is now ready to use with any HTTP requests and will respect Retry-After headers

The wait_retry_after function automatically detects Retry-After headers in 429 (rate limit) responses and waits for the specified time. If no header is present, it falls back to exponential backoff.

Network Error Handling

python

import httpx2
from tenacity import retry_if_exception_type, stop_after_attempt, wait_exponential

from pydantic_ai.retries import AsyncHTTPX2TenacityTransport, RetryConfig


def create_network_resilient_client():
    """Create a client that handles network errors with retries."""
    transport = AsyncHTTPX2TenacityTransport(
        config=RetryConfig(
            retry=retry_if_exception_type((
                httpx2.TimeoutException,
                httpx2.ConnectError,
                httpx2.ReadError
            )),
            wait=wait_exponential(multiplier=1, max=10),
            stop=stop_after_attempt(3),
            reraise=True
        )
    )
    return httpx2.AsyncClient(transport=transport)

# Example usage
client = create_network_resilient_client()
# Client will now retry on timeout, connection, and read errors

Custom Retry Logic

python

import httpx2
from tenacity import retry_if_exception, stop_after_attempt, wait_exponential

from pydantic_ai.retries import (
    AsyncHTTPX2TenacityTransport,
    RetryConfig,
    wait_retry_after,
)


def create_custom_retry_client():
    """Create a client with custom retry logic."""
    def custom_retry_condition(exception):
        """Custom logic to determine if we should retry."""
        if isinstance(exception, httpx2.HTTPStatusError):
            # Retry on server errors but not client errors
            return 500 <= exception.response.status_code < 600
        return isinstance(exception, httpx2.TimeoutException | httpx2.ConnectError)

    transport = AsyncHTTPX2TenacityTransport(
        config=RetryConfig(
            retry=retry_if_exception(custom_retry_condition),
            # Use wait_retry_after for smart waiting on rate limits,
            # with custom exponential backoff as fallback
            wait=wait_retry_after(
                fallback_strategy=wait_exponential(multiplier=2, max=30),
                max_wait=120
            ),
            stop=stop_after_attempt(5),
            reraise=True
        ),
        validate_response=lambda r: r.raise_for_status()
    )
    return httpx2.AsyncClient(transport=transport)

client = create_custom_retry_client()
# Client will retry server errors (5xx) and network errors, but not client errors (4xx)

Using with `httpx2`-Compatible Providers

The retry transports work with any provider whose http_client argument accepts an httpx2.AsyncClient. See each provider's docs for the client type it takes; Bedrock uses boto3 and configures retries its own way.

Providers whose SDKs still require a legacy httpx.AsyncClient (such as Anthropic, Groq, and Cohere) can use the deprecated [TenacityTransport][pydantic_ai.retries.TenacityTransport] and [AsyncTenacityTransport][pydantic_ai.retries.AsyncTenacityTransport] on that client during Pydantic AI v2; both are removed in v3 together with legacy client support.

OpenAI

python

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

from smart_retry_example import create_retrying_client

client = create_retrying_client()
model = OpenAIChatModel('gpt-5.2', provider=OpenAIProvider(http_client=client))
agent = Agent(model)

Any OpenAI-Compatible Provider

python

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

from smart_retry_example import create_retrying_client

client = create_retrying_client()
model = OpenAIChatModel(
    'your-model-name',  # Replace with actual model name
    provider=OpenAIProvider(
        base_url='https://api.example.com/v1',  # Replace with actual API URL
        api_key='your-api-key',  # Replace with actual API key
        http_client=client
    )
)
agent = Agent(model)

Best Practices

Start Conservative: Begin with a small number of retries (3-5) and reasonable wait times.
Use Exponential Backoff: This helps avoid overwhelming servers during outages.
Set Maximum Wait Times: Prevent indefinite delays with reasonable maximum wait times.
Handle Rate Limits Properly: Respect Retry-After headers when possible.
Log Retry Attempts: Add logging to monitor retry behavior in production. (This will be picked up by Logfire automatically if you instrument httpx2.)
Consider Circuit Breakers: For high-traffic applications, consider implementing circuit breaker patterns.

!!! tip "Monitoring Retries in Production" Excessive retries can indicate underlying issues and increase costs. Logfire helps you track retry patterns:

- See which requests triggered retries
- Understand retry causes (rate limits, server errors, timeouts)
- Monitor retry frequency over time
- Identify opportunities to reduce retries

With [HTTPX instrumentation](../logfire.md#monitoring-http-requests) enabled, retry attempts are automatically captured in your traces.

Error Handling

The retry transports will re-raise the last exception if all retry attempts fail. Make sure to handle these appropriately in your application:

python

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

from smart_retry_example import create_retrying_client

client = create_retrying_client()
model = OpenAIChatModel('gpt-5.2', provider=OpenAIProvider(http_client=client))
agent = Agent(model)

Performance Considerations

Retries add latency to requests, especially with exponential backoff
Consider the total timeout for your application when configuring retry behavior
Monitor retry rates to detect systemic issues
Use async transports for better concurrency when handling multiple requests

For more advanced retry configurations, refer to the tenacity documentation.

Provider-Specific Retry Behavior

AWS Bedrock

The AWS Bedrock provider uses boto3's built-in retry mechanisms instead of httpx2. To configure retries for Bedrock, use boto3's Config:

python

from botocore.config import Config

config = Config(retries={'max_attempts': 5, 'mode': 'adaptive'})

See Bedrock: Configuring Retries for complete examples.

HTTP Request Retries

HTTP Request Retries

Overview

Installation

Usage Example

Wait Strategies

wait_retry_after

Transport Classes

AsyncHTTPX2TenacityTransport

HTTPX2TenacityTransport

Common Retry Patterns

Rate Limit Handling with Retry-After Support

Network Error Handling

Custom Retry Logic

Using with httpx2-Compatible Providers

OpenAI

Any OpenAI-Compatible Provider

Best Practices

Error Handling

Performance Considerations

Provider-Specific Retry Behavior

AWS Bedrock

Using with `httpx2`-Compatible Providers