Cloudflare AI Gateway

Cloudflare AI Gateway is a proxy service that routes requests to AI providers through Cloudflare's infrastructure. It provides:

• Caching - Reduce costs by caching identical requests
• Rate limiting - Control request rates to avoid quota issues
• Analytics - Track usage and costs across providers
• Logging - Monitor requests and responses
• Fallback - Configure fallback providers for reliability

The cloudflare-gateway provider lets you route your promptfoo evaluations through Cloudflare AI Gateway to any supported AI provider.

Provider Format

cloudflare-gateway:{provider}:{model}

Examples:

• cloudflare-gateway:openai:gpt-5.2
• cloudflare-gateway:anthropic:claude-sonnet-4-5-20250929
• cloudflare-gateway:groq:llama-3.3-70b-versatile

Required Configuration

Set your Cloudflare account ID and gateway ID:

export CLOUDFLARE_ACCOUNT_ID=your_account_id_here
export CLOUDFLARE_GATEWAY_ID=your_gateway_id_here

Provider API Keys

You need API keys for the providers you're routing through:

# For OpenAI
export OPENAI_API_KEY=your_openai_key

# For Anthropic
export ANTHROPIC_API_KEY=your_anthropic_key

# For Groq
export GROQ_API_KEY=your_groq_key

Using BYOK (Bring Your Own Keys)

If you've configured BYOK in Cloudflare, you can omit provider API keys entirely. Cloudflare will use the keys stored in your gateway configuration.

providers:
  # No OPENAI_API_KEY needed - Cloudflare uses stored key
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      cfAigToken: '{{env.CF_AIG_TOKEN}}'

:::note BYOK works best with OpenAI-compatible providers. Anthropic requires an API key because the SDK mandates it. :::

Authenticated Gateways

If your gateway has Authenticated Gateway enabled, you must provide the cfAigToken:

export CF_AIG_TOKEN=your_gateway_token_here

Basic Usage

prompts:
  - 'Answer this question: {{question}}'

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      temperature: 0.7

tests:
  - vars:
      question: What is the capital of France?

Supported Providers

Cloudflare AI Gateway supports routing to these providers:

:::note AWS Bedrock is not supported through Cloudflare AI Gateway because it requires AWS request signing, which is incompatible with the gateway proxy approach. :::

Configuration Options

Gateway Configuration

Azure OpenAI Configuration

Azure OpenAI requires additional configuration:

providers:
  - id: cloudflare-gateway:azure-openai:gpt-4
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      resourceName: my-azure-resource
      deploymentName: my-gpt4-deployment
      apiVersion: 2024-12-01-preview

Workers AI Configuration

Workers AI routes requests to Cloudflare's edge-deployed models. The model name is included in the URL path:

providers:
  - id: cloudflare-gateway:workers-ai:@cf/meta/llama-3.1-8b-instruct
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

Provider-Specific Options

All options from the underlying provider are supported. For example, when using cloudflare-gateway:openai:gpt-5.2, you can use any OpenAI provider options.

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      temperature: 0.8
      max_tokens: 1000
      top_p: 0.9

Examples

Multiple Providers

Compare responses from different providers, all routed through your Cloudflare gateway:

prompts:
  - 'Explain {{topic}} in simple terms.'

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

  - id: cloudflare-gateway:anthropic:claude-sonnet-4-5-20250929
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

  - id: cloudflare-gateway:groq:llama-3.3-70b-versatile
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

tests:
  - vars:
      topic: quantum computing

Authenticated Gateway

If your AI Gateway requires authentication:

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      cfAigToken: '{{env.CF_AIG_TOKEN}}'

Custom Environment Variables

Use custom environment variable names for different projects or environments:

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountIdEnvar: MY_CF_ACCOUNT
      gatewayIdEnvar: MY_CF_GATEWAY
      apiKeyEnvar: MY_OPENAI_KEY

Gateway URL Structure

The provider constructs the gateway URL in this format:

https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}

For example, with accountId: abc123 and gatewayId: my-gateway, requests to OpenAI would be routed through:

https://gateway.ai.cloudflare.com/v1/abc123/my-gateway/openai

Benefits of Using AI Gateway

Cost Reduction Through Caching

AI Gateway can cache identical requests, reducing costs when you run the same prompts multiple times (common during development and testing).

Unified Analytics

View usage across all your AI providers in a single Cloudflare dashboard, making it easier to track costs and usage patterns.

Rate Limit Protection

AI Gateway can help manage rate limits by queuing requests, preventing your evaluations from failing due to provider rate limits.

Logging and Debugging

All requests and responses are logged in Cloudflare, making it easier to debug issues and audit AI usage.

See Also

• Cloudflare AI Gateway Documentation
• Cloudflare Workers AI Provider - For running models directly on Cloudflare's edge
• OpenAI Provider
• Anthropic Provider