ContextQMD
Back to Promptfoo

Vercel AI Gateway

site/docs/providers/vercel.md

0.121.98.1 KB
Original Source

Vercel AI Gateway

Vercel AI Gateway provides a unified interface to access AI models from 20+ providers through a single API. This provider uses the official Vercel AI SDK.

If you call the Vercel AI SDK directly from a file:// custom provider and enable experimental_telemetry, Promptfoo's trajectory assertions can normalize the SDK's tool-call spans from ai.toolCall.name plus the matching ai.toolCall.args, ai.toolCall.arguments, or ai.toolCall.input attributes.

Setup

  1. Enable AI Gateway in your Vercel Dashboard
  2. Get your API key from the AI Gateway settings
  3. Set the VERCEL_AI_GATEWAY_API_KEY environment variable or specify apiKey in your config
bash
export VERCEL_AI_GATEWAY_API_KEY=your_api_key_here

Usage

Provider Format

The Vercel provider uses the format: vercel:<provider>/<model>

yaml
providers:
  - vercel:openai/gpt-4o-mini
  - vercel:anthropic/claude-sonnet-4.5
  - vercel:google/gemini-2.5-flash

Embedding Models

For embedding models, use the embedding: prefix:

yaml
providers:
  - vercel:embedding:openai/text-embedding-3-small

Configuration

Basic Configuration

yaml
providers:
  - id: vercel:openai/gpt-4o-mini
    config:
      temperature: 0.7
      maxTokens: 1000

Full Configuration Options

yaml
providers:
  - id: vercel:anthropic/claude-sonnet-4.5
    config:
      # Authentication
      apiKey: ${VERCEL_AI_GATEWAY_API_KEY}
      apiKeyEnvar: CUSTOM_API_KEY_VAR # Use a custom env var name

      # Model settings
      temperature: 0.7
      maxTokens: 2000
      topP: 0.9
      topK: 40
      frequencyPenalty: 0.5
      presencePenalty: 0.3
      stopSequences:
        - '\n\n'

      # Request settings
      timeout: 60000
      headers:
        Custom-Header: 'value'

      # Streaming
      streaming: true

Configuration Parameters

ParameterTypeDescription
apiKeystringVercel AI Gateway API key
apiKeyEnvarstringCustom environment variable name for API key
temperaturenumberControls randomness (0.0 to 1.0)
maxTokensnumberMaximum number of tokens to generate
topPnumberNucleus sampling parameter
topKnumberTop-k sampling parameter
frequencyPenaltynumberPenalizes frequent tokens
presencePenaltynumberPenalizes tokens based on presence
stopSequencesstring[]Sequences where generation stops
timeoutnumberRequest timeout in milliseconds
headersobjectAdditional HTTP headers
streamingbooleanEnable streaming responses
responseSchemaobjectJSON schema for structured output
baseUrlstringOverride the AI Gateway base URL

Structured Output

Generate structured JSON output by providing a JSON schema:

yaml
providers:
  - id: vercel:openai/gpt-4o
    config:
      responseSchema:
        type: object
        properties:
          sentiment:
            type: string
            enum: [positive, negative, neutral]
          confidence:
            type: number
          keywords:
            type: array
            items:
              type: string
        required:
          - sentiment
          - confidence

prompts:
  - 'Analyze the sentiment of this text: {{text}}'

tests:
  - vars:
      text: 'I love this product!'
    assert:
      - type: javascript
        value: output.sentiment === 'positive'

Streaming

Enable streaming for real-time responses:

yaml
providers:
  - id: vercel:anthropic/claude-sonnet-4.5
    config:
      streaming: true
      maxTokens: 2000

Supported Providers

The Vercel AI Gateway supports models from these providers:

ProviderExample Models
OpenAIopenai/gpt-5, openai/o3-mini, openai/gpt-4o-mini
Anthropicanthropic/claude-sonnet-4.5, anthropic/claude-haiku-4.5
Googlegoogle/gemini-2.5-flash, google/gemini-2.5-pro
Mistralmistral/mistral-large, mistral/magistral-medium
Coherecohere/command-a
DeepSeekdeepseek/deepseek-r1, deepseek/deepseek-v3
Perplexityperplexity/sonar-pro, perplexity/sonar-reasoning
xAIxai/grok-3, xai/grok-4

For a complete list, see the Vercel AI Gateway documentation.

Embedding Models

Generate embeddings for text similarity, search, and RAG applications:

yaml
providers:
  - vercel:embedding:openai/text-embedding-3-small

prompts:
  - 'Generate embedding for: {{text}}'

tests:
  - vars:
      text: 'Hello world'
    assert:
      - type: is-valid-embedding

Supported embedding models:

ProviderExample Models
OpenAIopenai/text-embedding-3-small, openai/text-embedding-3-large
Googlegoogle/gemini-embedding-001, google/text-embedding-005
Coherecohere/embed-v4.0
Voyagevoyage/voyage-3.5, voyage/voyage-code-3

Examples

Multi-Provider Comparison

yaml
providers:
  - id: vercel:openai/gpt-4o-mini
    config:
      temperature: 0.7
  - id: vercel:anthropic/claude-sonnet-4.5
    config:
      temperature: 0.7
  - id: vercel:google/gemini-2.5-flash
    config:
      temperature: 0.7

prompts:
  - 'Explain {{concept}} in simple terms'

tests:
  - vars:
      concept: 'quantum computing'
    assert:
      - type: llm-rubric
        value: 'The response should be easy to understand'

JSON Response with Validation

yaml
providers:
  - id: vercel:openai/gpt-4o
    config:
      responseSchema:
        type: object
        properties:
          summary:
            type: string
          topics:
            type: array
            items:
              type: string
          wordCount:
            type: integer
        required:
          - summary
          - topics

prompts:
  - 'Analyze this article and return a structured summary: {{article}}'

tests:
  - vars:
      article: 'Long article text...'
    assert:
      - type: javascript
        value: 'Array.isArray(output.topics) && output.topics.length > 0'

Environment Variables

VariableDescription
VERCEL_AI_GATEWAY_API_KEYAPI key for AI Gateway
VERCEL_AI_GATEWAY_BASE_URLOverride the AI Gateway URL

Troubleshooting

Common Issues

  1. Authentication Failed: Ensure your VERCEL_AI_GATEWAY_API_KEY is set correctly
  2. Model Not Found: Check that the provider/model combination is supported
  3. Request Timeout: Increase the timeout configuration value

Debug Mode

Enable debug logging to see detailed request/response information:

bash
LOG_LEVEL=debug promptfoo eval