azure/comparison (Azure Model Comparison)

This example demonstrates how to compare models from different providers on Azure AI Foundry, including OpenAI, Anthropic Claude, Meta Llama, and Mistral.

You can run this example with:

npx promptfoo@latest init --example azure/comparison
cd azure/comparison

Setup

1. Deploy models from different providers in Azure AI Foundry
2. Set your environment variables:

export AZURE_API_KEY=your-api-key
# Set apiHost in promptfooconfig.yaml for each provider's deployment

Models Compared

Running the Example

npx promptfoo@latest eval
npx promptfoo@latest view

Customization

Modify promptfooconfig.yaml to:

• Add or remove models
• Change test questions
• Adjust evaluation criteria
• Compare cost vs performance

Use Cases

• Benchmark different models on your specific tasks
• Evaluate cost-effectiveness across providers
• Find the best model for your use case
• A/B test model updates

Documentation

• Azure Provider Documentation
• Azure AI Foundry