redteam-cyberseceval (CyberSecEval Example)

You can run this example with:

npx promptfoo@latest init --example redteam-cyberseceval
cd redteam-cyberseceval

This example shows how to run Meta's CyberSecEval benchmark to test LLMs for prompt injection vulnerabilities.

Setup

1. Install dependencies:

npm install

2. Configure your model in promptfooconfig.yaml:

providers:
  - openai:gpt-4o # OpenAI
  - anthropic:messages:claude-sonnet-4-6 # Anthropic
  - ollama:chat:llama3.3 # Ollama
  - replicate:meta/llama-2-70b-chat # Replicate

Usage

Run all tests:

npx promptfoo eval

Run a sample of tests:

npx promptfoo eval --filter-sample 30

View results:

npx promptfoo view

Configuration

The example includes:

• promptfooconfig.yaml: Main configuration file
• prompt.json: System prompt for the model
• prompt_injection.json: CyberSecEval test cases

Learn More

• CyberSecEval Documentation
• Prompt Injection Benchmarks
• Full Tutorial