config-stateful-session-management (Stateful Session Management)

This example demonstrates how to use server-generated UUIDs to identify test cases and maintain server-side conversation history when a test case has multiple turns. It is useful when your provider/target generates unique identifiers to stitch together messages into a conversation "threads".

Quick Start

npx promptfoo@latest init --example config-stateful-session-management
cd config-stateful-session-management

Requirements

Node.js 20+
An OpenAI API key in OPENAI_API_KEY

Setup

Install dependencies:
bash
```
npm install
```

Start the demo server (in one terminal):

bash

export OPENAI_API_KEY=your-api-key && npm run server

Run the eval (in another terminal):
bash
```
promptfoo redteam eval -c redteam.yaml
```

How it works

server.js: A minimal Express server that manages conversation history and forwards inference requests to OpenAI.
hooks.js: An extension hook that requests a new sessionId from the server for each test case and cleans up the session after the test case completes.
promptfooconfig.yaml: A red team configuration that includes single and multi-turn strategies and uses an HTTP provider to interface with the Express server.
redteam.yaml: The generated red team test cases to evaluate against.