Caching

promptfoo caches the results of API calls to LLM providers to help save time and cost.

The cache is managed by cache-manager with keyv and keyv-file for disk-based storage. By default, promptfoo uses disk-based storage (~/.promptfoo/cache).

How Caching Works

Cache Keys

Cache entries are stored using provider-specific composite keys that include:

• Provider identifier
• Prompt or request content, often represented as a deterministic digest
• Provider configuration
• Context variables (when applicable)

Cache key formats are implementation details and may change between versions. Sensitive request payloads and headers are hashed where possible instead of being embedded directly in cache keys.

// Provider-specific scope plus a digest of request material
const providerCacheKey = `openai:gpt-5:<request-digest>`;

// HTTP fetch cache entries include URL, method, headers, options, and body identity
const fetchCacheKey = `fetch:v3:<request-digest>`;

Cache Behavior

• Successful API responses are cached with their complete response data
• Error responses are not cached to allow for retry attempts
• When evaluateOptions.repeat or --repeat is greater than 1, each repeat index uses a separate cache namespace. Re-running the same eval can reuse those per-repeat cached responses, while preserving distinct outputs between repeat 0, repeat 1, etc.
• Cache is automatically invalidated when:
  • TTL expires (default: 14 days)
  • Cache is manually cleared
• Memory storage is used automatically when NODE_ENV=test

Command Line

If you're using the command line, call promptfoo eval with --no-cache to disable the cache, or set { evaluateOptions: { cache: false }} in your config file.

Use --no-cache with --repeat when you want every run to make fresh LLM calls instead of replaying each repeat index from cache.

Use promptfoo cache clear command to clear the cache.

Node package

Set EvaluateOptions.cache to false to disable cache:

promptfoo.evaluate(testSuite, {
  cache: false,
});

Tests

If you're integrating with jest or vitest, mocha, or any other external framework, you'll probably want to set the following for CI:

PROMPTFOO_CACHE_TYPE=disk
PROMPTFOO_CACHE_PATH=...

Configuration

The cache is configurable through environment variables:

Additional Cache Details

• Rate limit responses (HTTP 429) are automatically handled with exponential backoff
• Empty responses are not cached
• HTTP 500 responses can be retried by setting PROMPTFOO_RETRY_5XX=true

Managing the Cache

Clearing the Cache

You can clear the cache in several ways:

1. Using the CLI command:

promptfoo cache clear

2. Through the Node.js API:

const promptfoo = require('promptfoo');
await promptfoo.cache.clearCache();

3. Manually delete the cache directory:

rm -rf ~/.promptfoo/cache

Cache Busting

You can force a cache miss in two ways:

1. Pass --no-cache to the CLI:

promptfoo eval --no-cache

2. Set cache busting in code:

const result = await fetchWithCache(url, options, timeout, 'json', true); // Last param forces cache miss