Back to Promptfoo

Reasoning DoS Plugin

site/docs/red-team/plugins/reasoning-dos.md

0.121.122.0 KB
Original Source

Reasoning DoS Plugin

Reasoning DoS (Denial of Service) covers prompts that push reasoning-capable models into unnecessary work, long runtimes, or inflated inference cost. Use this plugin to test whether the system handles those pressure cases without giving the attacker an easy resource-exhaustion path.

Purpose

It focuses on whether the system:

  1. Avoids unnecessary recursive reasoning.
  2. Resists prompts designed to trigger excessive computation.
  3. Preserves reasonable cost and latency under adversarial input.

Configuration

To include the Reasoning DoS plugin in your LLM red teaming setup:

yaml
redteam:
  plugins:
    - reasoning-dos

How It Works

The Reasoning DoS plugin generates test cases that attempt to trigger excessive computation through:

  1. Complex mathematical or logical problems requiring iterative solutions
  2. Nested decision-making scenarios
  3. Multi-step optimization problems
  4. Game theory scenarios
  5. Markov Decision Processes (MDPs)
  6. Recursive reasoning chains
  7. Puzzle-solving tasks (e.g., Sudoku)

Evaluation Criteria

The plugin assesses the AI's response based on:

  1. Computation Limits: Does the system properly limit computational resources?
  2. Pattern Recognition: Can it identify potential computational traps?
  3. Response Efficiency: Does it provide direct answers without unnecessary computation?
  4. Resource Management: How well does it manage computational resources under stress?
  5. Attack Resistance: Can it resist attempts to trigger excessive computation?

Importance in Gen AI Red Teaming

These tests matter when a model can spend far more effort on a prompt than the user spent crafting it. The risk is operational: degraded latency, higher cost, and reduced service quality under targeted abuse.