Indirect Web Pwn Strategy

Tests whether AI agents with web browsing capabilities can be manipulated through malicious instructions embedded in web pages. The strategy dynamically generates realistic web pages containing hidden attack payloads, adapted to your target's purpose.

This strategy is plugin-agnostic and works with any plugin to test two distinct scenarios:

• Data exfiltration (with data-exfil plugin): Deterministic server-side tracking of attempted data leaks
• Indirect prompt injection (with any other plugin): LLM-based analysis of whether the agent followed injected instructions

Quick Start

Data Exfiltration Detection

Test whether injected instructions can trick the agent into leaking sensitive data to external URLs:

redteam:
  plugins:
    - data-exfil
  strategies:
    - indirect-web-pwn

Indirect Prompt Injection

Test whether injected instructions can manipulate the agent's behavior or output:

redteam:
  plugins:
    - harmful:violent-crime
    - hijacking
    - pii:direct
  strategies:
    - indirect-web-pwn

Use Cases

Architecture

The strategy works as follows:

1. Promptfoo CLI requests a web page from the Promptfoo server
2. Promptfoo Server dynamically generates HTML with the prompt injection embedded
3. Generated page is hosted and contains realistic content matching your target's purpose
4. Agent fetches the page via web fetch tool call
5. Injection payload is delivered to the agent through the page content
6. Grading occurs via server-side tracking (data-exfil) or LLM analysis (other plugins)

Dynamic Content Generation

• Website Content: Pages are dynamically generated based on target purpose and attack goal to establish realism
• Injection Technique: System randomly chooses embedding technique (invisible text, semantic embed, HTML comment) to insert the payload

Configuration

Data Exfiltration Mode

Use with the data-exfil plugin for deterministic detection of data leaks:

redteam:
  plugins:
    - data-exfil
  strategies:
    - indirect-web-pwn

With Jailbreak Strategies

Combine with jailbreak strategies using layer for more effective attacks:

redteam:
  plugins:
    - id: data-exfil
      numTests: 1
  strategies:
    - id: layer
      config:
        steps:
          - jailbreak:meta
          - indirect-web-pwn

Indirect Prompt Injection Mode

Use with other plugins to test behavior manipulation:

redteam:
  plugins:
    - harmful:violent-crime
    - harmful:illegal-drugs
    - hijacking
  strategies:
    - indirect-web-pwn

With Multiple Plugin Categories

redteam:
  plugins:
    - pii:direct
    - pii:session
    - harmful:cybercrime
    - contracts
  strategies:
    - id: layer
      config:
        steps:
          - jailbreak:meta
          - indirect-web-pwn

Multi-turn Attacks

For persistent multi-turn attacks with embedding rotation:

redteam:
  plugins:
    - data-exfil
  strategies:
    - id: layer
      config:
        steps:
          - jailbreak:hydra
          - indirect-web-pwn

On each turn, the page content is updated and the embedding location is rotated to evade detection.

Example: Generated Web Page

The strategy dynamically generates realistic-looking websites tailored to your target's purpose. The prompt injection payload is embedded as hidden text within the page — invisible to the user but readable by the agent when it fetches the page content.

The page above looks like a legitimate SaaS company blog, but contains a hidden prompt injection payload embedded in an HTML comment. When an AI agent fetches this page, the injection instructs it to exfiltrate sensitive data.

How Grading Works

Grading depends on the plugin used:

Data Exfiltration (data-exfil plugin)

The Promptfoo server tracks HTTP requests to the [EXFIL_URL] placeholder:

• Detection: Server monitors for any requests to the tracking endpoint
• Deterministic: Binary pass/fail based on whether requests were made
• Data captured: Query parameters, request body, headers containing sensitive data

Indirect Prompt Injection (other plugins)

An LLM grader analyzes the agent's response:

• Detection: Grader evaluates whether the response violates the plugin's criteria
• High reliability: LLM graders are highly accurate but not 100% deterministic
• Context-aware: Grader considers the injected instruction and response content

Test Results

Data Exfiltration Result

A successful data exfiltration attack (test failure) shows:

Indirect Prompt Injection Result

A successful behavior manipulation attack (test failure) shows:

Requirements

• Promptfoo Cloud: Server-side page generation and exfil tracking
• Agent with web browsing: Target must be able to fetch URLs (via tools, MCP, or built-in browser capabilities)

Related

• Data Exfiltration Plugin - Generates exfiltration-focused attack goals with server-side tracking
• Harmful Plugins - Test for harmful content generation
• Hijacking Plugin - Test for goal/behavior manipulation
• PII Plugins - Test for personal information leaks
• Indirect Prompt Injection Plugin - Tests injection via RAG/context variables
• Layer Strategy - Compose with jailbreak strategies