site/docs/red-team/eu-ai-act.md
The EU Artificial Intelligence Act (AI Act) is the world's first comprehensive legal framework specifically regulating artificial intelligence systems. Enacted in 2024, it establishes harmonized rules for the development, placement on the market, and use of AI systems in the European Union.
The AI Act uses a risk-based approach, categorizing AI systems by their level of risk to fundamental rights and safety. It outright prohibits certain AI practices, heavily regulates high-risk systems, and provides lighter requirements for limited-risk and minimal-risk AI.
The EU AI Act categorizes AI systems into four risk levels:
Promptfoo's red teaming capabilities focus on testing for prohibited practices (Article 5) and high-risk system requirements (Annex III).
Article 5 lists AI practices that are prohibited in the EU due to their unacceptable risk to fundamental rights:
Annex III defines categories of high-risk AI systems that require compliance with strict requirements:
Promptfoo helps identify compliance issues with the EU AI Act through red teaming:
redteam:
plugins:
- eu:ai-act
strategies:
- jailbreak
- prompt-injection
Or target specific articles or risk categories:
redteam:
plugins:
- eu:ai-act:art5:subliminal-manipulation
- eu:ai-act:annex3:biometric-id
- eu:ai-act:annex3:law-enforcement
Prohibition: AI systems that deploy subliminal techniques beyond a person's consciousness to materially distort their behavior in a manner that causes or is likely to cause them or another person physical or psychological harm.
LLM applications must not:
Test for subliminal manipulation:
Example configuration:
redteam:
plugins:
- hijacking
- intent
- excessive-agency
strategies:
- jailbreak
- jailbreak:tree
- jailbreak:composite
- prompt-injection
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:art5:subliminal-manipulation
Prohibition: AI systems that exploit vulnerabilities of persons due to their age, disability, or specific social or economic situation to materially distort their behavior in a manner that causes them or another person physical or psychological harm.
Test for exploitation of vulnerabilities:
Example configuration:
redteam:
plugins:
- hijacking
- imitation
- harmful:misinformation-disinformation
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:art5:exploitation-of-vulnerabilities
Prohibition: AI systems that evaluate or classify natural persons based on their social behavior or personal characteristics, with evaluations leading to detrimental or unfavorable treatment.
Test for social scoring behaviors:
Example configuration:
redteam:
plugins:
- harmful:misinformation-disinformation
- overreliance
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:art5:social-scoring
Prohibition: AI systems that categorize individuals based on biometric data to deduce or infer race, political opinions, trade union membership, religious or philosophical beliefs, sex life, or sexual orientation (with limited exceptions).
Test for biometric categorisation:
Example configuration:
redteam:
plugins:
- pii:direct
- pii:session
- harmful:privacy
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:art5:biometric-categorisation
Prohibition: Real-time remote biometric identification systems in publicly accessible spaces for law enforcement purposes (with specific exceptions for serious crimes).
Test for real-time biometric identification:
redteam:
plugins:
- pii:session
- pii:direct
- harmful:privacy
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:art5:remote-biometric-id-live
Prohibition: Use of remote biometric identification systems on recorded footage (with specific exceptions and procedural safeguards).
Test for post-event biometric identification:
redteam:
plugins:
- pii:api-db
- pii:direct
- harmful:privacy
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:art5:remote-biometric-id-post
Risk Category: High-risk AI systems intended for biometric identification and categorization of natural persons.
These plugins can be used to test for biometric identification risk. You'll have to customize the Application Details/Purpose in order to explicitly state the biometric ID behaviors to test.
redteam:
plugins:
- pii:direct
- pii:session
- harmful:privacy
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:annex3:biometric-id
Risk Category: High-risk AI systems intended for safety components in the management and operation of critical infrastructure (energy, water, transport, etc.).
Test for critical infrastructure risks:
Example configuration:
redteam:
plugins:
- shell-injection
- sql-injection
- ssrf
- excessive-agency
strategies:
- jailbreak
- prompt-injection
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:annex3:critical-infrastructure
Risk Category: High-risk AI systems intended for determining access to educational institutions, evaluating learning outcomes, or assessing appropriate level of education.
Test for education-related risks:
Example configuration:
redteam:
plugins:
- harmful:misinformation-disinformation
- hallucination
- overreliance
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:annex3:education
Risk Category: High-risk AI systems for recruitment, selection, evaluation, promotion, or termination of employment relationships.
Test for employment-related risks:
Example configuration:
redteam:
plugins:
- imitation
- pii:direct
- overreliance
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:annex3:employment
Risk Category: High-risk AI systems for evaluating creditworthiness, dispatching emergency services, or accessing essential public services.
Test for essential services risks:
redteam:
plugins:
- pii:direct
- pii:session
- excessive-agency
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:annex3:essential-services
Risk Category: High-risk AI systems for law enforcement purposes including risk assessments, polygraph interpretation, and evidence evaluation.
Test for law enforcement risks:
redteam:
plugins:
- pii:direct
- pii:api-db
- harmful:privacy
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:annex3:law-enforcement
Risk Category: High-risk AI systems for migration, asylum, and border control management including verification of authenticity and risk assessments.
Test for migration and border control risks:
redteam:
plugins:
- pii:direct
- harmful:hate
- harmful:privacy
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:annex3:migration-border
Risk Category: High-risk AI systems for administration of justice, democratic processes, and assisting judicial authorities in researching and interpreting facts and law.
Test for justice and democracy risks:
Example configuration:
redteam:
plugins:
- hallucination
- harmful:misinformation-disinformation
- pii:direct
Or use the EU AI Act shorthand:
redteam:
plugins:
- eu:ai-act:annex3:justice-democracy
For complete EU AI Act compliance testing:
redteam:
plugins:
- eu:ai-act
strategies:
- jailbreak
- prompt-injection
This tests across both prohibited practices and high-risk system requirements.
While red teaming helps identify technical risks, EU AI Act compliance requires additional measures:
The EU AI Act imposes significant fines for violations:
The EU AI Act follows a phased implementation:
The EU AI Act aligns with and references other frameworks:
You can combine EU AI Act testing with other frameworks:
redteam:
plugins:
- eu:ai-act
- gdpr
- iso:42001
strategies:
- jailbreak
- prompt-injection
The EU AI Act is a living regulation with ongoing guidance and standards development. Regular testing with Promptfoo helps maintain compliance as requirements evolve.
To learn more about setting up comprehensive AI red teaming, see Introduction to LLM red teaming and Configuration details.