Model Inversion

AI Red Teamers perform model inversion tests to assess if an attacker can reconstruct sensitive training data (like images, text snippets, or personal attributes) by repeatedly querying the model and analyzing its outputs. Success indicates privacy risks due to data memorization, requiring mitigation techniques like differential privacy or output filtering.

Learn more from the following resources:

• @article@Model inversion and membership inference: Understanding new AI security risks
• @paper@Extracting Training Data from LLMs (arXiv)
• @paper@Model Inversion Attacks: A Survey of Approaches and Countermeasures