AI Red-Teaming Workshop Will Explore Best Practices
• Article
December 16, 2024—Red-teaming is a common cybersecurity practice in which testers emulate cyber attacks on networked systems to find weak points before adversaries do. The technique is becoming more common for evaluating the safety risks of generative artificial intelligence (AI) systems, but the practice is poorly defined. To help the AI community mature AI red-teaming, the Software Engineering Institute (SEI) will hold Probing the Limits: A Workshop on Red-Teaming AI Systems, a free, hybrid event on January 28, 2025, in Pittsburgh and online.
Red-teaming is a flexible method that appeals to AI practitioners facing the broad risk surface of modern AI systems, especially generative AI. While the practice borrows useful approaches from cybersecurity, penetration testing in particular, the AI community lacks clear guidance on how to conduct red-teaming effectively. Without defined protocols and expectations for red-teaming, confidence in AI safety diminishes among AI developers, consumers, and policy makers.
SEI machine learning research scientist Anusha Sinha decided to organize January’s workshop after characterizing the current state of the practice in an award-winning 2024 paper. Sinha and her coauthors, SEI intern Michael Feffer and researchers from Carnegie Mellon University (CMU), found that red-teaming for generative AI has no standardized protocols and there is little consensus among researchers and industry practitioners on its best practices.
The paper proposed essential criteria to guide AI red-teaming. However, Sinha believes that another way to mature the practice is to foster direct conversations among practitioners, as CMU did at a February 2024 event to inform National Institute of Standards and Technology (NIST) guidance on AI red-teaming. Sinha and her SEI colleagues are hosting Probing the Limits to further collaboration among the cybersecurity and generative AI communities.
Cybersecurity practitioners, AI policy makers and researchers, and AI safety testers, evaluators, and auditors are invited to attend the workshop. The program includes interactive panels on best practices for red-teaming and penetration testing, red-teaming for generative AI systems in practice, and policy considerations for AI red-teaming.
“The goal is for practitioners to learn from each other,” said Sinha. “They might take away new techniques, connect with others who can inform their practice, and get an idea of what policy around AI red-teaming might look like. They can also engage with researchers from academia who are putting out the cutting-edge research.”
Cybersecurity and AI experts do not often have the opportunity to trade their techniques. Operating at the intersection of government, academia, and industry, the SEI convenes experts from each domain to advance the practices of software engineering, cybersecurity, and AI engineering.
Register online to attend Probing the Limits: A Workshop on Red-Teaming AI Systems virtually or in person.
