Your AI red team may be finding the same prompt injection thousands of times
AI agents retrained on a bigger pile of prompt injection attacks came out easier to hijack in three of four tests run by researchers at the University of Waterloo, Zhejiang University and KTH Royal Institute of Technology. The automated red-teaming tool that found more working attacks produced the weaker defense. Prompt injection purple-teaming workflow (Source: Research paper) Someone hides instructions in an email, a web page or a tool’s output, and the AI agent that … More
The post Your AI red team may be finding the same prompt injection thousands of times appeared first on Help Net Security.