AI Agents Remain Vulnerable to Prompt Injection Attacks, Study Finds

A recent study reveals that AI agents using models like GPT-5 and Gemini are unable to consistently defend against prompt injection attacks, which have been shown to succeed over 79% of the time. Conducted by researchers from Nanyang Technological University, ST Engineering, IBM Research, and the University of Illinois Urbana-Champaign, the study emphasizes the critical need for improved security measures as companies deploy these technologies.

Prompt injection occurs when malicious instructions are embedded within web content, leading AI agents to follow these directives instead of adhering to user instructions. This vulnerability becomes particularly concerning as developers release AI agents capable of various tasks, including internet browsing and cryptocurrency trading.

The study introduces StakeBench, a new benchmark designed to assess how well AI agents respond to prompt injections in real-world scenarios. The research demonstrates that factors such as the semantic distance between an injected command and the user’s intention can significantly influence the effectiveness of attacks. In total, the researchers conducted 3,168 simulations using the NanoBrowser and BrowserUse tools to evaluate the security of the tested models.

"Existing security benchmarks adopt an attack-centric perspective, focusing on the technical feasibility of injections while overlooking the nuanced distribution of resulting harms," the researchers stated. They argue that the risk associated with prompt injection can yield varying consequences depending on the target, which suggests a deeper investigation into these vulnerabilities is necessary.

The findings underline the importance of addressing security weaknesses in AI systems as they continue to be integrated into public services and private applications.