Prompt Injection Testing for AI Agents

What is prompt injection testing?

Prompt injection testing evaluates whether an AI system can be manipulated by instructions that conflict with the application's intended behavior. The attack may arrive directly from a user or indirectly through content the agent retrieves from documents, websites, email, databases or tool responses.

Direct and indirect prompt injection

Direct injection occurs when a user explicitly asks the model to ignore rules, reveal hidden information or perform an unauthorized action. Indirect injection is often harder to notice because malicious instructions are embedded inside content the agent considers data.

For an agent with tools, the real question is not simply whether the model says something unusual. Testing should determine whether injected instructions can influence sensitive actions such as sending messages, changing data, accessing files or calling privileged APIs.

What a useful test should examine

• Attempts to override system or developer instructions.
• Requests to reveal secrets, hidden prompts or private context.
• Instructions embedded in retrieved documents or webpages.
• Attempts to persuade the agent to misuse connected tools.
• Encoded, fragmented or multi-step instructions.
• Behavior after repeated attacks across a longer conversation.

Defensive controls

Do not rely only on telling the model to ignore malicious instructions. Restrict tool permissions, validate parameters outside the model, require authorization for sensitive actions and separate untrusted retrieved content from trusted control instructions.

Where possible, high-impact operations should be constrained by normal application logic so that unsafe model output cannot directly become a privileged action.