Adversarial Context Injection

Malicious instructions are hidden within external data sources that an agent retrieves. When the agent processes a compromised webpage, PDF, or email, it treats the injected text as a new directive, overriding its original system prompt.

• Mechanism: Attacker embeds [SYSTEM] Ignore previous instructions... in a document indexed by the RAG pipeline
• Impact: Agent executes attacker-specified tool calls or exfiltrates data
• Real-world example: A resume uploaded to a hiring agent contains invisible text instructing the agent to classify the candidate as 'HIRED'

The adversarial corruption of a RAG pipeline's external knowledge base. By injecting malicious documents into the vector store, attackers ensure the agent retrieves and grounds its responses in attacker-controlled content.

• Attack surface: Publicly crawlable documentation, user-generated content platforms, shared knowledge bases
• Key distinction: Unlike prompt injection, this poisons the retrieval source rather than the query
• Mitigation: Cryptographic document signing and provenance verification

A persistent attack where adversarial content injected into an agent's long-term memory influences behavior across multiple, independent user sessions. The poisoned memory entry acts as a sleeper agent, activating when specific retrieval conditions are met.

• Storage targets: Conversation history databases, episodic memory buffers, user profile stores
• Amplification effect: One poisoned entry can compromise all future sessions for affected users
• Detection challenge: The malicious payload may remain dormant for weeks before triggering

An attack where the response from an API or function call is intercepted and replaced with malicious content. The agent ingests this as trusted context for subsequent reasoning and actions.

• Attack vector: Compromised third-party APIs, man-in-the-middle interception, dependency confusion
• Cascading failure: A poisoned tool output can cause the agent to execute additional malicious tool calls
• Example: A weather API response includes hidden instructions to disable safety filters

The injection of malicious reasoning steps into an agent's scratchpad or reflection loop. By seeding the intermediate reasoning trace with flawed logic, attackers cause the agent to adopt and execute an attacker-intended conclusion.

• Target: Agents using explicit reasoning frameworks like ReAct, Tree-of-Thoughts, or Reflexion
• Subtlety: The contamination appears as the agent's own reasoning, bypassing output filters
• Defense: Sandboxed reasoning traces with cryptographic integrity verification

An attack that exploits token limits by flooding the context window with irrelevant data. This displaces critical system prompts, safety instructions, or few-shot examples from the model's attention.

• Mechanism: Attacker submits a query with thousands of tokens of filler text
• Lost-in-the-Middle effect: Safety instructions in the middle of the context are most vulnerable to displacement
• Variants: Token budget attacks that force truncation of guardrails before processing the malicious payload

An attack where malicious instructions are hidden within external data sources that an agent retrieves, causing the agent to execute those instructions as if they were part of its system prompt. This is the most common delivery mechanism for Adversarial Context Injection.

• Payloads are embedded in web pages, PDFs, or emails
• The agent retrieves and processes the poisoned document
• Instructions override system prompts, triggering tool use or data exfiltration

The adversarial corruption of a RAG pipeline's external knowledge base, causing the agent to retrieve and ground its responses in attacker-controlled documents. This targets the retrieval step before context injection occurs.

• Attacker seeds the vector database with malicious documents
• Semantic search returns poisoned chunks as top-k results
• The agent treats the injected content as authoritative ground truth

An attack that exploits token limits by flooding the context window with irrelevant data to displace critical system prompts, safety instructions, or few-shot examples. This creates the preconditions for successful injection.

• Filler content pushes safety instructions out of the attention window
• The agent loses its behavioral constraints
• Subsequent malicious instructions face no opposition

The injection of malicious reasoning steps into an agent's scratchpad or reflection loop, causing it to adopt a flawed logic path and reach an attacker-intended conclusion. This targets the reasoning process rather than the final output.

• Attacker inserts fake intermediate reasoning tokens
• The agent treats the contaminated chain as its own prior thought
• Subsequent decisions are anchored to the adversarial premise

An attack where the response from an API or function call is intercepted and replaced with malicious content, which the agent then ingests as trusted context for subsequent actions. This exploits the implicit trust agents place in their own tool outputs.

• A compromised API returns attacker-crafted JSON
• The agent processes the response as verified data
• The poisoned output cascades into downstream decisions

A persistent attack where adversarial content injected into an agent's long-term memory or conversation history influences its behavior across multiple, independent user sessions. This weaponizes the memory persistence that makes agents useful.

• Attacker plants malicious context in a single session
• The agent stores the poisoned data in long-term memory
• All future sessions are compromised by the latent payload