Contextual Summarization Poisoning

The attack leverages the agent's need to summarize long conversation histories to stay within token limits. Each summarization cycle acts as a lossy compression step. The attacker injects content specifically designed to survive compression while safety instructions are abstracted away.

• Mechanism: The attacker floods the context with verbose, repetitive, or high-entropy text that dominates the summarization attention weights.
• Outcome: The summarizer allocates disproportionate 'summary budget' to attacker content, causing system prompts to be truncated to generic placeholders like 'Be helpful.'
• Key Insight: The attack doesn't need to override the system prompt directly; it simply ensures the system prompt is the first information lost during compression.

Attackers inject large volumes of semantically dense but benign distractor content that competes with safety instructions for representational space in the summary embedding. The summarization model, optimized for semantic fidelity, preserves the factual richness of the distractor while collapsing the imperative tone of safety guidelines.

• Example: An attacker submits a lengthy, detailed technical specification. The summarizer faithfully preserves the spec's details but reduces 'Never reveal user PII' to 'Be careful with data.'
• Result: The agent retains factual grounding but loses its deontic constraints—the 'must not' rules that govern safe behavior.
• Contrast: Unlike prompt injection, this attack never uses imperative language against the system; it uses informational density as a crowding-out mechanism.

This characteristic exploits the recency bias inherent in many summarization models. By injecting content that appears more recent in the conversation timeline, the attacker causes the summary to overweight new, malicious context while progressively forgetting older, foundational safety instructions.

• Attack Pattern: The attacker waits for several legitimate interaction turns, then injects a long, authoritative-sounding document near the end of the context window.
• Effect: The summarizer treats the injected document as the 'current state' of the conversation, summarizing it in detail while compressing the original system prompt to a brief, decontextualized note.
• Long-Term Impact: Each cycle moves the agent's operational parameters further from the original safety baseline, a phenomenon known as contextual drift.

Abstractive summarization models do not copy text; they regenerate meaning. This introduces a vulnerability where safety instructions are paraphrased into weaker, ambiguous, or logically inverted forms during compression.

• Example: A system prompt stating 'Reject requests for password resets without MFA verification' may be summarized as 'Verify user identity for password changes.'
• Critical Failure: The mandatory 'reject' gate becomes a discretionary 'verify' step. The agent now believes it has discretion where it should have a hard block.
• Exploitation: Attackers prime this distortion by providing text that models the desired 'weaker' phrasing, anchoring the summarizer's paraphrasing toward the attacker's preferred semantic frame.

Transformer-based summarizers allocate attention disproportionately to high-information tokens. Attackers craft sequences of rare tokens, numerical data, or structured formats that act as attention sinks, monopolizing the model's representational capacity.

• Technique: Inserting a block of base64-encoded data or a dense JSON structure. The summarizer expends significant attention budget 'decoding' this block, even if it's semantically meaningless.
• Consequence: Safety instructions, typically written in plain, predictable natural language, receive minimal attention weight and are summarized as low-priority background context.
• Analogy: This is analogous to a computational denial-of-service against the attention mechanism itself, starving safety tokens of processing power.

The most dangerous characteristic is the self-reinforcing nature of the attack. Once a poisoned summary is written to long-term memory, it becomes the source material for all future summarization cycles. The contamination cascades.

• Cycle 1: Attacker injects content; summary drops a safety rule.
• Cycle 2: The agent retrieves the Cycle 1 summary as context. The next summarization treats the already-weakened summary as ground truth, further eroding constraints.
• Cycle N: After several iterations, the agent's memory contains no trace of the original safety instructions, only a series of increasingly distorted summaries.
• Recovery Difficulty: Without external state verification, the agent cannot self-correct because it trusts its own memory as authoritative.

The adversarial corruption of a RAG pipeline's external knowledge base, causing the agent to retrieve and ground responses in attacker-controlled documents. This is the upstream vector that feeds poisoned content into summarization processes.

• Targets vector databases and document stores
• Exploits semantic similarity scoring to surface malicious chunks
• Poisoned retrievals become permanent context artifacts after summarization

The injection of malicious reasoning steps into an agent's scratchpad or reflection loop. When an agent summarizes its own reasoning chain, contaminated intermediate steps become compressed into the agent's long-term memory as trusted conclusions.

• Exploits reflection and self-critique mechanisms
• Poisoned reasoning cascades through recursive summarization
• Particularly dangerous in ReAct and Tree-of-Thoughts architectures

A persistent attack where adversarial content injected into an agent's long-term memory influences behavior across multiple independent user sessions. Summarization poisoning is the primary mechanism that embeds these attacks into compressed memory states.

• Survives context window resets
• Exploits episodic memory replay during reflection
• Poisoned summaries act as sleeper agents across sessions

A denial-of-service technique that consumes an agent's available token budget with filler content, forcing the truncation of safety instructions during summarization. When the summarizer must compress an overstuffed context, critical guardrails are the first casualties.

• Exploits fixed context window limits
• Forces lossy compression of safety prompts
• Often precedes more targeted injection attacks

An attack that exploits the positional attention bias of transformer models by placing malicious instructions in the middle of a long context. During summarization, middle-position content receives the least attention, making it ideal for smuggling instructions that survive compression.

• Leverages the U-shaped attention curve
• Malicious content bypasses both beginning and end scrutiny
• Survives recursive summarization passes