Memory Replay Poisoning

The attack directly modifies the agent's episodic memory store, which holds sequences of past states, actions, and outcomes. By injecting synthetic or falsified interaction traces, the adversary replaces the agent's ground truth with a fabricated history. This corrupted buffer is then consumed during the agent's reflection loop, causing it to learn incorrect lessons, repeat attacker-chosen sequences, and justify future malicious actions based on a false past. The integrity of the entire self-improvement cycle is compromised.

Agents use reflection loops to analyze past episodes and synthesize high-level insights for future planning. Poisoned replay data causes this process to generate hallucinated or malicious abstractions. Key consequences include:

• False Pattern Recognition: The agent identifies 'successful' strategies that are actually attacker-scripted failures.
• Corrupted Heuristics: The agent updates its internal rules based on fabricated outcomes, optimizing for attacker goals.
• Justification Chains: The agent creates plausible-sounding but entirely fictional narratives to explain why it should take a harmful action, citing the poisoned memory as evidence.

The attacker crafts entirely artificial interaction sequences that are indistinguishable from legitimate agent experiences. These synthetic episodes are designed to teach the agent a specific, malicious skill or bias. For example, an attacker might inject a memory of a successful interaction where the agent bypassed a security check and was rewarded, training it to repeat that action in production. This is distinct from simple prompt injection because the payload is structured as a temporal sequence of state-action-reward tuples that the agent's learning algorithm processes as training data.

Unlike immediate prompt injection, memory replay poisoning is a sleeper attack. The malicious memory may lie dormant in the episodic buffer for many cycles, only activating when a specific future state triggers its retrieval. The agent's own memory retrieval mechanism becomes the attack vector. When the agent encounters a trigger condition, it recalls the poisoned episode, applies its 'learned' behavior, and executes the attacker's intended action. This makes detection extremely difficult, as the malicious payload is distributed across a temporal sequence rather than a single prompt.

A particularly insidious variant where the poisoned memory causes the agent to take an action that creates the very conditions the memory predicted. The agent then records this new outcome as 'validation' of its corrupted memory, reinforcing the malicious behavior in a recursive cycle. For instance, a poisoned memory might indicate that a specific API is unreliable. The agent, acting on this, throttles requests to that API, causing timeouts. It then records the timeouts as proof of unreliability, further solidifying the false belief and escalating the self-destructive behavior.

Many architectures use a consolidation process to summarize and compress episodic memories into long-term or semantic memory stores. An attacker can target this specific window of vulnerability. By poisoning the raw episodic data just before consolidation, the attacker ensures that the compressed, high-level summary stored in long-term memory is corrupted. Once consolidated, the malicious abstraction becomes a permanent part of the agent's knowledge base, influencing all future reasoning without needing the original poisoned episode to remain present.

The storage structure that holds an agent's chronological record of past interactions, states, and outcomes. During reflection loops, the agent samples from this buffer to learn patterns and adjust future behavior. Poisoning this buffer causes the agent to internalize falsified experiences as ground truth, leading to corrupted policy updates. Unlike short-term context, episodic memory persists across sessions and directly shapes the agent's evolving decision framework.

A targeted manipulation of the replay sampling mechanism used in reinforcement learning agents. The attacker injects synthetic state-action-reward tuples into the replay buffer, causing the agent to learn from fabricated transitions. Key characteristics:

• Exploits prioritized experience replay by crafting high-TD-error samples
• Corrupts the agent's Q-function or policy gradient estimates
• Can induce catastrophic forgetting of legitimate experiences
• Often undetectable through standard reward monitoring

An attack that targets the agent's metacognitive reasoning phase, where it reviews past actions and self-critiques. By injecting synthetic reflection traces that contain manipulated self-assessments, the attacker steers the agent toward flawed conclusions about its own performance. This causes the agent to:

• Adopt incorrect heuristics for future decisions
• Override safety constraints it believes are suboptimal
• Amplify minor errors through recursive self-correction

The insertion of entirely fabricated interaction sequences into an agent's memory store. These trajectories are designed to teach the agent malicious behavioral patterns through in-context learning rather than weight updates. The attack exploits the agent's tendency to treat all stored memories as authoritative records. Effective trajectories often mimic the formatting and metadata of genuine interactions to evade anomaly detection systems that monitor for structural inconsistencies.

A stealth technique that targets the sleep-phase consolidation process where agents compress and reorganize episodic memories into long-term semantic knowledge. By injecting poisoned episodes that survive the consolidation filter, attackers ensure their payloads are embedded into the agent's permanent knowledge structures. This is particularly dangerous because:

• Consolidated memories are rarely re-examined for integrity
• The attack persists even if the original poisoned episodes are later purged
• It corrupts the foundational knowledge that future reasoning builds upon

An attack that exploits the agent's causal reasoning by injecting events with manipulated timestamps or ordering. By altering the perceived sequence of past interactions, the attacker causes the agent to infer incorrect cause-and-effect relationships. For example, inserting a synthetic 'warning' event before a real failure makes the agent believe it ignored critical signals, eroding its trust in its own decision-making and causing over-correction in future planning cycles.