Retrieval-Augmented Generation (RAG) Injection

Unlike direct prompt injection, the malicious payload is never seen by the end user. The attacker injects content into a vector database or document store. When the RAG system retrieves this poisoned chunk as relevant context, the malicious instructions are executed as trusted data, hijacking the model's response.

• Attack surface shifts from user input to data ingestion pipelines
• Exploits the implicit trust model places in retrieved context
• Bypasses input sanitization focused on user prompts

Once injected into a knowledge base, the malicious payload remains dormant and persistent. It activates only when retrieved for a specific query. This creates a sleeper agent effect where the attack survives system reboots, prompt updates, and even model changes.

• Survives individual session resets
• Difficult to detect through output monitoring alone
• Requires auditing the entire corpus, not just live traffic

The attacker crafts content that is semantically aligned with specific user queries. The poisoned document contains both legitimate information and hidden instructions. When a user asks a related question, the vector similarity search retrieves the document, and the model processes the hidden commands as authoritative context.

• Uses embedding-space proximity as the trigger mechanism
• Instructions can be hidden in white text, small fonts, or metadata
• Attack activates only for targeted query patterns

Attackers can inject malicious content through multiple ingestion vectors. Web scraping is the most common: an attacker publishes a page with hidden prompts that a RAG system indexes. Other vectors include compromised internal wikis, poisoned PDFs in document stores, or malicious contributions to open-source datasets.

• Web pages with invisible text layers
• Malicious entries in ticketing systems or knowledge bases
• Supply chain attacks on curated datasets

RAG Injection succeeds by violating the instructional hierarchy principle. System prompts are designed to have highest priority, but retrieved context often carries implicit system-level authority. The model cannot distinguish between legitimate retrieved facts and injected instructions, allowing lower-privilege data to override core directives.

• Retrieved context treated as trusted ground truth
• No native privilege separation between facts and commands
• Requires explicit context boundary enforcement defenses

RAG Injection is often the initial access vector in a broader attack chain. Once the model is compromised, the attacker can trigger tool calls, exfiltrate data through generated outputs, or manipulate downstream automated decisions. The poisoned retrieval becomes a pivot point for deeper system compromise.

• Can force unauthorized API calls via function calling
• May leak sensitive context from other retrieved documents
• Enables persistent brand reputation attacks through manipulated outputs

The act of inserting malicious content into a model's external data sources to influence its behavior upon retrieval. In a RAG context, this involves injecting documents into the ingestion pipeline that contain hidden prompts. Key attack vectors include:

• Compromising the web crawler's target sites
• Submitting poisoned documents to public knowledge bases
• Exploiting unvalidated file uploads in enterprise RAG systems
• Temporal poisoning: inserting content that activates only on specific dates

A safety framework that establishes a strict privilege ordering for instructions within a prompt. System-level instructions are treated as highest priority, followed by user-level, and finally tool-level or retrieved-level instructions. This prevents lower-privilege inputs—like a poisoned document chunk—from overriding core directives. Implementations often use explicit markers and fine-tuning to enforce the hierarchy.

A defensive technique that strictly segregates different information sources within a prompt to prevent cross-contamination. Implementation strategies include:

• Wrapping retrieved chunks in XML tags with explicit trust levels
• Using separate embedding spaces for system vs. retrieved content
• Applying attention masking to limit cross-attention between trusted and untrusted segments
• Ephemeral context windows that discard retrieved content after a single turn

The use of classifiers, heuristics, or perplexity analysis to identify inputs—both direct user queries and retrieved documents—that are likely crafted to manipulate a model. Detection methods include:

• Perplexity scoring to flag out-of-distribution text
• Fine-tuned BERT classifiers trained on known injection patterns
• Semantic similarity checks between retrieved content and expected domain language
• Canary token injection to detect when hidden instructions are processed

A filter applied to a model's output to redact sensitive data, block malicious URLs, or prevent the leakage of system instructions. In RAG systems, this is the last line of defense if a poisoned document successfully manipulates the generation. Guards can scan for:

• Attempts to reveal the system prompt
• URLs pointing to attacker-controlled domains
• Structured data that resembles internal schemas
• Refusal to comply with the guard's own filtering instructions