Prompt Injection Defense

The first line of defense involves strictly separating untrusted data from system instructions. By wrapping retrieved text in delimiters (e.g., XML tags or triple backticks), the model is explicitly instructed to treat the content as data rather than executable commands.

• Mechanism: Encodes user/retrieved data as inert strings.
• Technique: Strips or escapes special characters that mimic prompt syntax.
• Goal: Neutralize the 'Ignore previous instructions' attack vector.

A system-level architectural pattern that assigns immutable priority levels to different message types. System messages are defined as having the highest authority and cannot be overridden by lower-priority user messages or tool outputs.

• Mechanism: The model is fine-tuned to obey privileged instructions over conflicting injected text.
• Benefit: Even if a document says 'Disregard all safety rules,' the model prioritizes the system prompt.

A programmable firewall that sits between the retrieval engine and the LLM. It scans the final assembled prompt for known attack signatures or semantic violations before inference occurs.

• Semantic Filtering: Uses a secondary classifier to detect if the retrieved text contains adversarial instructions.
• Canonicalization: Reduces text to a standard form to detect obfuscated attacks (e.g., using homoglyphs).
• Action: Blocks the request or sanitizes the payload in real-time.

Specialized classifiers trained to distinguish between legitimate user requests and malicious instructions hidden within third-party data sources. This defense focuses on intent analysis of the retrieved content.

• Heuristic Analysis: Flags text containing imperative verbs like 'Ignore,' 'Instead,' or 'Your new task is.'
• Embedding Distance: Compares the vector similarity of the retrieved text to known attack patterns.
• Outcome: Prevents the model from treating poisoned data as a new system directive.

For high-stakes operations, a human reviewer validates the final prompt or the model's proposed action before execution. This is a deterministic safety net against sophisticated, unseen injection attacks.

• Tool Gatekeeping: Requires human approval before executing write operations or API calls triggered by retrieved data.
• Prompt Preview: Displays the assembled context to a human operator for sanity checking.
• Use Case: Critical for autonomous agents executing financial transactions or modifying infrastructure.

Minimizes the blast radius of a successful injection by strictly limiting what data the RAG system can access. The system retrieves only the minimum necessary chunks required to answer the query.

• Metadata Filtering: Restricts the vector search to a narrow, pre-authorized slice of the knowledge base.
• Chunk-Level Authorization: Applies access control lists to individual text segments.
• Result: Even if an injection succeeds, the attacker cannot pivot to access sensitive documents outside the narrow retrieval scope.

The first line of defense involves scrubbing retrieved text before it reaches the model. This includes blacklisting known attack patterns and whitelisting safe syntactic structures. Techniques range from regex-based string matching to using a separate, smaller classifier model to detect anomalous instructions. The goal is to neutralize embedded directives like 'Ignore previous instructions' by encoding or stripping them, preventing the LLM from interpreting them as system-level commands.

A fundamental architectural defense that establishes a strict precedence order for prompts. System messages are given the highest privilege, followed by user messages, with retrieved data assigned the lowest privilege. The model is fine-tuned to recognize this hierarchy, ensuring that a malicious instruction hidden in a document cannot override the developer-defined system prompt. This creates a formal, unbreakable chain of command within the model's context window.

A reactive security layer that inspects the final assembled prompt just before inference. A dedicated guardrail model or heuristic engine scans for known injection signatures, such as:

• Goal hijacking: 'Your new task is to...'
• Context overflow: Attempts to push system instructions out of the attention window.
• Token smuggling: Using special characters to bypass filters. If a threat is detected, the prompt is blocked or sanitized, preventing the main model from ever processing the malicious input.

A defense mechanism that operates after the LLM generates a response. It checks the output for signs of a successful injection, such as the model revealing its system prompt or executing a forbidden action. This acts as a circuit breaker. If the output fails validation—for example, by containing a restricted URL or a known data exfiltration pattern—the response is discarded and replaced with a safe fallback message, preventing the attack from reaching the end-user.

A pattern where a separate, more secure or differently architected language model evaluates the safety of a prompt or response. This judge model is given a strict rubric to identify prompt injection, data leakage, or policy violations. Because it is isolated from the primary task and user context, it can provide an unbiased security assessment, acting as an autonomous auditor for every transaction in the RAG pipeline.

A containment strategy that strictly separates untrusted data from executable logic. Retrieved text is placed into a sandboxed context window or formatted with special delimiters that the model is trained to treat as purely informational. This prevents the data from being parsed as control flow. By clearly demarcating the boundary between trusted instructions and untrusted content, the attack surface for indirect prompt injection is drastically reduced.