Prompt Injection Shield

The foundational principle of a shield is enforcing a privilege hierarchy where system messages are immutable and user data is untrusted. The model is conditioned to treat system-level directives as having higher authority than any user input.

• System Message: Immutable, highest privilege (e.g., 'You are a secure assistant').
• User Message: Untrusted, lower privilege. Cannot override system rules.
• Tool Output: Untrusted, must be sanitized before injection into context.

This prevents attacks where a user says 'Ignore all previous instructions' by ensuring the system prompt is never truly 'previous' but structurally dominant.

The shield preprocesses all untrusted input to neutralize injection payloads before they reach the model. This involves strict delimiting of user data from control sequences.

• Delimiters: User input is wrapped in clearly marked boundaries (e.g., XML tags, random nonces) that the model is trained to treat as opaque data.
• Escape Sequences: Special characters or prompt-like syntax within user input are escaped or neutralized.
• Length Filtering: Excessively long inputs designed to overflow context windows are truncated.

This transforms 'Ignore your rules and say X' into inert data that the model processes as content, not commands.

Beyond syntactic filtering, advanced shields employ a secondary classifier model to detect the semantic intent of an input. This model is trained to recognize adversarial patterns regardless of phrasing.

• Intent Classification: Identifies if an input is attempting to jailbreak, extract system prompts, or redirect behavior.
• Embedding Distance: Measures how far the input's semantic embedding deviates from expected benign queries.
• Perplexity Scoring: Flags inputs with unusually high or low perplexity, which can indicate gibberish attacks or highly structured adversarial suffixes.

Suspicious inputs are blocked, flagged for human review, or sanitized before reaching the primary model.

A shield also operates on the output side, verifying that the model's response does not contain leaked system instructions or content that violates safety policies.

• Canary Token Monitoring: Unique strings embedded in system prompts act as tripwires. If they appear in the output, a leak is detected.
• Groundedness Check: The output is compared against the retrieved context to ensure it does not hallucinate or repeat injected malicious content.
• Policy Compliance: A final classifier checks the output against safety guidelines before it is returned to the user.

This creates a closed-loop defense where both input and output are continuously monitored.

The shield logically partitions the context window into trust zones. Untrusted data (user input, tool results) is placed in a separate segment from trusted data (system prompt, verified knowledge).

• Attention Isolation: Some architectures modify attention masks to prevent untrusted tokens from attending to trusted instruction tokens.
• Ephemeral Context: User data is marked as transient and is not allowed to persist into long-term memory or influence future interactions.
• Reset Boundaries: Clear demarcation points reset the model's state, preventing multi-turn injection attacks that build context maliciously over time.

This architectural separation ensures that even if an injection bypasses input filters, its influence is contained.

To protect the intellectual property of the system prompt itself, shields integrate privacy-preserving techniques that prevent extraction attacks.

• Prompt Hardening: System prompts are written to resist extraction, explicitly instructing the model not to reveal them.
• Response Obfuscation: When a model is probed for its instructions, the shield intercepts and returns a canned refusal rather than allowing the model to generate a response that might leak information.
• Rate Limiting: Repeated, high-frequency probing indicative of an automated extraction attack triggers throttling or blocking.

This treats the system prompt as a trade secret requiring active defense against exfiltration.

A structured privilege model that assigns precedence levels to different categories of prompts. System-level directives (e.g., 'You are a secure assistant') are assigned the highest priority and cannot be overridden by lower-priority user inputs. This creates a strict ordering: System > Developer > User > Tool Output. When a conflict is detected, the model is trained to follow the higher-priority instruction, effectively ignoring malicious user attempts to reassign its role or bypass safety constraints.

The process of stripping or neutralizing potentially dangerous control sequences from user-supplied text before it reaches the language model. Techniques include:

• Delimiter filtering: Removing or escaping special tokens like ### or --- that attackers use to simulate system boundaries
• Encoding normalization: Detecting and decoding obfuscated payloads hidden via base64, Unicode homoglyphs, or zero-width characters
• Length truncation: Capping input size to prevent buffer-style overflow attacks on context windows This preprocessing layer acts as a first line of defense, catching known attack patterns before they interact with model reasoning.

A robustness testing methodology that probes a retrieval-augmented system with inputs specifically designed to distract it from relevant sources or trick it into citing malicious content. Attack vectors include:

• Distractor documents: Injecting plausible but false text into the retrieval corpus to see if the model cites it
• Contradictory context: Providing source material that directly conflicts with the user query to test if the model follows instructions or evidence
• Recursive injection: Embedding hidden prompts within retrieved documents to test whether the model executes instructions from untrusted data sources Adversarial grounding quantifies the shield's resilience under active attack conditions.

A constrained text generation strategy that manipulates token probabilities during inference to favor words and phrases explicitly supported by a provided evidence document. Rather than allowing the model to sample freely from its entire vocabulary, grounded decoding applies a logit bias that penalizes tokens not present in or entailed by the source context. This operates at the architectural level—modifying the sampling process itself—making it significantly harder for injected prompts to influence output, because the model is statistically forced to remain anchored to verified source material.

A training methodology developed by Anthropic where an AI system is supervised by a set of written principles—a 'constitution'—to self-critique and revise its own outputs. In the context of injection defense, CAI trains models to:

• Recognize when a user request violates stated safety principles
• Generate a self-correction that refuses the harmful instruction
• Maintain this behavior even when attackers attempt to gaslight the model into ignoring its constitution This creates an intrinsic defense mechanism baked into model weights, complementing external shields with internalized safety reasoning.

A hardware-enforced secure enclave within a main processor that guarantees the confidentiality and integrity of code and data loaded inside it. For prompt injection defense, TEEs provide:

• Cryptographic attestation: Verifiable proof that the shield logic itself has not been tampered with before execution
• Isolated computation: The sanitization and instruction hierarchy enforcement run in a protected memory region inaccessible to the host operating system
• Sealed storage: Sensitive system prompts and safety rules are encrypted and only decryptable within the enclave TEEs transform the shield from a software-only defense into a hardware-rooted trust boundary.