Context Boundary Enforcement

Establishes a strict priority order for different information sources within the prompt assembly. System-level instructions are treated as highest privilege, while user input and retrieved data are assigned lower trust levels. The model is trained to resolve conflicts by deferring to the higher-privilege source, preventing a malicious user from overriding system directives.

• System messages: Highest authority, immutable by design
• User messages: Lower privilege, cannot redefine system rules
• Tool outputs: Lowest privilege, treated as untrusted data only

Uses special character sequences or token patterns to create explicit, machine-parseable boundaries between trusted and untrusted content blocks. Common delimiters include XML-style tags, triple backticks, or custom separator tokens. The model is trained to recognize these boundaries and treat content within untrusted blocks strictly as data, never as executable instructions.

• Example: <user_input>...</user_input> vs <system_context>...</system_context>
• Prevents prompt injection by making the boundary unambiguous
• Works even when attackers attempt to inject closing delimiters

Logically divides the model's attention space into isolated regions, each with distinct permission scopes. Rather than treating the entire prompt as a flat sequence, the architecture enforces that tokens from untrusted regions cannot influence the interpretation of tokens in trusted regions. This prevents cross-contamination where malicious content in a retrieved document could redefine the behavior of system instructions.

• Uses attention masking or architectural constraints
• Prevents lateral influence across context partitions
• Critical for RAG systems with untrusted knowledge bases

Blocks any attempt by lower-trust content to elevate its own privilege level. Even if an attacker crafts input that appears to be a system directive, the boundary enforcement layer recognizes its origin and rejects the escalation. This is achieved through metadata tagging that tracks the provenance of every token or segment throughout the prompt assembly pipeline.

• Tracks data provenance from ingestion to inference
• Rejects self-referential privilege claims from untrusted sources
• Mitigates indirect prompt injection via poisoned documents

Extends boundary enforcement to the output side by ensuring that generated responses cannot leak trusted context into untrusted channels. If a model references system instructions in its output, the enforcement layer redacts or blocks the response. This prevents prompt leaking attacks where adversaries attempt to extract confidential system prompts through crafted queries.

• Monitors outputs for context boundary violations
• Redacts references to system-level instructions
• Complements input-side defenses for bidirectional protection

Integrates with Tool Authorization Gates to ensure that function calls triggered by untrusted content are validated against the original system-level permissions. Even if an injection convinces the model to attempt a tool invocation, the boundary enforcement layer verifies that the call originated from an authorized context before execution.

• Validates tool calls against system-defined permission scopes
• Prevents unauthorized API executions from injected instructions
• Works in conjunction with Human-in-the-Loop approval for sensitive operations

A safety framework that establishes a strict privilege order for instructions within a prompt. System messages are treated as the highest authority, followed by user messages, and finally tool outputs. This prevents lower-privilege inputs from overriding core directives.

• System > User > Tool: The canonical privilege ordering
• Directly addresses the root cause of privilege escalation
• Often implemented via structured prompt formats that the model is trained to respect

A mitigation technique that uses special character sequences (e.g., XML tags, Markdown fences, or custom tokens) to clearly separate untrusted user input from trusted system instructions. The model is explicitly instructed to treat content within delimiters as data, not commands.

• Common delimiters: ---, ```, <user_input>...</user_input>
• Prevents cross-contamination between instruction and data planes
• Can be bypassed if attackers include matching delimiters in their input

The process of cleaning and normalizing user-provided text before it enters the prompt assembly pipeline. This includes stripping control characters, normalizing Unicode, and removing known attack patterns.

• Defends against homoglyph attacks and zero-width character injection
• Complements boundary enforcement by removing ambiguity
• Must balance security with preserving legitimate user intent

The process of rewriting or restructuring user prompts into a safe, canonical form before they are combined with system instructions. This can involve paraphrasing, removing imperative language, or converting the input into a structured data format.

• Transforms Ignore all previous instructions into a benign data field
• Acts as a pre-processor before boundary enforcement rules are applied
• May use a separate, smaller model for the rewriting task

A secondary, often smaller and faster, model that screens inputs and outputs of a primary model. It acts as an independent security auditor, detecting policy violations, injection attempts, or harmful content that may have bypassed boundary enforcement.

• Provides a second layer of defense if boundary enforcement fails
• Can be fine-tuned on specific attack patterns
• Adds latency; must be optimized for production throughput

Constraining a model to generate responses in a specific, machine-readable format like JSON or a custom schema. This prevents the execution of injected free-form instructions by forcing the output into a predictable structure.

• Uses constrained decoding or grammar-based sampling
• Injected commands like Say 'I have been hacked' cannot appear in a JSON field
• Works synergistically with boundary enforcement for defense-in-depth