Prompt Hardening

A safety framework that establishes a strict privilege ordering for instructions based on their origin. System-level directives are assigned the highest priority, followed by user inputs, and finally third-party data sources. When conflicts arise, the model is trained to defer to the higher-privilege source, preventing lower-trust inputs from overriding core operational constraints. This is implemented through structured prompt formatting that explicitly demarcates authority levels using delimiters and priority tags.

A structural defense that uses explicit boundary markers such as XML tags, triple backticks, or custom tokens to logically partition the context window. By wrapping untrusted user input in clearly defined delimiters and instructing the model to treat content within those boundaries as data rather than instructions, the attack surface for delimiter injection is reduced. This technique requires consistent parsing logic and explicit instructions that the model must never reinterpret delimited content as executable commands.

A design principle that restricts an agent's access to tools, data, and execution scope to the absolute minimum required for a specific task. Each prompt is crafted to grant only the necessary capabilities for its defined workflow, limiting the blast radius of a successful injection. Key practices include:

• Tool allowlisting per task context
• Scoped API keys with narrow permissions
• Stateless execution where possible to prevent context persistence attacks
• Explicit rejection of out-of-scope requests

A mitigation technique that constrains model generation to a predefined, machine-readable schema such as JSON, function calls, or typed responses. By forcing outputs through a deterministic parsing layer, free-form text injection vectors are neutralized. The model is instructed to only produce outputs that conform to the schema, and any deviation triggers a validation failure. This makes it significantly harder for attackers to exfiltrate data or execute arbitrary instructions through the response channel.

A detection mechanism that embeds unique, decoy strings within system prompts that serve as tripwires for unauthorized extraction attempts. These tokens are designed to be statistically improbable in normal conversation and are monitored through external logging systems. If a canary token appears in model outputs or external forums, it signals a successful prompt extraction attack. This technique provides post-breach detection and attribution capabilities rather than prevention.

A strategy that logically partitions the context window into isolated zones with strict separation between untrusted data and system instructions. Each segment is assigned a trust level, and cross-segment contamination is explicitly forbidden through prompt engineering. Implementation approaches include:

• Attention masking to limit token interaction across segments
• Prefix caching to maintain immutable system instruction blocks
• Sequential processing where user input is evaluated before integration with system context

An architectural pattern that executes agent function calls in sandboxed environments to prevent a compromised model from directly affecting critical infrastructure. Each tool invocation runs with:

• Least privilege access to only the resources required
• Ephemeral containers that are destroyed after execution
• Parameter validation against strict allowlists
• Human-in-the-loop approval gates for high-risk operations This containment strategy limits the blast radius of a successful prompt injection, ensuring that even a fully compromised agent cannot escalate privileges or access sensitive systems.