Canary Tokens

A canary token must be a high-entropy string, typically a UUID or a long random base64 sequence. It must be statistically impossible to guess or brute-force. This ensures that any appearance of the token in an unexpected context is a definitive signal of a breach, not a coincidence.

• Use UUIDv4 or 256-bit random values
• Avoid dictionary words or predictable patterns
• Ensures a near-zero false positive rate

The token must be embedded naturally within the system prompt's semantic structure to avoid easy detection. It should appear as a plausible, mundane configuration parameter or a forgotten debug comment. Honeytokens hidden in plain sight are less likely to be filtered out by automated extraction scripts.

• Disguised as a fake API key or debug flag
• Placed in a comment block typical of the codebase
• Avoids standing out as an obvious security mechanism

To prevent replay attacks or false alarms from accidental leakage, the token should be cryptographically bound to the specific deployment or session. A HMAC (Hash-based Message Authentication Code) can tie the token to a deployment ID, ensuring that a token from a staging environment cannot be used to trigger an alert in production.

• Binds token to a specific agent instance
• Prevents cross-deployment token reuse
• Uses HMAC with a secret key for verification

The detection mechanism must operate completely independently of the compromised system. When a canary token is used, it should trigger an alert via a separate, hardened channel—such as a dedicated SIEM webhook, an AWS Lambda function, or a PagerDuty event—that the attacker cannot intercept or suppress.

• Uses a distinct API endpoint for logging
• Triggers alerts in a separate security infrastructure
• Avoids relying on the compromised agent's own logging

The token must be designed so that its mere presence in an external log or output is irrefutable proof of exfiltration. It should be a non-functional credential that, if submitted to its designated listener, immediately logs the source IP, timestamp, and user-agent, creating a forensic artifact that cannot be denied.

• Acts as a decoy credential (e.g., a fake AWS key)
• Logs all metadata upon use
• Provides a clear forensic trail for incident response

Tokens should have a defined time-to-live (TTL) and be scoped to a specific agent or session. A token that is valid indefinitely increases the window for false positives. Rotating tokens with each new deployment or session ensures that a leaked token is only valid for a short, known period.

• Implements automatic expiry after a set duration
• Scoped to a single agent or conversation thread
• Reduces noise from long-tail accidental leaks

The defensive practice of reinforcing system prompts with explicit boundaries and fallback logic to resist adversarial manipulation. Key techniques include:

• Adding delimiters to clearly separate instructions from data
• Embedding self-reminders that reiterate core constraints
• Implementing refusal templates for out-of-scope requests
• Using few-shot examples that demonstrate resilient behavior

Hardened prompts treat all user input as potentially hostile and never rely on politeness or assumed cooperation.

The process of cleansing user-supplied data by removing or encoding potentially dangerous characters and patterns before they reach the language model. Common operations include:

• Stripping null bytes and invisible Unicode control characters
• Normalizing homoglyphs to their ASCII equivalents
• Escaping markdown and code fence delimiters
• Truncating inputs to maximum safe token lengths

Sanitization acts as a first-line filter but must be paired with semantic defenses, as obfuscated payloads often evade pattern-based detection.

A security layer that intercepts and analyzes prompts and model responses in real-time to block injection attacks and enforce content policies. A prompt firewall typically:

• Scores inputs for adversarial intent using classifier models
• Validates outputs against allowed schemas and topics
• Logs all blocked attempts for security auditing
• Operates as a proxy between the application and the LLM

Unlike static sanitization, prompt firewalls leverage semantic understanding to detect novel attack patterns.

A secondary, smaller classifier trained to evaluate the safety and integrity of inputs and outputs, acting as an independent auditor for a primary language model. Guard models:

• Run in parallel with the main inference pipeline
• Detect policy violations, toxicity, and injection attempts
• Can trigger human review or automatic blocking
• Are often fine-tuned on adversarial examples specific to the deployment domain

This defense-in-depth approach ensures that even if the primary model is compromised, a separate system can halt malicious execution.

A design principle that restricts an agent's access to tools and data to the absolute minimum required for a specific task, limiting the blast radius of a compromise. Implementation involves:

• Scoping API keys to single-purpose functions
• Passing only necessary context to each tool call
• Revoking permissions immediately after task completion
• Avoiding ambient authority where agents inherit full user privileges

This principle ensures that even a fully prompt-injected agent cannot perform actions beyond its narrowly defined mandate.

A mitigation technique that constrains a model's generation to a predefined, machine-readable schema, making it difficult for attackers to inject free-form malicious commands. Methods include:

• JSON mode and function calling with strict parameter validation
• Grammar-constrained decoding using formal language specifications
• Rejecting any output that deviates from the expected structure
• Using deterministic post-processing to strip extraneous content

By eliminating the model's ability to produce arbitrary text, structured output closes a major exfiltration and command injection vector.