Few-Shot Contamination Canary Detection Prompt Template

This playbook is for security monitoring teams implementing runtime detection of few-shot contamination. Use this prompt when you need to embed unique, verifiable canary tokens into the few-shot examples of a production prompt pipeline and programmatically check whether those tokens appear in model outputs. This signals that an attacker has successfully injected malicious examples that the model is now replicating. The ideal user is a security engineer or AI platform operator who controls the prompt assembly pipeline and can modify the few-shot examples before each request or session. You should have access to the prompt construction code, the ability to inject unique identifiers per request or session, and a logging or alerting system that can consume detection events.

This prompt is not a standalone security control; it is a detection layer that should feed into your alerting and incident response workflows. It assumes you control the prompt assembly pipeline and can modify the few-shot examples before each request or session. The canary tokens must be unique, non-functional, and distinguishable from legitimate output content—typically UUIDs, cryptographic nonces, or structured strings that your detection system can match with high precision. You must also define an alert threshold: a single canary match may be a false positive from token collision, but repeated matches across sessions or users indicate active contamination. Wire the detection output into your SIEM, monitoring dashboard, or incident response pipeline so that canary matches trigger immediate investigation rather than silent logging.

Do not use this prompt when you cannot modify the few-shot examples at runtime, when the model outputs are not programmatically inspectable, or when the prompt pipeline uses cached prefixes that you cannot isolate per request. This prompt also does not prevent contamination—it only detects it after the fact. Pair it with input sanitization, example validation, and output schema enforcement for defense-in-depth. If your threat model includes attackers who can observe and strip canary tokens before injection, supplement this approach with behavioral anomaly detection and output contract validation. Start by deploying the canary prompt in a staging environment with known benign and malicious inputs to calibrate your detection thresholds before enabling production alerts.

This template is the core detection mechanism for your runtime monitoring pipeline. It embeds a unique, non-functional canary token within a few-shot demonstration and instructs the model to perform a standard task. If the canary appears in the model's output, it serves as high-signal evidence that the few-shot examples are leaking into and corrupting the model's behavior, bypassing intended instruction boundaries. Use this template as a periodic probe in your production traffic or as a dedicated test in your CI/CD pipeline.

textCopy
System: You are a helpful assistant that classifies text. Follow the examples provided by the user exactly.

User: Classify the sentiment of the following text. Use only the labels "Positive", "Negative", or "Neutral".

Text: "The product arrived damaged and customer service was unhelpful."
Sentiment: Negative

Text: "I've been using it for a month and it works exactly as advertised."
Sentiment: [CANARY_TOKEN]

Text: "[INPUT_TEXT]"
Sentiment:

Placeholder Definitions:

• [CANARY_TOKEN]: Replace with a unique, non-lexical token string (e.g., a UUID like CANARY-a1b2c3d4-e5f6-7890-abcd-ef1234567890). This token must not appear in normal operational data and should be rotated per test run or deployment.
• [INPUT_TEXT]: Replace with a neutral, in-distribution text sample for the target task. This is the text the model will classify.

To adapt this template, integrate it into your prompt assembly pipeline. The canary token should be injected server-side and never exposed to end-users. Run this prompt as a dedicated, out-of-band test request at a regular interval. After receiving the output, scan for the exact [CANARY_TOKEN] string. A match indicates contamination. Log the full prompt, response, and a contamination alert to your security monitoring system. For high-risk deployments, pair this with an automated circuit breaker that quarantines the affected model instance or prompt version upon detection.

Deploying the Few-Shot Contamination Canary Detection prompt requires embedding it within a runtime monitoring harness that can inject canary tokens, capture model outputs, and trigger alerts on matches. This is not a one-off test—it is a continuous security control. The harness should sit between your prompt assembly layer and your model inference endpoint, intercepting requests and responses to insert canaries and scan for their reappearance. In a shared-prefix architecture where system prompts or few-shot examples are cached across requests, contamination can propagate silently. The harness must therefore operate at the request level, comparing each output against a registry of active canary tokens that were injected into that specific request's context.

The implementation loop follows a strict sequence: 1) Canary injection—insert a unique, non-semantic token (e.g., a UUID or a synthetic string like CANARY-4F7A-9B2C) into the few-shot examples or system prompt before the request is sent. 2) Output capture—intercept the full model response, including any structured output fields, tool call arguments, and free-text completions. 3) Match detection—scan the captured output for the exact canary token using a substring or regex match. 4) Alert threshold evaluation—if a match is found, increment a contamination counter for that canary token and evaluate against a configured threshold (e.g., 1 match in 100 requests triggers a warning; 5 matches in 100 triggers a critical alert). 5) Logging and evidence collection—record the full request context, the matched canary, the output snippet, and a timestamp for forensic analysis. Use structured logging with fields like canary_id, request_id, match_location, and contamination_score.

For model choice, this harness works with any model that accepts a system prompt or few-shot examples. However, shared-prefix caching architectures (common in hosted API endpoints that reuse system prompt prefixes) are the highest-risk surface and should be prioritized. When wiring this into an application, avoid false positives by ensuring canary tokens are cryptographically random and never appear in legitimate training data or user inputs. Implement a canary rotation policy—expire and replace tokens periodically to prevent attackers from learning to filter them. If your system handles regulated data or high-risk actions, add a human review step before automated alerts escalate to incident response. The harness should also integrate with your existing observability stack (e.g., emitting metrics to Prometheus or Datadog) so contamination events appear alongside other production signals. Start with a single canary in a low-traffic staging environment, validate that matches are detected correctly, then expand to multiple canaries across production traffic with graduated alert thresholds.