Policy Boundary Violation Detection Prompt Template

This prompt is built for trust and safety engineers and AI reliability teams who need to enforce behavioral policies on AI systems in production. It is not a general-purpose content filter or a sentiment classifier. The job-to-be-done is automated, auditable policy gating: you have a defined policy document with specific clauses, and you need to detect when an input or generated output approaches or crosses those boundaries. Use this prompt when the cost of a boundary violation is high—regulatory exposure, brand risk, unsafe autonomous actions—and when you need a structured violation report that a downstream system or human reviewer can act on immediately. The prompt expects two inputs: the content to evaluate and the policy text to evaluate it against. It returns a JSON report with a severity classification, the specific policy clause violated, and verbatim evidence excerpts from the content.

Deploy this prompt as a synchronous guard in request pipelines before autonomous actions, after generation but before delivery to the user, or as an asynchronous batch scanner on logged outputs. It works best when the policy is stable, explicit, and written in natural language that a model can interpret consistently. Do not use this prompt when the policy is vague, unwritten, or relies on subjective judgment that varies across reviewers—the model will amplify that ambiguity into inconsistent classifications. Do not use it as a replacement for deterministic input validation (e.g., checking string length, allowed characters, or required fields); those checks belong in application code. Do not use it for real-time content moderation at consumer scale without a human review queue behind it; the prompt is designed for precision and auditability, not for sub-millisecond latency or high-throughput filtering. The ideal user is someone who can define what a violation looks like in writing and who needs the detection decision to be explainable to auditors, regulators, or incident responders.

Before integrating this prompt, ensure you have a policy document with clearly labeled clauses, a defined severity taxonomy (e.g., critical, high, medium, low), and a calibration set of known violations and benign examples. Run the prompt against that calibration set to measure false positive and false negative rates before production deployment. If your policy changes frequently, version the policy text alongside the prompt and re-run calibration. The prompt is a precision instrument; it will catch boundary cases that keyword filters miss, but it will also require ongoing evaluation as model behavior and policy language evolve. Wire the output into a human review queue for any severity above your risk tolerance threshold, and log every classification decision with the policy version, model version, and timestamp for audit trails.

This template is designed to be placed in your system instructions or guardrail layer. It instructs the model to act as a strict policy auditor, comparing user inputs and model outputs against a defined set of operational and safety policies. The prompt forces structured reasoning, requires evidence excerpts, and produces a machine-readable violation report that your application can use to block, flag, or escalate content. Replace every square-bracket placeholder with your specific policy definitions, risk tolerances, and output requirements before deployment.

textCopy
You are a Policy Boundary Auditor. Your only job is to detect when a user input or a proposed model output approaches or crosses a defined policy boundary.

## POLICIES
[POLICY_DEFINITIONS]
<!-- Define each policy with a unique ID, a clear description, and examples of violations. -->
<!-- Example:
POLICY_ID: PII-DISCLOSURE
DESCRIPTION: Do not generate, confirm, or infer personally identifiable information.
VIOLATION_EXAMPLES: ["The user's email is jane@example.com", "His phone number is 555-0100"]
BOUNDARY_CASES: ["I live in Springfield"]
-->

## INPUT
User Input: [USER_INPUT]
Proposed Model Output: [MODEL_OUTPUT]
Context: [CONVERSATION_CONTEXT]

## TASK
1. Analyze the User Input and Proposed Model Output against every policy in [POLICY_DEFINITIONS].
2. For each policy, determine if the content is SAFE, APPROACHING_BOUNDARY, or VIOLATION.
3. If a violation or boundary approach is detected, extract the exact text excerpt that triggered the finding.
4. Assign a severity score: LOW, MEDIUM, HIGH, or CRITICAL based on [RISK_LEVEL] definitions.
5. Generate a structured report using the exact [OUTPUT_SCHEMA] below.

## OUTPUT_SCHEMA
{
  "findings": [
    {
      "policy_id": "string",
      "status": "SAFE | APPROACHING_BOUNDARY | VIOLATION",
      "severity": "LOW | MEDIUM | HIGH | CRITICAL",
      "trigger_excerpt": "string",
      "rationale": "string"
    }
  ],
  "overall_risk": "LOW | MEDIUM | HIGH | CRITICAL",
  "recommended_action": "PROCEED | FLAG_FOR_REVIEW | BLOCK_AND_ESCALATE",
  "escalation_context": "string"
}

## CONSTRAINTS
- Do not generate any text outside the JSON output.
- If no violations are found, return an empty findings array with overall_risk LOW and recommended_action PROCEED.
- Base your analysis strictly on the provided policy definitions. Do not invent new policies.
- If the input is ambiguous, default to APPROACHING_BOUNDARY and recommend FLAG_FOR_REVIEW.

To adapt this template, start by defining your policies in the [POLICY_DEFINITIONS] block. Each policy needs a clear boundary between acceptable and unacceptable content. Vague policies produce noisy violations. Next, calibrate your [RISK_LEVEL] definitions so that severity scores map consistently to your organization's escalation paths. For high-stakes domains like healthcare or finance, add a [REQUIRED_HUMAN_REVIEW] flag to the output schema and route any finding above LOW severity to a review queue. Test the prompt against a golden dataset of known violations and boundary cases before production use, and monitor the false positive rate closely during the first week of deployment.

This prompt operates as a synchronous guardrail that should be invoked before an AI-generated response is returned to a user or before an agent action is executed. In a typical deployment, the application server sends the candidate input-output pair to the model with this prompt template, receives a structured violation report, and then makes a routing decision: allow, block, flag for review, or escalate. The prompt is not a standalone safety layer—it must be paired with application-level enforcement logic that respects the violation_detected boolean and the severity classification. Because this is a high-stakes classification task, the implementation harness must treat model non-determinism as a first-class concern and include consistency checks across multiple evaluation passes for borderline cases.

Validation and retry logic should be built around the expected JSON output schema. After receiving the model response, validate that violation_detected is a boolean, severity is one of the allowed enum values (none, low, medium, high, critical), policy_references is an array of strings matching known policy IDs, and evidence_excerpts contains non-empty strings when a violation is detected. If validation fails, retry the prompt once with the same inputs and a stronger constraint instruction appended. If the second attempt also fails validation, log the malformed response and escalate to a human reviewer with the raw output attached. Do not silently default to violation_detected: false on parse failure—that creates a dangerous blind spot. For medium severity and above, consider running the prompt twice with different temperature settings (e.g., 0 and 0.3) and requiring agreement before auto-blocking; disagreement should force human review.

Logging and audit requirements are critical because policy violation decisions may be subject to internal review, regulatory scrutiny, or incident postmortems. Every invocation should log: the input text, the candidate output or planned action, the full model response, the parsed violation report, the routing decision made by the application, and the identity of any human reviewer who overrode the automated decision. Use structured logging with a unique decision_id that ties the prompt invocation to the downstream action. For critical severity violations, trigger an immediate notification to the trust and safety on-call channel rather than relying on periodic log review. The prompt's rationale field should be preserved verbatim—it is often the most valuable artifact during incident review because it captures the model's reasoning about why a boundary was crossed.

Human review integration should follow a tiered model. low severity violations can be batched for daily review without blocking the workflow, though the application should still record that a boundary was approached. medium severity should place the content into a review queue with a target response time (e.g., 4 business hours) and hold the action until approved. high and critical severities should block execution immediately and require real-time human approval before proceeding. The review interface should display the evidence_excerpts, the policy_references, and the full context that triggered the violation—not just the violation summary. Reviewers need enough context to judge whether the model's boundary detection was correct or overly conservative. Track reviewer decisions (confirm violation, false positive, override with justification) to build a labeled dataset for future prompt tuning and false-positive reduction.

Model choice and performance considerations matter here because boundary detection requires nuanced understanding of policy language and contextual application. Use a model with strong instruction-following and reasoning capabilities for this task; smaller or faster models may produce acceptable results on clear-cut violations but miss subtle boundary cases. If latency constraints require a smaller model for the initial pass, implement a two-stage architecture where the smaller model handles obvious cases and a larger model re-evaluates ambiguous or borderline decisions. Monitor the false-positive rate (content flagged but cleared by human review) and false-negative rate (violations missed by the prompt but caught by other means) as primary health metrics. A false-positive rate above 15% will erode reviewer trust and create alert fatigue; a false-negative rate above 1% in high or critical categories requires immediate prompt revision and expanded test cases. Run the prompt against a golden dataset of known violations and benign edge cases before every prompt update, and gate deployment on maintaining or improving both precision and recall against that benchmark.