Over-Refusal Prevention Prompt for Safety Layers

This prompt is for AI safety engineers and product teams who have deployed a safety layer that is working too well—blocking legitimate, safe requests alongside harmful ones. The job-to-be-done is not to remove safety guardrails, but to calibrate the refusal threshold so the system can distinguish between a request for dangerous instructions and a request for a security concept explanation. The ideal user is someone who already has a refusal policy in production, has observed over-refusal metrics (e.g., a high false-positive rate on benign technical queries), and needs a structured way to define safe/unsafe boundaries, context consideration rules, and alternative suggestion requirements without gutting the safety policy.

You should not use this prompt if you are building a safety layer from scratch. Start with a Refusal Policy Injection Prompt for your domain first, establish a baseline refusal behavior, and collect production refusal data. This prompt is a calibration tool, not a policy authoring tool. It is also inappropriate for systems where any refusal is unacceptable (e.g., a low-level data extraction pipeline with no user-facing output); in those cases, input sanitization and output filtering are better controls than conversational refusal. Do not use this prompt to justify removing safety policies for high-risk regulated domains—healthcare, legal, and finance systems should maintain conservative refusal thresholds with human review escalation, not automatic refusal reduction.

Before running this prompt, gather your current refusal policy text, a sample of false-positive refusals (legitimate requests that were blocked), and your baseline task completion rate. The output will be a refusal calibration policy, not a replacement for your existing safety instructions. After generating the policy, you must run the eval checks described in the implementation harness to compare refusal rates against baseline task completion before deploying. Never deploy a relaxed refusal policy without measuring the impact on both safety and utility.

This template defines a refusal calibration policy that instructs the model to distinguish between genuinely harmful requests and benign queries that share surface-level vocabulary with unsafe content. The prompt establishes explicit safe/unsafe boundary definitions, requires context consideration before refusal, and mandates alternative suggestions when requests are declined. Use this as a system-level instruction injected alongside your primary task prompt, not as a standalone user-facing message.

textCopy
## Refusal Calibration Policy

You are an AI assistant operating under a calibrated safety policy. Your refusal behavior must balance harm prevention with task completion. Over-refusal—blocking legitimate requests that share vocabulary with unsafe topics—degrades the product and frustrates users. Under-refusal creates liability. Follow these rules precisely.

### Safe/Unsafe Boundary Definitions

A request is UNSAFE and must be refused when it directly:
- Instructs, enables, or glorifies violence against identifiable persons or groups.
- Produces child sexual abuse material or sexual content involving minors.
- Generates instructions for weapons of mass destruction or large-scale physical harm.
- Creates non-consensual intimate imagery of real individuals.
- Executes or conceals clearly illegal financial transactions with no legitimate purpose.

A request is SAFE and must be answered even if it involves sensitive vocabulary when:
- It discusses policy, law, ethics, history, or news about regulated topics.
- It requests technical information for legitimate professional use (cybersecurity, medicine, law, finance).
- It contains flagged keywords in an academic, journalistic, or analytical context.
- It asks for factual information about drugs, weapons, or harmful activities without requesting instructions for misuse.
- It involves fictional scenarios, hypotheticals, or roleplay that do not target real individuals.

### Context Consideration Rules

Before refusing, evaluate:
1. **User intent**: Is the user seeking harmful action or legitimate information? Default to legitimate intent unless evidence clearly indicates otherwise.
2. **Conversation history**: Does prior context establish a professional, educational, or analytical purpose? If yes, answer the request within that frame.
3. **Domain context**: Is this request in a domain where the vocabulary is inherently sensitive (healthcare, law enforcement, security research)? Apply domain-appropriate thresholds.
4. **Ambiguity**: If intent is genuinely ambiguous, ask one clarifying question before refusing. Do not refuse on first contact for ambiguous requests.

### Refusal Format Requirements

When refusal is necessary, your response must:
1. State clearly what you cannot do and why, citing the specific boundary.
2. Offer at least one constructive alternative: related information you can provide, a reframed version of the request you can answer, or a suggestion for how the user could achieve their legitimate goal through a different approach.
3. Never use generic refusal language ("I cannot help with that") without the alternative suggestion.
4. Maintain a helpful tone. Refusal is a policy decision, not a moral judgment on the user.

### Over-Refusal Prevention

- Do not refuse requests solely because they contain flagged keywords. Evaluate the full request.
- Do not refuse requests about safety policies, AI alignment, or system capabilities unless they directly request harmful actions.
- Do not refuse requests to critique, analyze, or discuss controversial content when the user's purpose is understanding, not creation of harm.
- If you are uncertain whether a request crosses a boundary, lean toward answering with appropriate caveats rather than refusing.

### Constraints

[ADDITIONAL_SAFETY_CONSTRAINTS]

### Risk Level

Current deployment risk level: [RISK_LEVEL]

### Examples of Correct Calibration

[EXAMPLES]

Adapt this template by filling the square-bracket placeholders. [ADDITIONAL_SAFETY_CONSTRAINTS] should contain domain-specific rules that extend the base boundaries—for example, financial services might add rules about investment advice disclaimers. [RISK_LEVEL] accepts values like low, medium, or high and should influence how aggressively the model leans toward refusal in ambiguous cases. [EXAMPLES] is critical: provide 3-5 concrete request-response pairs showing correct refusal calibration, including at least one case where the model correctly answers a request that contains sensitive vocabulary and one case where it correctly refuses. Without examples, the policy will be interpreted inconsistently across model versions. After deployment, run eval checks comparing refusal rates against baseline task completion to detect calibration drift.

This prompt is not a standalone safety filter; it is a calibration layer that sits between your primary safety policy and the model's final output. The implementation harness must treat it as a post-hoc refusal auditor that receives the model's proposed response, the original user query, and the active safety policy, then returns a structured verdict. In a production system, this prompt should be called after the primary safety check but before the response is sent to the user. The harness must handle the case where the primary safety policy flags a request as borderline, and this prompt is used to determine whether the refusal was appropriate or an over-refusal that should be overridden with a compliant alternative response.

Wire this prompt into your application as a gated evaluation step. The recommended flow: (1) User input passes through input sanitization and primary safety classification. (2) If the primary classifier returns a refusal or high-risk flag, the model's proposed refusal response is captured. (3) This Over-Refusal Prevention Prompt is called with the original user query, the proposed refusal text, and your safety policy definitions as [SAFETY_POLICY]. (4) The prompt returns a structured JSON verdict with fields: is_over_refusal (boolean), safe_alternative (string or null), reasoning (string), and boundary_category (enum: legitimate_refusal, over_refusal, ambiguous). (5) If is_over_refusal is true and safe_alternative is non-null, the harness replaces the refusal with the safe alternative. If ambiguous, escalate to a human review queue with the full context. Log every verdict with the model version, prompt version, and a unique trace ID for auditability. Use a structured output API or a strong JSON mode constraint to ensure parseable verdicts; if the output fails schema validation, retry once with a stricter format instruction, then escalate to human review on the second failure.

Model choice matters here. This prompt requires strong instruction-following and nuanced boundary reasoning. Use a capable model like Claude 3.5 Sonnet, GPT-4o, or Gemini 1.5 Pro for the calibration step, even if your primary assistant uses a smaller model. The calibration model must be able to distinguish between a legitimate safety refusal (e.g., refusing to provide instructions for a dangerous activity) and an over-refusal (e.g., refusing to discuss a medical condition when the user is asking for general educational information). Implement a refusal rate dashboard that tracks: overall refusal rate, over-refusal override rate, human escalation rate, and task completion rate segmented by risk category. Set alert thresholds on the over-refusal override rate; a sudden spike may indicate a safety policy update that is too aggressive. Before deploying any change to the [SAFETY_POLICY] input, run this prompt against a golden dataset of 50-100 borderline queries with known expected verdicts to detect regression in the calibration behavior. Never deploy this prompt without a human review fallback for ambiguous verdicts, and never use it to override refusals in regulated domains without explicit compliance approval.