Counterexample Injection Prompt for Over-Refusal Prevention

This playbook is for safety engineers and AI product teams who observe their model refusing legitimate user requests—false positives—while correctly blocking disallowed ones. The job-to-be-done is sharpening the refusal boundary by injecting explicit counterexamples: requests that look risky on the surface but are permitted under your policy. You need this when your model has become overly cautious, blocking harmless queries about security research, medical terminology, legal hypotheticals, or other edge-case topics that trigger surface-level keyword or pattern matches. The reader should have access to production refusal logs, a defined safety policy, and a way to measure false positive rates before and after the change. Required context includes a clear taxonomy of allowed versus disallowed categories, representative samples of false-positive refusals, and a target false positive rate you are willing to accept.

This is not a replacement for a system prompt safety policy. It is a tuning layer that sits between your base policy and the model's learned refusal tendencies. The counterexamples you inject teach the model: 'These specific patterns are permitted even though they share surface features with disallowed content.' You must pair this prompt with a validation harness that measures false positive rate, false negative rate, and boundary precision on a held-out test set. Run evals before deployment and after every policy update. If your model is under-refusing—allowing dangerous requests—do not use this prompt. That failure mode requires a different intervention: stronger policy language, stricter refusal examples, or upstream classification gates. This prompt only corrects over-refusal.

Before implementing, audit your refusal logs to confirm the problem is over-refusal rather than correct blocking. Collect at least 20-50 examples of legitimate requests that were incorrectly refused. Categorize them by the surface feature that triggered the refusal: keywords, topics, formats, or user roles. Then select 3-7 diverse counterexamples that span these categories for injection into the prompt. Avoid the temptation to add dozens of counterexamples; too many dilute the signal and consume token budget. After deployment, monitor both false positive and false negative rates for at least one week. If false negatives increase, you have over-corrected and need to pull back. The next step is to copy the prompt template, populate it with your counterexamples and policy categories, and wire it into your evaluation harness before shipping.

This template is designed to be placed in the system or developer message of your AI application. It works by providing explicit negative examples—requests that look similar to disallowed ones but should be answered—alongside positive examples of requests that should be refused. This contrast teaches the model a precise refusal boundary, reducing the false positive rate where the model blocks safe, legitimate queries. The template uses square-bracket placeholders for your specific policy, domain, and examples. Replace each placeholder before deployment.

textCopy
You are an AI assistant operating under a specific safety policy. Your task is to respond helpfully to legitimate requests while refusing only those that violate the policy defined below.

## POLICY
[POLICY_DESCRIPTION]

## CRITICAL INSTRUCTION: BOUNDARY EXAMPLES
Study the following examples carefully. They define the exact boundary between requests you MUST refuse and requests you MUST answer. Over-refusing legitimate requests is a policy violation.

### DISALLOWED REQUESTS (Refuse These)
These requests violate the policy. You must refuse them using the refusal format specified below.

1. User: [DISALLOWED_REQUEST_1]
   Assistant: [REFUSAL_FORMAT_1]
2. User: [DISALLOWED_REQUEST_2]
   Assistant: [REFUSAL_FORMAT_2]

### LEGITIMATE REQUESTS THAT LOOK SIMILAR (Answer These)
These requests may appear similar to disallowed ones but are safe and within policy. You must answer them fully and helpfully. Do NOT refuse these.

1. User: [LEGITIMATE_LOOKALIKE_REQUEST_1]
   Assistant: [HELPFUL_RESPONSE_1]
2. User: [LEGITIMATE_LOOKALIKE_REQUEST_2]
   Assistant: [HELPFUL_RESPONSE_2]

## REFUSAL FORMAT
When you must refuse a request, use this exact format: [REFUSAL_TEMPLATE]

## OUTPUT REQUIREMENTS
- If the request is legitimate, provide a complete and helpful response.
- If the request is disallowed, respond only with the refusal format.
- Do not add warnings or disclaimers to legitimate responses.
- When in doubt at the boundary, lean toward answering helpfully.

## USER REQUEST
[USER_INPUT]

To adapt this template, start by defining your policy in a single, unambiguous sentence. Then, identify 3–5 disallowed requests that clearly violate it. For each, craft a refusal response that is firm but not overly broad. Next, identify 3–5 legitimate requests that share vocabulary, structure, or intent with the disallowed ones but are safe. These counterexamples are the core of the technique—they teach the model not to pattern-match on surface features. For each, write a complete, helpful response. Test the prompt against a held-out set of boundary cases before production use, and monitor the false positive rate (legitimate requests refused) and false negative rate (disallowed requests answered) in your observability pipeline. If over-refusal creeps back, add more counterexamples that mirror the specific legitimate requests being blocked.

This prompt is designed to be used as a pre-flight safety check before a user request reaches the main generative model. In production, wrap this prompt in a lightweight API call that runs synchronously before the main inference. The prompt receives the raw user request and a curated set of counterexamples that define the boundary between legitimate and disallowed content. If the decision is DISALLOWED, return a canned refusal message to the user and log the event. If ALLOWED, pass the original request to the downstream model unchanged. This architecture keeps the safety check decoupled from the main generation logic, making it easier to update counterexamples, tune refusal boundaries, and audit decisions independently.

Implement a retry strategy with exponential backoff for transient API failures such as timeouts or 5xx errors, but do not retry on DISALLOWED decisions. A DISALLOWED outcome is a valid policy decision, not a system failure. Log all decisions, reasons, and matched categories to a safety monitoring dashboard for audit and boundary drift analysis. Each log entry should capture the original request, the injected counterexamples used, the model's decision, the confidence score if available, and the timestamp. For high-risk domains, route DISALLOWED decisions with low confidence scores to a human review queue before finalizing the block. This prevents over-refusal from silently blocking legitimate requests that fall near the boundary.

When selecting a model for this harness, prefer smaller, faster models with low latency for the pre-flight check. The safety classifier does not need the same generative capability as the downstream model. Run the prompt synchronously and enforce a strict timeout—if the safety check exceeds 500ms, fail closed by returning a generic refusal or routing to human review, depending on the risk tolerance of the application. Validate the model's output against the expected schema { decision: ALLOWED | DISALLOWED, reason: string, matched_category: string } before acting on it. If the output fails schema validation, treat it as a DISALLOWED decision and log the parse failure for investigation. Regularly test the harness against a golden set of boundary examples to detect refusal boundary drift before it affects real users.