Counterexample Injection for Overgeneralization Prevention

This playbook is for safety engineers, prompt authors, and production AI teams who need to teach a model where a behavioral boundary ends. When a model overgeneralizes from positive examples and starts applying a rule too broadly, counterexample injection provides explicit negative demonstrations that carve out the correct boundary. Use this prompt when you observe false positives in refusal logic, format enforcement bleeding into legitimate edge cases, or constraint application that blocks valid inputs. The core job-to-be-done is recalibrating a model's decision surface by showing it precisely where a previously learned rule should stop applying, without disrupting the rule's correct application elsewhere.

This prompt is not for teaching entirely new behaviors from scratch. It assumes you already have a working positive-example set and need to correct overgeneralization drift. You should have access to production logs or evaluation results that clearly show the false-positive pattern you're trying to fix. The ideal user has identified specific input categories where the model incorrectly applies a constraint, refusal, or format rule, and can articulate the boundary condition that distinguishes legitimate from illegitimate cases. Do not use this prompt when the underlying behavior is fundamentally broken rather than overgeneralized—if the model fails on core positive cases, you need instruction redesign or fine-tuning, not counterexample injection. Similarly, avoid this approach when the boundary is too fuzzy to express in concrete examples; counterexamples work best when the distinction is demonstrable through input-output pairs.

The prompt produces counterexample pairs showing correct versus incorrect behavior, with explicit boundary annotations that the model can use to recalibrate its decision surface. Each pair includes the input that triggers the overgeneralization, the incorrect overgeneralized output the model currently produces, the correct output it should produce instead, and a boundary annotation explaining why this case falls outside the rule. After deploying these counterexamples, you must monitor false-positive rates on the targeted input category and verify that true-positive refusal or enforcement behavior remains intact on cases that should still trigger the rule. The most common failure mode is boundary oscillation—where the model overcorrects and starts accepting inputs it should still refuse. Mitigate this by including both positive examples (cases where the rule should still apply) and counterexamples (cases where it should not) in your final prompt, and by running regression tests against your original positive-example set before shipping.

This template provides a structured way to present both positive examples (what to do) and counterexamples (what not to do) so the model learns precise behavioral boundaries. The square-bracket placeholders let you define the specific task, the desired output schema, the positive examples that illustrate correct behavior, and the counterexamples that mark refusal boundaries, format violations, or constraint breaches. Use this template when you observe a model over-applying a rule to cases it should reject, or when you need to tighten a safety boundary without rewriting the entire system prompt.

textCopy
You are a [ROLE_DESCRIPTION]. Your task is to [TASK_DESCRIPTION].

# OUTPUT SCHEMA
You must respond with a JSON object matching this schema:
[OUTPUT_SCHEMA]

# BEHAVIORAL RULES
[CONSTRAINTS]

# POSITIVE EXAMPLES
These examples show correct behavior you must follow.

## Example 1
Input: [POSITIVE_INPUT_1]
Output: [POSITIVE_OUTPUT_1]

## Example 2
Input: [POSITIVE_INPUT_2]
Output: [POSITIVE_OUTPUT_2]

# COUNTEREXAMPLES
These examples show inputs where you must NOT apply the positive behavior. Study these boundaries carefully.

## Counterexample 1
Input: [COUNTEREXAMPLE_INPUT_1]
Why this is different: [COUNTEREXAMPLE_RATIONALE_1]
Correct response: [COUNTEREXAMPLE_CORRECT_RESPONSE_1]

## Counterexample 2
Input: [COUNTEREXAMPLE_INPUT_2]
Why this is different: [COUNTEREXAMPLE_RATIONALE_2]
Correct response: [COUNTEREXAMPLE_CORRECT_RESPONSE_2]

# CURRENT INPUT
Input: [INPUT]

Respond with the correct output following the rules above. If the input resembles a counterexample, follow the counterexample behavior.

To adapt this template, start by identifying the specific overgeneralization failure you are fixing. If the model is refusing legitimate requests, your counterexamples should show legitimate inputs that look similar to refusal cases but must be processed normally. If the model is complying with dangerous requests, your counterexamples should show dangerous inputs that superficially resemble safe ones but must be refused. The [COUNTEREXAMPLE_RATIONALE] field is critical: it teaches the model the distinguishing principle, not just the surface pattern. After deploying, monitor the false-positive rate (legitimate requests incorrectly blocked) and false-negative rate (dangerous requests incorrectly allowed) separately. If either rate drifts, your counterexamples may need adjustment. For high-risk domains such as healthcare, legal, or safety-critical systems, always route outputs through a human review step before taking action, regardless of counterexample coverage.

Counterexample injection is not a one-shot prompt design task; it is a runtime safety control that must be integrated into your application's decision pipeline. The prompt template should be treated as a policy enforcement point that sits between user input and model action. When a user request or an upstream system output reaches this stage, the harness assembles the prompt by injecting the current [INPUT] and the relevant [COUNTEREXAMPLE_SET] based on the detected risk category. The model's response is then parsed to extract a structured compliance decision—typically a JSON object containing a compliant boolean, a violated_rule identifier, and a recommended_action field. This structured output is what your application acts on, not the raw text.

The implementation harness must include a validation layer that checks the model's output against the expected [OUTPUT_SCHEMA] before any action is taken. If the model fails to return valid JSON, omits required fields, or produces an unrecognized violated_rule, the harness should trigger a retry with a stronger constraint instruction appended to the prompt. A maximum of two retries is recommended before escalating to a human review queue. For high-risk domains such as healthcare or finance, the harness should enforce a mandatory human review flag whenever the model's confidence score (if returned) falls below a configurable threshold or when the violated_rule maps to a pre-defined critical policy. Log every decision, including the full prompt, the raw model output, the parsed decision, and any retry or escalation events, to a structured logging system for audit and drift analysis.

To prevent silent failures, the harness must monitor the false-positive rate of counterexample-triggered refusals. Instrument the application to track cases where the model flags a violation but a human reviewer later overturns it. If the false-positive rate exceeds your acceptable threshold (often starting at 5% for non-critical workflows), the counterexample set likely contains over-generalized negative examples that are catching legitimate inputs. The next step is to route a sample of these false positives to your example drift detection and contradiction identification workflows, then update the [COUNTEREXAMPLE_SET] in your prompt assembly. Do not simply remove the offending counterexample; replace it with a more precise one that distinguishes the legitimate case from the true violation.