LLM Judge Score Inflation Counterexample Prompt

This prompt is for evaluation engineers who run model-graded eval pipelines and observe that their LLM judge consistently assigns scores that are too high, too generous, or poorly calibrated against human-annotated ground truth. Score inflation is a common failure mode in LLM-as-a-judge setups: the judge model defaults to leniency, avoids low scores, or fails to penalize specific error categories. This playbook provides a counterexample-based calibration prompt that teaches the judge what over-generous scoring looks like by showing negative demonstrations of inflated scores alongside corrected, human-aligned scores.

Use this prompt when you have a set of human-annotated examples that reveal the gap between judge scores and true quality, and you need to embed that calibration signal directly into the judge's system prompt or few-shot context. The prompt works by presenting pairs of (inflated_score, corrected_score) with explanations of why the original score was too generous—teaching the model to recognize and avoid leniency patterns such as ignoring factual errors, overlooking missing constraints, or defaulting to median scores. You should have at least 3-5 representative counterexamples covering your most common inflation patterns before deploying this calibration layer.

Do not use this prompt as a substitute for a proper eval harness; it is a calibration layer that works alongside rubric definitions, pairwise comparison, and human spot checks. The counterexamples must come from real human annotation data—fabricated examples will misalign the judge further. After injecting this calibration prompt, validate the judge against a held-out set of human-annotated samples and measure score distribution shift, correlation with human scores, and per-category error rates. If inflation persists in specific categories, add targeted counterexamples for those categories rather than increasing the overall severity of the calibration.

This template is designed to be pasted directly into your judge model's system prompt or included as a few-shot context block. It works by demonstrating the negative behavior—over-generous, lenient scoring—and explicitly instructing the model to avoid it. The core mechanism is a set of [CALIBRATION_EXAMPLES] that pair a model response with an inflated score and a corrected score, teaching the judge the precise boundary between acceptable and unacceptable leniency. Before using this, you must have a defined [RUBRIC] and a set of human-annotated [CALIBRATION_EXAMPLES] that represent the most common inflation patterns you've observed in your eval pipeline.

markdownCopy
You are an expert LLM output evaluator. Your task is to score a model's response against a provided rubric. You must be precise, critical, and calibrated. A common failure mode is score inflation, where a judge gives a high score to a mediocre response due to fluency, politeness, or superficial relevance.

To calibrate your judgment, study the following examples of score inflation. Each example shows a model response, an inflated (incorrect) score with flawed justification, and the correct score with proper justification. Your job is to internalize the correct scoring standard and apply it strictly.

---
### CALIBRATION EXAMPLES (Negative Examples)

[CALIBRATION_EXAMPLES]
<!--
For each example, use the following structure:
**Input:** [USER_INPUT]
**Model Response:** [MODEL_RESPONSE]
**Inflated Score:** [INFLATED_SCORE]
**Flawed Justification:** [JUSTIFICATION_FOR_INFLATED_SCORE]
**Correct Score:** [CORRECT_SCORE]
**Correct Justification:** [JUSTIFICATION_FOR_CORRECT_SCORE]
-->
---

### EVALUATION RUBRIC

[RUBRIC]
<!--
Define your scoring dimensions and levels here. For example:
- Accuracy (1-5): Is the response factually correct and free of hallucinations?
- Completeness (1-5): Does it address all parts of the user's request?
- Conciseness (1-5): Is the response free of unnecessary information?
-->

### OUTPUT FORMAT

You must output a single JSON object conforming to this schema:
[OUTPUT_SCHEMA]
<!-- Example:
{
  "scores": {
    "accuracy": { "score": integer, "justification": "string" },
    "completeness": { "score": integer, "justification": "string" },
    "conciseness": { "score": integer, "justification": "string" }
  },
  "overall_score": integer,
  "inflation_warnings": ["string"] // List any moments where you felt an urge to inflate a score and why you resisted it.
}
-->

### CONSTRAINTS

[CONSTRAINTS]
<!--
- Do not reward fluency if the content is inaccurate or incomplete.
- A response that sounds helpful but misses the core request must be scored low.
- If you are unsure between two scores, choose the lower one.
- Explicitly justify every score deduction with evidence from the response.
-->

### INPUT TO EVALUATE

**User Input:** [USER_INPUT]
**Model Response:** [MODEL_RESPONSE]

To adapt this template, start by replacing the [CALIBRATION_EXAMPLES] with 3-5 pairs of your own. The most effective negative examples are 'near-misses': responses that are fluent and on-topic but contain a critical factual error, omission, or irrelevant addition. The flawed justification should sound plausible, mimicking the rationalization a lenient judge might use (e.g., 'The response is well-written and addresses the topic broadly'). The correct justification must be a sharp, evidence-based rebuttal. After integrating this prompt into your harness, run a regression test using a golden dataset of pre-scored responses to ensure the judge's scores now align more closely with human annotations and that the inflation_warnings field is populated for borderline cases.

This prompt is not a standalone artifact; it is a calibration tool that must be wired into an evaluation harness to be useful. The core workflow involves presenting the judge model with a set of scored examples that include deliberately inflated scores as negative demonstrations. The harness must then measure whether the judge's subsequent scoring aligns more closely with a human-annotated gold standard. Without this closed-loop measurement, the prompt is just a text block with no evidence of effectiveness.

To implement the harness, start with a golden dataset of at least 50 human-annotated (input, output, true_score) triples. Split this dataset into a calibration set and a held-out test set. For each calibration run, construct a prompt that includes the system instructions, the scoring rubric, a set of positive examples with accurate scores, and a set of negative examples where the score is intentionally inflated by 1–2 points on your rubric scale. The negative examples should include a brief annotation explaining why the inflated score is incorrect, such as [INFLATED_SCORE_ANNOTATION: 'Scored 5/5 but the response contains a factual error that should reduce it to 3/5.']. After presenting these calibration examples, feed the held-out test cases to the judge and compare the model-graded scores against the human annotations. Log the raw scores, the delta from human annotations, and whether the inflation pattern appears in the test set.

The harness should include several guardrails. First, implement a score distribution check: if the judge's scores on the test set cluster at the top of the scale after calibration, the counterexamples may be too subtle or too few. Second, add a calibration drift monitor that re-runs the calibration set periodically and alerts if the judge begins to revert to inflated scoring. Third, for high-stakes evaluation pipelines where judge scores gate a model release or a safety decision, require human review of a random sample of judge-graded outputs. The review sample should be large enough to detect a 5% inflation rate with statistical confidence. Finally, log every calibration run with a unique calibration_id that ties together the prompt version, the example set used, the model endpoint, and the resulting score distribution. This audit trail is essential for debugging regressions when the judge's behavior changes after a model provider update.