Contradiction Severity Scoring Prompt with Rubric

This prompt is designed for triage systems that must prioritize detected contradictions by their real-world impact. It takes a structured contradiction record—containing the conflicting claims, their sources, and the immediate context—and produces a severity score with a rubric-aligned justification. The primary job-to-be-done is automated, explainable prioritization: when your verification pipeline generates more contradictions than your team can manually review, this prompt routes the most critical conflicts to human reviewers first, ensuring that high-impact factual disagreements are addressed before minor terminological mismatches.

Use this prompt when you have already completed the detection phase. Your upstream system must provide a clean contradiction object as input, including the two conflicting statements, their source documents, and a classification of the conflict type (e.g., direct contradiction, numerical disagreement). The ideal user is a verification pipeline engineer or an operations lead who needs to implement a cost-effective triage step before human review. This prompt is not a detection tool; it will not find new contradictions. It is also not a resolution tool; it will not reconcile the conflict or declare which claim is correct. Its sole function is to assess the potential harm or consequence of a known contradiction.

Do not use this prompt for real-time, user-facing chatbots where latency is critical, as the rubric-based reasoning step adds processing time. Avoid it when the cost of a missed low-severity contradiction is catastrophic; in regulated domains like pharmacovigilance or safety-critical engineering, a human should review every detected contradiction, and this prompt should only serve as a secondary prioritization aid, not a filter. Before deploying, you must calibrate the severity thresholds against your organization's risk tolerance by running a golden dataset of contradictions with known human severity judgments through the prompt and tuning the rubric criteria until the scores align with your operational priorities.

This template is the core instruction set for a model acting as a contradiction triage classifier. It forces structured, rubric-aligned output that your application can parse and act on—routing critical factual conflicts for immediate human review while allowing minor stylistic disagreements to pass through. The prompt is designed to be stateless and idempotent; given the same input pair and rubric, it should produce the same severity score. Replace every square-bracket placeholder with live data from your upstream contradiction detection system before sending this to the model.

textCopy
You are a contradiction severity scoring engine. Your task is to evaluate a detected contradiction between two text snippets and assign a severity score based on a defined rubric. You must output only a valid JSON object conforming to the provided schema. Do not include any other text, commentary, or markdown formatting.

## INPUT DATA
- **Snippet A:** [SOURCE_A_TEXT]
- **Snippet B:** [SOURCE_B_TEXT]
- **Contradiction Summary:** [CONTRADICTION_DESCRIPTION]
- **Domain Context:** [DOMAIN]

## SCORING RUBRIC
Evaluate the contradiction across the following three dimensions, each on a 1-5 scale (1=Negligible, 5=Critical). Then, calculate the final severity score as the maximum of the three dimension scores.

### 1. Factual Materiality (1-5)
- **1 (Negligible):** Trivial detail with no impact on understanding (e.g., minor date typo, synonym variation).
- **2 (Minor):** Peripheral detail; a reader's core understanding remains intact.
- **3 (Moderate):** A supporting fact is wrong; could mislead a reader about a non-central point.
- **4 (Major):** A central claim or key evidence is contradicted; significantly alters the narrative.
- **5 (Critical):** The core thesis or primary conclusion is directly and completely contradicted.

### 2. Downstream Consequence (1-5)
- **1 (None):** No conceivable real-world impact if left uncorrected.
- **2 (Low):** Potential for minor confusion or a small operational nuisance.
- **3 (Medium):** Could lead to a flawed decision in a low-stakes process or waste minor resources.
- **4 (High):** Likely to cause a wrong decision with financial, legal, or health implications.
- **5 (Severe):** Imminent risk of significant harm, major financial loss, or violation of a critical safety constraint.

### 3. Correction Urgency (1-5)
- **1 (Routine):** Can be corrected in the next scheduled update cycle.
- **2 (Low Priority):** Should be flagged for the next review but does not require immediate action.
- **3 (Standard):** Requires a correction task to be created and prioritized within the current sprint.
- **4 (High Priority):** Needs immediate attention; should interrupt the current workflow.
- **5 (Critical):** Requires an immediate halt to publication or dependent processes and an emergency notification.

## OUTPUT SCHEMA
You must output a single JSON object with the following exact structure:
{
  "severity_score": <integer 1-5>,
  "severity_label": "<Negligible|Minor|Moderate|Major|Critical>",
  "dimension_scores": {
    "factual_materiality": <integer 1-5>,
    "downstream_consequence": <integer 1-5>,
    "correction_urgency": <integer 1-5>
  },
  "justification": "<Concise explanation linking the specific contradiction to the chosen scores.>"
}

## CONSTRAINTS
- The final `severity_score` must be the maximum of the three dimension scores.
- Map the final score to `severity_label` as: 1=Negligible, 2=Minor, 3=Moderate, 4=Major, 5=Critical.
- The `justification` must reference specific content from Snippet A and Snippet B.
- If the input is nonsensical or the contradiction is logically impossible, default to a severity_score of 1 with an appropriate justification.

To adapt this for your application, replace the placeholders with the output of your contradiction detection step. [SOURCE_A_TEXT] and [SOURCE_B_TEXT] should be the raw, conflicting text excerpts. [CONTRADICTION_DESCRIPTION] is a one-line summary from your detector. [DOMAIN] provides essential context (e.g., 'medical', 'legal', 'financial') that the model can use to calibrate the 'Downstream Consequence' score. The output schema is intentionally strict; parse the JSON response in your application code and use the severity_score integer to route the item to the correct queue. For high-risk domains, always log the full JSON payload for audit trails and potential human review of the justification.

This prompt is designed to sit inside a triage pipeline that processes contradiction outputs from upstream detection systems. The typical flow: a contradiction detection prompt identifies a conflict pair, then this severity scoring prompt assigns a priority score and justification. The output drives routing decisions—low-severity contradictions might be logged for audit, while high-severity contradictions trigger immediate human review or automated correction workflows. The harness must enforce a strict contract: the prompt expects a pre-validated contradiction pair with source excerpts and conflict type labels as input, and it must return a structured severity assessment that downstream systems can act on without re-parsing free text.

Wire this prompt into your application with a validation layer that checks both input completeness and output schema compliance before the score enters any routing logic. On the input side, verify that [CLAIM_PAIR] contains both claims with their source identifiers, that [CONFLICT_TYPE] is a recognized category from your upstream detector, and that [CONTEXT] includes the domain or subject matter for materiality assessment. On the output side, enforce that the response contains a numeric severity_score (typically 1-5 or 1-10), a rubric_category label matching your defined severity tiers, and a justification field that cites specific rubric criteria. Implement a retry loop with a maximum of two attempts: if the model returns an unparseable score or omits required fields, re-invoke with a repair prompt that includes the original response and a terse instruction to fix only the structural errors. Log every scoring attempt—including retries—with the prompt version, model, and timestamp for auditability. For model selection, use a model with strong instruction-following and structured output capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller or older models that may drift toward verbose justifications without consistent numeric scoring.

Calibration is the most critical operational concern. Before deploying this prompt to production routing, run a calibration batch against a golden dataset of 50-100 contradiction pairs that have been independently scored by at least two human reviewers. Compare the model's severity distribution against human judgments using Cohen's kappa or quadratic weighted kappa to measure agreement. Tune the rubric language in the prompt—particularly the boundary definitions between severity levels—until agreement exceeds your threshold (typically 0.7+ for production triage). After deployment, implement a sampling-based monitoring loop: pull a random sample of scored contradictions weekly, have a human reviewer re-score them, and track drift in the score distribution. If the model starts over-scoring (severity inflation) or under-scoring (missed critical contradictions), recalibrate the rubric or adjust the routing thresholds. Never route high-severity contradictions directly to automated correction without human review unless your calibration data shows near-perfect precision at the top severity tier. The cost of a false high-severity flag is wasted reviewer time; the cost of a missed high-severity contradiction is uncorrected factual error in production. Set your routing thresholds conservatively and widen them only as calibration data supports it.