Evidence Contradiction Severity Scoring Prompt Template

This prompt is a precision instrument for a specific point in your verification pipeline: after claim extraction and evidence matching have identified a contradiction, but before a final confidence score is assigned. Its job is to quantify the severity of that contradiction—not to detect it, not to resolve it, and not to make the final routing decision. The ideal user is a verification system architect or pipeline developer who needs a calibrated, structured severity signal to feed into downstream logic. You should use this prompt when your system has already paired a specific claim with a specific piece of contradicting evidence and now needs to answer: 'Is this a minor discrepancy, a major rebuttal, or a direct refutation?' The output is a severity score, an impact assessment on the claim's overall confidence, and a resolution difficulty estimate—all of which become inputs to your confidence adjustment model, triage router, or human review queue.

This prompt is not a standalone fact-checker. Do not use it to detect contradictions from raw text; it assumes the contradiction is already identified and the claim-evidence pair is provided as input. It is also not a replacement for a full confidence scoring system. The severity score it produces is a sub-signal, not a final verdict. In a production pipeline, this prompt sits between your evidence matching module and your confidence calibration module. For example, if your retrieval system returns a source that directly states the opposite of a claim, you would first run a contradiction detection prompt to confirm the conflict exists, then pass the confirmed pair to this severity scorer. The resulting severity score (e.g., 0.85 on a 0-1 scale) would then be consumed by your confidence adjustment prompt to reduce the claim's overall confidence proportionally. The resolution difficulty estimate helps your triage router decide whether this contradiction can be auto-resolved by finding more evidence or requires human judgment.

Before implementing this prompt, ensure your pipeline has clear contracts for the input claim and evidence objects. The claim should be a discrete, atomic assertion with its source context preserved. The evidence passage should be a specific, cited excerpt, not a full document. If your upstream claim extraction produces compound claims or your evidence matching returns entire documents, the severity scorer will produce unreliable results. Start by testing this prompt against a labeled dataset of human-judged contradiction severities to calibrate your thresholds. Common failure modes include over-scoring semantic disagreements as factual contradictions and under-scoring numerical contradictions where the units differ. The next step after reading this playbook is to copy the prompt template, adapt the output schema to match your pipeline's data model, and run it against your calibration set before wiring it into production.

This prompt template is designed to be dropped into your verification pipeline when you have already matched evidence to a claim and need to quantify the severity of any contradictions found. It produces a structured contradiction severity score, an impact assessment on overall claim confidence, and a resolution difficulty rating. The template assumes you have already completed claim extraction and evidence matching; it does not perform those steps. Use this when your system needs to decide whether a contradiction is a minor discrepancy that can be noted or a fundamental conflict that should downgrade confidence significantly.

textCopy
You are a contradiction severity assessor for a fact-checking verification pipeline. Your task is to evaluate how severely a set of provided evidence contradicts a specific claim, and to produce a structured severity score with supporting justification.

## INPUT

**Claim to evaluate:**
[CLAIM_TEXT]

**Claim domain/category:** [CLAIM_DOMAIN]

**Evidence set (each item includes source metadata):**
[EVIDENCE_ARRAY]

**Current claim confidence before contradiction assessment:** [PRIOR_CONFIDENCE]

## OUTPUT SCHEMA

Return a valid JSON object with exactly these fields:

```json
{
  "contradiction_severity_score": <float 0.0-1.0>,
  "severity_level": <"none" | "minor" | "moderate" | "major" | "critical">,
  "contradicted_elements": [<list of specific claim components contradicted>],
  "contradicting_evidence_ids": [<list of evidence item IDs that contradict>],
  "supporting_evidence_ids": [<list of evidence item IDs that support>],
  "impact_on_confidence": <float representing recommended confidence reduction>,
  "resolution_difficulty": <"trivial" | "easy" | "moderate" | "difficult" | "unresolvable">,
  "contradiction_type": <"direct_factual" | "numerical_disagreement" | "temporal_inconsistency" | "source_disagreement" | "definitional_conflict" | "scope_mismatch">,
  "severity_justification": <string explaining the score with specific evidence references>,
  "resolution_suggestion": <string describing what additional evidence or clarification would resolve the contradiction>,
  "requires_human_review": <boolean>
}

SEVERITY CALIBRATION REFERENCE

• 0.0-0.1 (none): No contradiction detected. Evidence is consistent with the claim.
• 0.1-0.3 (minor): Minor discrepancy in detail, wording, or precision that does not affect the core claim truth value.
• 0.3-0.6 (moderate): Meaningful disagreement on a supporting element. Claim may still be substantially true but requires qualification.
• 0.6-0.85 (major): Direct contradiction of a central claim element. Claim as stated is likely false or misleading.
• 0.85-1.0 (critical): Multiple independent sources directly contradict the core claim. Claim is demonstrably false.

ASSESSMENT RULES

1. Evaluate contradictions against the claim as stated, not against what you think the claim intended to mean.
2. Distinguish between contradictions in core claim elements versus peripheral details. Weight core contradictions more heavily.
3. Consider source authority and recency when weighing contradicting evidence. A contradiction from a low-authority or outdated source should reduce severity compared to a contradiction from a highly authoritative, current source.
4. If multiple independent sources contradict the claim, increase severity. If all contradicting sources trace back to a single origin, note this and reduce severity accordingly.
5. For numerical claims, apply a tolerance window of [NUMERICAL_TOLERANCE_PERCENT]% before flagging as contradiction.
6. If the evidence set contains both supporting and contradicting items, assess the net contradiction severity rather than averaging.
7. Flag for human review when severity_level is "major" or "critical", or when contradicting sources have comparable authority to supporting sources.

CONSTRAINTS

• Do not invent evidence not present in the input evidence set.
• Do not evaluate the truth of the claim independently; only assess contradiction against the provided evidence.
• If the evidence set is empty or insufficient for contradiction assessment, set severity_score to 0.0, severity_level to "none", and note the insufficiency in severity_justification.
• If evidence items conflict with each other (not just with the claim), note this in severity_justification but do not let it inflate the contradiction score against the claim.
• Output only the JSON object. No additional text, explanations, or markdown fences.

To adapt this template for your pipeline, replace the square-bracket placeholders with actual values before each call. [CLAIM_TEXT] should contain the exact claim string from your extraction step. [EVIDENCE_ARRAY] should be a serialized JSON array of evidence objects, each with at minimum an id, text, source_authority_score, and recency_date field. [PRIOR_CONFIDENCE] is the confidence score your system assigned before contradiction assessment, which helps the model calibrate impact. [NUMERICAL_TOLERANCE_PERCENT] should be set based on your domain—5% for general claims, 1-2% for financial or scientific claims. [CLAIM_DOMAIN] helps the model apply domain-appropriate severity thresholds. If your evidence objects use different field names, update the output schema field descriptions accordingly, but preserve the JSON structure for downstream parsing.

The contradiction severity scoring prompt is not a standalone tool—it is a scoring function inside a larger verification pipeline. In production, it typically sits after claim extraction and evidence matching, receiving a claim and a set of contradictory evidence items as input. The output severity score feeds into downstream confidence adjustment, triage routing, or human review escalation. The harness must treat this prompt as a deterministic scorer with well-defined input contracts, not as an open-ended conversational agent. Every invocation should receive the same structured input schema, and every output should be validated against the expected severity scale before any downstream system consumes it.

Wire the prompt into your application with a strict JSON input contract: {"claim": "[CLAIM_TEXT]", "contradictory_evidence": [{"source_id": "[ID]", "source_text": "[TEXT]", "source_authority_tier": "[TIER]"}], "context_domain": "[DOMAIN]", "severity_scale": "[SCALE_DEFINITION]"}. Parse the model's JSON output and validate that severity_score falls within the defined scale range, impact_on_confidence is a valid adjustment value, and resolution_difficulty matches your expected enum. If validation fails, retry once with the validation error appended to the prompt as a correction instruction. If the retry also fails, route the claim to a human review queue with the raw model output and validation failure reason attached. Log every invocation—input, output, validation result, retry count, and final disposition—for audit and calibration analysis.

Model choice matters for consistency. Use a model with strong JSON-following behavior and low temperature (0.0–0.2) to reduce score variance across identical inputs. If your pipeline processes claims in batches, consider caching severity scores for identical claim-evidence pairs to avoid redundant API calls. For high-stakes domains—legal, medical, financial—always require human review when the severity score exceeds a pre-defined threshold or when the resolution difficulty is marked as 'high.' Never treat the severity score as ground truth; treat it as a calibrated signal that reduces, but does not eliminate, the need for human judgment on consequential contradictions.