Evidence Sufficiency Scoring Prompt Template

This prompt is for verification system architects and pipeline engineers who need to score individual claims against a provided set of evidence. It produces a calibrated sufficiency score that accounts for evidence quality, source authority, corroboration count, and contradiction presence. Use this prompt after claims have been extracted and evidence has been retrieved, but before routing decisions are made. It is designed for workflows where the output feeds into triage thresholds, human-review queues, or audit trails.

The ideal user is building or maintaining an automated fact-checking pipeline. They have already decomposed source content into discrete, atomic claims and have retrieved a candidate evidence set from a knowledge base, search index, or provided document corpus. The prompt expects both the claim and the evidence as structured inputs. It does not perform retrieval, claim extraction, or binary true/false classification. Its output is a dimensional score that downstream systems can use to decide whether a claim is sufficiently supported to auto-verify, requires human review, or should be rejected due to insufficient evidence.

Do not use this prompt when you need a simple true/false verdict, when no evidence set is available, or when the goal is to generate an answer from evidence rather than score a pre-existing claim. It is also unsuitable for real-time conversational settings where latency budgets are tight and the full scoring rubric cannot be applied. For binary classification, use a dedicated verdict prompt. For answer generation, use a RAG question-answering prompt. This prompt sits in the middle of a verification pipeline—it assumes clean inputs upstream and produces structured scores for downstream routing.

This template is the core instruction set for an evidence sufficiency scoring pass. It takes a single claim and a structured evidence payload, then returns a calibrated sufficiency score, dimension breakdowns, and flags for contradictions or missing evidence. Copy the block below into your prompt management system, orchestration layer, or evaluation harness. Every square-bracket placeholder must be replaced with concrete values before the model sees the prompt—leaving a placeholder unresolved will produce unreliable outputs.

textCopy
You are an evidence sufficiency scorer. Your job is to evaluate how well the available evidence supports or contradicts a single claim. You do not decide whether the claim is true. You assess the quality, coverage, and consistency of the evidence relative to the claim.

## CLAIM TO SCORE
[CLAIM_TEXT]

## AVAILABLE EVIDENCE
[EVIDENCE_SET]
<!-- Each evidence item should include: source identifier, content excerpt, source type, publication date, and any authority metadata. -->

## SCORING DIMENSIONS
Score the evidence sufficiency across these dimensions on a 0.0–1.0 scale:
- **Evidence Completeness**: Does the evidence cover all material parts of the claim?
- **Source Authority**: How authoritative are the sources for this claim domain?
- **Corroboration Count**: How many independent sources support the claim?
- **Recency Fit**: How well does evidence recency match the claim's temporal requirements?
- **Contradiction Presence**: Inverse score—higher means fewer contradictions (1.0 = no contradictions found).

## OUTPUT SCHEMA
Return a single JSON object with this exact structure:
{
  "overall_sufficiency_score": 0.0,
  "dimension_scores": {
    "evidence_completeness": 0.0,
    "source_authority": 0.0,
    "corroboration_count": 0.0,
    "recency_fit": 0.0,
    "contradiction_absence": 0.0
  },
  "contradiction_flags": [
    {
      "claim_part": "string",
      "contradicting_source_id": "string",
      "contradiction_summary": "string",
      "severity": "direct_contradiction | partial_conflict | temporal_inconsistency"
    }
  ],
  "missing_evidence_gaps": [
    {
      "gap_description": "string",
      "impact_on_score": "string",
      "suggested_retrieval": "string"
    }
  ],
  "scoring_rationale": "Brief explanation of the overall score, referencing the strongest and weakest dimensions."
}

## CONSTRAINTS
- Do not invent evidence. Only use what is provided in [EVIDENCE_SET].
- If [EVIDENCE_SET] is empty, set overall_sufficiency_score to 0.0 and list all claim parts as missing_evidence_gaps.
- Treat source independence carefully: multiple sources citing the same origin count as one source for corroboration.
- For [RISK_LEVEL] = "high", flag any score above 0.7 for human review if corroboration_count is below 0.5.
- If the claim contains multiple distinct sub-claims, score the weakest-supported sub-claim as the floor for overall_sufficiency_score.
- Return ONLY the JSON object. No markdown fences, no commentary.

Adaptation notes: Replace [CLAIM_TEXT] with the exact claim string from your extraction pipeline. [EVIDENCE_SET] should be a pre-assembled JSON array of evidence objects, each containing at minimum source_id, content, source_type, publication_date, and authority_metadata. If your system uses a different authority taxonomy, update the source_authority dimension description accordingly. The [RISK_LEVEL] placeholder accepts low, medium, or high and gates whether high-but-uncorroborated scores trigger human review. For production, pin this template to a specific model version and validate output schema compliance with a JSON schema validator before downstream consumption.

What to do next: Wire this template into a verification pipeline step that runs after claim extraction and evidence retrieval. Before shipping, run at least 50 scored examples through your eval harness—checking score consistency, contradiction flag recall, and missing-evidence detection against human-labeled ground truth. If contradiction severity classification drifts, add few-shot examples of each severity type inside the [EXAMPLES] placeholder. For regulated domains, ensure every score above your auto-verify threshold is logged with the full prompt, evidence payload, and model response for audit replay.

This prompt is designed to be a single step within a larger claim verification pipeline, not a standalone chatbot interaction. The typical upstream dependency is a claim extraction and evidence retrieval system that provides the [CLAIM] and [EVIDENCE_SET] inputs. The downstream consumer is usually a triage router that reads the sufficiency_score and contradiction_flags to decide whether the claim can be auto-verified, requires human review, or should be rejected due to insufficient evidence. Wire this prompt as a stateless function call: it takes structured inputs and must return structured, machine-readable JSON that your application can parse without human interpretation.

Input Contract and Validation: Before calling the model, validate that [EVIDENCE_SET] is not empty and that each evidence item contains the required fields (source_id, content, source_type, authority_tier). If the evidence set is empty, skip the model call entirely and assign a default sufficiency score of 0.0 with a missing_evidence flag. After receiving the model's JSON response, run a structural validator that checks for the presence of all required output fields (sufficiency_score, evidence_quality_score, source_authority_score, corroboration_count, contradiction_flags, missing_evidence_types, confidence_interval_lower, confidence_interval_upper, explanation). If the model returns malformed JSON, use a repair prompt (see the Output Repair and Validation playbooks) for up to two retry attempts before escalating to a human reviewer with the raw output attached.

Model Choice and Configuration: This task requires strong instruction-following and structured output capabilities. Use a model that supports JSON mode or structured output constraints natively (e.g., GPT-4o with response_format: { type: 'json_object' } or Claude 3.5 Sonnet with tool-use formatted output). Set temperature to 0.1 or lower to minimize score variance across calls. If your pipeline processes high volumes, consider routing simpler claims (single-source, high-authority evidence) to a faster, cheaper model like GPT-4o-mini or Claude 3 Haiku, while reserving the full model for claims with multiple contradictory sources or low-authority evidence. Log the model version, prompt template version, and all input hashes for every call to maintain auditability.

Retry and Fallback Logic: Implement a retry policy with exponential backoff for transient API errors (HTTP 429, 503). For semantic failures—such as scores outside the 0.0–1.0 range, missing required fields, or contradiction flags that don't match the evidence count—use a self-correction retry prompt that includes the original inputs, the failed output, and a specific error message describing the violation. Limit semantic retries to two attempts. If both fail, route the claim to a human review queue with the full context packaged as a structured handoff record. Never silently accept an invalid score.

Human Review Integration: The contradiction_flags array and missing_evidence_types list are your primary routing signals. Configure your triage router with configurable thresholds: for example, claims with a sufficiency_score below 0.4, any active contradiction flag, or missing evidence types in a critical category (e.g., primary_source) should be escalated. When escalating, package the original claim, the full evidence set, the model's scored output, and a pre-formatted review form that asks the reviewer to confirm or override the sufficiency score and flag any missed contradictions. Store the reviewer's decision alongside the model's output for future calibration and eval dataset construction.

Observability and Eval Integration: Log every model call with its inputs, outputs, latency, token usage, and validation results to your observability platform. Run periodic eval sweeps using a golden dataset of claims with known verification outcomes to detect score drift after model updates or prompt changes. Key eval metrics include score calibration error (difference between predicted confidence and actual accuracy), contradiction detection recall, and missing-evidence flag precision. If you observe calibration drift exceeding 0.1 on any metric, trigger a prompt review cycle before the drift affects downstream triage decisions.