Evidence Sufficiency Assessment Report Prompt

This prompt is for verification pipeline operators who need to classify claims by evidence sufficiency rather than just true or false. It is designed for a specific job-to-be-done: producing a structured report that separates 'proven true' from 'not enough evidence,' with gap analysis and false-confidence detection checks. The ideal user is an engineer or technical operator integrating this prompt into an automated fact-checking system, positioned after claim extraction and evidence retrieval, and before final verdict generation or human review routing. Use this when your system must explain why a claim remains unverified, not just that it does.

Do not use this prompt for binary true/false classification, for generating conversational answers to user questions, or for making final publishing decisions without human review. It is not a replacement for a RAG answer generation prompt, nor is it designed for real-time chat. The prompt assumes that claims have already been extracted and that evidence has already been retrieved and provided in the [EVIDENCE_SET] input. If you feed it raw documents or unprocessed user queries, it will fail silently by hallucinating evidence structures or misclassifying claims. The output is a machine-readable JSON report intended for downstream routing logic, audit logging, or reviewer dashboards, not for direct display to end users.

Before deploying this prompt, ensure your pipeline validates the output against the expected schema, checks for hallucinated evidence references, and routes any claim with a confidence score below your defined threshold to a human review queue. The prompt includes instructions for false-confidence detection, but it cannot replace a robust eval harness. Run regression tests with claims that have known evidence gaps to confirm the model correctly distinguishes 'insufficient evidence' from 'proven false.' If your use case involves regulated domains like healthcare or finance, always require human sign-off on the final report before any action is taken.

This prompt template is designed to be pasted directly into your system prompt or a user message in your verification pipeline. It instructs the model to classify each claim by evidence sufficiency level, produce a structured report, and flag cases where false confidence might occur. Replace every square-bracket placeholder with actual values before execution. The template enforces a strict output schema, requires explicit reasoning for each classification, and includes a mandatory gap analysis section to prevent silent failures when evidence is missing.

textCopy
You are an evidence sufficiency assessment engine. Your task is to evaluate a set of claims against provided evidence sources and produce a structured report classifying each claim by evidence sufficiency level.

## INPUT
Claims to assess:
[CLAIMS]

Available evidence sources:
[EVIDENCE_SOURCES]

## CLASSIFICATION THRESHOLDS
Use the following calibrated thresholds to classify each claim:
- **SUFFICIENT**: At least [MIN_CORROBORATION_COUNT] independent sources directly support the claim, no sources contradict it, and all supporting sources meet the minimum authority score of [MIN_AUTHORITY_SCORE].
- **PARTIAL**: At least one source supports the claim but the corroboration count is below [MIN_CORROBORATION_COUNT], or supporting sources have authority scores below [MIN_AUTHORITY_SCORE], or minor contradictions exist from sources with authority below [CONTRADICTION_TOLERANCE_THRESHOLD].
- **INSUFFICIENT**: No source directly supports the claim, or all supporting sources fall below the minimum authority score, or the evidence is tangential and does not address the claim's core assertion.
- **CONTRADICTED**: At least one source with authority score at or above [CONTRADICTION_TOLERANCE_THRESHOLD] directly contradicts the claim, and the contradicting source has equal or higher authority than any supporting source.

## OUTPUT SCHEMA
Return a valid JSON object with this exact structure:
{
  "report_id": "string",
  "generated_at": "ISO-8601 timestamp",
  "methodology": {
    "thresholds_applied": {
      "min_corroboration_count": number,
      "min_authority_score": number,
      "contradiction_tolerance_threshold": number
    },
    "evidence_scope": "string describing what evidence was available",
    "limitations": ["string array of known limitations"]
  },
  "claims_assessment": [
    {
      "claim_id": "string matching input claim identifier",
      "claim_text": "string",
      "classification": "SUFFICIENT | PARTIAL | INSUFFICIENT | CONTRADICTED",
      "confidence_score": number between 0.0 and 1.0,
      "supporting_sources": ["array of source identifiers that support"],
      "contradicting_sources": ["array of source identifiers that contradict"],
      "reasoning": "string explaining the classification with explicit reference to thresholds",
      "false_confidence_risk": "LOW | MEDIUM | HIGH",
      "false_confidence_note": "string explaining why false confidence might occur, or null if LOW"
    }
  ],
  "gap_analysis": {
    "claims_with_insufficient_evidence": ["array of claim_ids"],
    "missing_evidence_types": ["string array describing what kind of evidence would be needed"],
    "search_coverage_assessment": "string describing whether the available evidence set is representative or likely incomplete",
    "recommended_remediation": ["string array of actions to close evidence gaps"]
  },
  "aggregate_statistics": {
    "total_claims": number,
    "sufficient_count": number,
    "partial_count": number,
    "insufficient_count": number,
    "contradicted_count": number,
    "high_false_confidence_risk_count": number
  }
}

## CONSTRAINTS
[CONSTRAINTS]

## EVALUATION CRITERIA
Before returning the output, verify:
1. Every claim in the input appears in the claims_assessment array.
2. Every classification includes explicit reasoning that references the thresholds.
3. The gap_analysis section is never empty, even if all claims are SUFFICIENT.
4. False-confidence risk is HIGH when a claim is classified as SUFFICIENT but relies on a single source or sources with borderline authority scores.
5. No source is cited as supporting without a direct, quotable connection to the claim.
6. Confidence scores reflect evidence quality, not just classification certainty.

## RISK LEVEL
[RISK_LEVEL]

To adapt this template, start by defining your classification thresholds in your application layer and injecting them into the prompt. The [MIN_CORROBORATION_COUNT], [MIN_AUTHORITY_SCORE], and [CONTRADICTION_TOLERANCE_THRESHOLD] placeholders should be replaced with numeric values calibrated to your domain's evidence standards. For high-risk domains like healthcare or legal review, set [RISK_LEVEL] to HIGH and include additional constraints in [CONSTRAINTS] such as requiring human review for all PARTIAL and INSUFFICIENT classifications. The [CLAIMS] placeholder expects a pre-extracted, atomic claim set with unique identifiers; do not pass raw documents here. The [EVIDENCE_SOURCES] placeholder should contain structured source objects with identifiers, text excerpts, and pre-computed authority scores. Wire the output through a JSON schema validator before accepting it into your pipeline, and log any validation failures for retry or escalation.

The Evidence Sufficiency Assessment Report prompt is designed to sit downstream of claim extraction and evidence retrieval steps in a verification pipeline. It expects a structured input payload containing extracted claims, their associated evidence records, and source metadata. The prompt is not a standalone chat interaction—it should be called programmatically with a validated input schema, and its output should be parsed into a typed object before any downstream consumption. The primary integration points are the claim-evidence ingestion layer, the model inference call, the output validator, and the triage router that decides whether a claim proceeds to auto-verdict, human review, or insufficient-evidence flagging.

Wire the prompt into your application by constructing a strict input object with the following fields: claims (array of claim objects with claim_id, claim_text, claim_type, and domain), evidence_bundle (array of evidence records with evidence_id, source_id, source_authority_score, content_snippet, relevance_to_claim, and retrieval_timestamp), thresholds (object with sufficient_corroboration_count, minimum_authority_score, and contradiction_tolerance), and pipeline_metadata (object with run_id, model_version, and previous_verdict if this is a re-assessment). Before calling the model, validate that every claim has at least one evidence record mapped to it—empty evidence bundles should be caught at the application layer and routed directly to an 'insufficient evidence' disposition without consuming model tokens. After receiving the model response, parse the JSON output and run a schema validator that checks for required fields (claim_id, evidence_sufficiency_level, sufficiency_rationale, evidence_gaps, false_confidence_risk_flag). Any response that fails schema validation should trigger a single retry with the validation error message appended to the prompt as a correction hint. If the retry also fails, log the failure and escalate to a human review queue with the raw model output attached.

For model selection, use a model with strong JSON-following behavior and low refusal rates on analytical tasks—GPT-4o, Claude 3.5 Sonnet, or equivalent. Set temperature to 0 or a very low value (0.1 maximum) to reduce variance in sufficiency classifications. Implement structured output mode if your provider supports it (e.g., OpenAI's response_format with a JSON schema, or Anthropic's tool-use with a defined output schema). Log every inference call with the full prompt, response, parse status, and validator results to your observability platform. This trace is essential for debugging false-confidence detections and threshold calibration. For high-stakes domains (legal, healthcare, financial compliance), always route claims with evidence_sufficiency_level: "insufficient" or false_confidence_risk_flag: true to human review before any downstream action. Never auto-accept a 'sufficient' classification without spot-checking a sample of outputs against the evidence bundle—build a periodic human audit step into your pipeline to detect threshold drift and model overconfidence.

The most common production failure mode is threshold miscalibration: if sufficient_corroboration_count is set too low, the model will classify weakly-supported claims as sufficient; if set too high, legitimate claims will flood the human review queue. Start with conservative thresholds, run a calibration batch against a labeled dataset of known-sufficient and known-insufficient claims, and adjust thresholds based on false-positive and false-negative rates. A second failure mode is evidence-context truncation—if evidence snippets are too long, they will exceed the context window and get silently dropped. Implement a pre-flight context budget check that estimates token usage and truncates or summarizes evidence records before they reach the prompt. Finally, avoid using this prompt for real-time user-facing applications where latency expectations are under two seconds; the evidence assessment reasoning chain benefits from the model having time to compare multiple evidence records, and rushing it with short max_tokens limits degrades sufficiency classification accuracy.