Verification Outcome Summary Prompt for Reviewers

This prompt is the final stage of a verification pipeline, designed for a human reviewer who needs to make fast, accurate decisions on a batch of pre-checked claims. It assumes that claim extraction, evidence matching, and initial verdict assignment have already been completed by upstream systems. The prompt's job is to transform a large volume of structured verification results into a prioritized summary that triages claims by risk, highlights contradictions, and flags unsupported statements, thereby accelerating human review and reducing cognitive load.

The ideal user is a verification operations lead, a compliance analyst, or a domain expert tasked with signing off on automated fact-checking results. The required context is a structured input payload containing per-claim verification records, each with a claim ID, the original claim text, the assigned verdict, a confidence score, and a list of evidence sources with their relevance. Do not use this prompt for initial fact-checking, evidence retrieval, or generating the raw verification data; it is strictly a summarization and triage engine for completed work. Using it on unverified claims will produce a misleading summary that hides the absence of evidence.

Before integrating this prompt, ensure your upstream pipeline produces a consistent schema. The prompt's value collapses if input fields are missing or if confidence scores are uncalibrated. A common failure mode is feeding it claims with a 'verified' status that were actually checked against an empty evidence set. To avoid this, implement a pre-validation step that rejects any claim where the evidence array is empty but the verdict is not 'unsupported.' The next step is to wire this prompt into a review queue where a human can approve, reject, or reassign each triaged claim based on the summary's risk prioritization.

This prompt template is designed to be pasted directly into your application's prompt layer. It takes a batch of structured verification results—each containing a claim, its verdict, confidence score, and supporting evidence—and produces a prioritized summary for a human reviewer. The summary highlights high-risk claims, unsupported statements, and contradictions, enabling the reviewer to focus on the most critical items first rather than re-reading every individual result.

textCopy
You are a verification triage assistant. Your task is to produce a concise, prioritized summary of a batch of verification results for a human reviewer.

## INPUT
[BATCHED_VERIFICATION_RESULTS]

## OUTPUT_SCHEMA
Return a JSON object with the following structure:
{
  "summary": {
    "total_claims": <number>,
    "high_risk_count": <number>,
    "unsupported_count": <number>,
    "contradiction_count": <number>,
    "overall_assessment": "<string: one-sentence summary of batch quality>"
  },
  "high_priority_items": [
    {
      "claim_id": "<string>",
      "claim_text": "<string>",
      "verdict": "<string: e.g., True, False, Unsupported, Contradicted>",
      "risk_level": "<string: High, Medium, Low>",
      "reason_for_priority": "<string: brief explanation of why this needs review>",
      "recommended_action": "<string: e.g., Escalate, Verify manually, Approve, Reject>"
    }
  ],
  "unsupported_claims_summary": [
    {
      "claim_id": "<string>",
      "claim_text": "<string>",
      "evidence_gap": "<string: what type of evidence is missing>"
    }
  ],
  "contradictions_summary": [
    {
      "claim_id": "<string>",
      "claim_text": "<string>",
      "conflicting_sources": ["<string>"],
      "conflict_description": "<string>"
    }
  ]
}

## CONSTRAINTS
- Only include claims in `high_priority_items` if their verdict is False, Unsupported, or Contradicted, or if their confidence score is below [CONFIDENCE_THRESHOLD].
- Sort `high_priority_items` by risk_level (High first), then by lowest confidence.
- Limit `high_priority_items` to a maximum of [MAX_PRIORITY_ITEMS] entries.
- If no high-priority items exist, return an empty array and state "No high-priority items requiring review" in the overall_assessment.
- Do not fabricate or alter any claim text, verdict, or source information from the input.
- If the input contains contradictory verdicts for the same claim, flag it in `contradictions_summary`.

## RISK_LEVEL
[RISK_LEVEL]

## EXAMPLES
[EXAMPLES]

To adapt this template, replace the square-bracket placeholders with your application's data and configuration. [BATCHED_VERIFICATION_RESULTS] should be a JSON array of your structured verification objects. [CONFIDENCE_THRESHOLD] is a float (e.g., 0.7) that defines the cutoff for flagging low-confidence claims. [MAX_PRIORITY_ITEMS] prevents reviewer overload by capping the list. [RISK_LEVEL] can be set to a static string like "High" or injected dynamically based on the domain. [EXAMPLES] should contain one or two few-shot demonstrations of correctly formatted output for your specific claim types. Before deploying, validate the output against the JSON schema and run eval checks for triage accuracy—specifically, that no high-risk claim is omitted from the priority list and that unsupported claims are not misclassified as false.

The Verification Outcome Summary prompt is designed to sit at the end of a batch verification pipeline, consuming structured per-claim verification results and producing a triage summary for human reviewers. In a production harness, this prompt should never receive raw, unvalidated claim data. Instead, the application layer must first assemble a payload containing the batch of verified claims, each with its verdict, confidence score, evidence chain, and any contradiction or unsupported flags. The prompt template expects these fields in a structured format, typically a JSON array of claim objects, which the harness injects into the [VERIFICATION_RESULTS] placeholder. The harness should also inject the [REVIEWER_CONTEXT] placeholder with metadata such as the batch ID, verification pipeline version, source document identifiers, and any domain-specific risk thresholds that the reviewer needs to know before triaging.

Before calling the model, the harness must validate the input payload against a strict schema. Each claim object must contain required fields: claim_id, claim_text, verdict (one of SUPPORTED, CONTRADICTED, UNSUPPORTED, INCONCLUSIVE), confidence (a float between 0.0 and 1.0), and evidence_summary. Missing or malformed fields should trigger a repair step before the summary prompt is invoked. After the model returns the summary, the harness must validate the output structure: the summary must include a high_risk_claims array, a contradiction_pairs array, an unsupported_claims array, and a triage_recommendation string. If the output fails schema validation, the harness should retry with a repair prompt that includes the validation error message and the original input. Set a maximum of two retries before escalating to a human operator with the raw verification results and the failed summary attempts logged for diagnosis.

For high-risk domains such as healthcare, legal, or financial compliance, the harness must enforce a human-review gate after the summary is generated. The summary should be presented in a review interface that displays each high-risk claim alongside its evidence chain and the model's triage recommendation. The reviewer must explicitly confirm or override each triage decision before the summary is finalized. Log every reviewer action, including timestamps, reviewer identity, and any overrides, to produce an audit trail. Avoid using the summary prompt as a final decision-maker; its role is to prioritize and structure human attention, not to replace it. When wiring this into a batch pipeline, consider rate limits and token budgets: large batches of claims may exceed context windows, so the harness should chunk the input into batches of 20–30 claims and merge the resulting summaries into a consolidated report for the reviewer.