Low-Confidence Claim Escalation Prompt Template

This prompt is designed for the precise moment in an automated fact-checking pipeline where a claim cannot be auto-verified with sufficient confidence. It is not a general-purpose summarizer or a verdict generator. Its sole job is to take a claim that has already been extracted, matched against evidence, and scored below your operational triage threshold, and transform that machine context into a structured, actionable briefing for a human reviewer. The ideal user is a verification pipeline engineer or an operations lead who has already built the upstream components—claim extraction, evidence retrieval, and confidence scoring—and now needs a reliable, programmatic way to package the 'hard cases' for manual review without losing critical context.

Use this prompt when your system has executed the full automated verification attempt and the resulting confidence score sits below your defined triage threshold. The input must include the original claim, the scored evidence set, the confidence score that triggered the escalation, and the threshold it failed to meet. The output is an escalation packet containing identified evidence gaps, a summary of resolution steps already attempted by the system, and a clear set of specific questions the human reviewer must answer. This packet becomes the input to your review queue, ticketing system, or human-in-the-loop interface. Do not use this prompt for claims that scored above the threshold—those should follow your auto-verification reporting path. Do not use it to make the routing decision itself; that binary threshold comparison belongs in your application code, not in the language model's reasoning.

A common failure mode is treating this prompt as a final verification report. It is not. The escalation packet should explicitly state that the claim remains unverified and that the evidence is insufficient for an automated determination. Another failure mode is providing the model with the threshold but then asking it to decide whether to escalate, which introduces unnecessary variance. Instead, your application code should make the deterministic routing decision and only invoke this prompt when escalation is already confirmed. Before deploying, validate the output against a golden set of known low-confidence claims to ensure the packet structure is parseable, the evidence gaps are correctly identified, and the reviewer questions are specific rather than generic. If the claim domain is regulated or high-risk, the escalation packet must also flag the need for domain-expert review and preserve a complete audit trail of the automated attempt.

This prompt template is designed to be the final step in a verification pipeline, activated when a claim's confidence score drops below your operational threshold. Its job is to produce a structured escalation record that a human reviewer or downstream system can act on immediately. The prompt does not re-score the claim; it assumes the score is already calculated and provided as input. It focuses on packaging the evidence gaps, summarizing what automated resolution was attempted, and recommending a specific next action.

textCopy
You are an escalation specialist in a fact-checking pipeline. A claim has been flagged for human review because its confidence score fell below the acceptable threshold. Your task is to produce a structured escalation packet.

INPUT DATA:
- Claim: [CLAIM_TEXT]
- Claim ID: [CLAIM_ID]
- Confidence Score: [CONFIDENCE_SCORE]
- Confidence Threshold: [CONFIDENCE_THRESHOLD]
- Scoring Rationale: [SCORING_RATIONALE]
- Evidence Inventory: [EVIDENCE_INVENTORY]
- Attempted Resolutions: [ATTEMPTED_RESOLUTIONS]

OUTPUT SCHEMA:
{
  "escalation_packet": {
    "claim_id": "string",
    "escalation_reason": "string (summary of why the score is below threshold)",
    "evidence_gaps": [
      {
        "gap_type": "string (e.g., missing_source, authority_insufficient, contradiction_unresolved)",
        "description": "string",
        "impact_on_confidence": "string"
      }
    ],
    "attempted_resolution_summary": "string (what automated steps were tried and their outcomes)",
    "recommended_next_action": "string (e.g., manual_review, source_retrieval, domain_expert_consultation)",
    "reviewer_questions": ["string (specific questions for the human reviewer to answer)"],
    "urgency": "string (low, medium, high, critical)"
  }
}

CONSTRAINTS:
- Do not invent new evidence or sources.
- Base all evidence gaps strictly on the provided Evidence Inventory.
- If the Attempted Resolutions list is empty, state that no automated resolution was attempted.
- The urgency must be derived from the claim's potential impact, not the confidence score alone.

To adapt this template, replace each square-bracket placeholder with live data from your scoring system. The [EVIDENCE_INVENTORY] should be a serialized list of evidence objects, each containing the source, extracted text, and authority score. The [ATTEMPTED_RESOLUTIONS] should be a log of actions your pipeline already took, such as 're-queried with expanded search terms' or 'applied contradiction resolution prompt.' Before deploying, run this prompt against a golden set of 20 known low-confidence claims and have a domain expert judge whether the evidence_gaps and reviewer_questions are actionable and complete. If the output fails validation against the defined JSON schema, implement a retry with the validation error included in the next prompt's context.

The Low-Confidence Claim Escalation prompt is not a standalone tool; it is a decision node inside a verification pipeline. It fires when a claim's confidence score falls below a configured threshold and the system must produce a structured escalation packet for human review. The implementation harness must enforce that this prompt only executes after upstream scoring is complete, that its output conforms to a strict schema, and that the resulting escalation record is routed to the correct review queue with full traceability. Without this harness, escalation packets become inconsistent, reviewers receive incomplete context, and audit trails break.

Wire the prompt into your pipeline as a conditional step: after the claim confidence scorer returns a score below [ESCALATION_THRESHOLD], assemble the prompt with the original claim text, the evidence set used during scoring, the confidence score and its breakdown, and any contradiction flags. The model call should be wrapped in a retry loop with schema validation. Use a JSON schema validator to check that the output contains required fields—claim_id, escalation_reason, evidence_gaps, attempted_resolutions, recommended_actions, and reviewer_questions—before the packet enters the review queue. If validation fails, retry up to [MAX_RETRIES] times with the validation error message appended to the prompt as a correction hint. Log every attempt, including the raw model output, validation errors, and final accepted payload, to an observability store keyed by claim_id and trace_id. For high-stakes domains such as healthcare or finance, insert a mandatory human-approval gate: the escalation packet must be reviewed and acknowledged before any downstream action is taken based on the unresolved claim.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities—GPT-4o, Claude 3.5 Sonnet, or equivalent—because the prompt requires precise field population and must not hallucinate evidence gaps that were not present in the input. Avoid smaller or older models that tend to drop required fields or invent resolutions. If your pipeline already uses a cheaper model for high-confidence auto-verification, route escalation prompts to a separate, higher-capability model endpoint. Set a timeout of 15–30 seconds and a token budget appropriate to your evidence payload size. Monitor escalation rates as a pipeline health metric: a sudden spike in low-confidence claims may indicate upstream retrieval degradation, evidence source drift, or a scoring model regression, not a genuine increase in unverifiable content. Finally, build a feedback loop: when human reviewers resolve an escalated claim, capture their resolution and use it to evaluate whether the escalation packet contained the right context and questions. This closed-loop data is essential for tuning both the confidence threshold and the escalation prompt itself over time.