Verification Triage Decision Prompt Template

This prompt is designed for verification pipeline operators who need to make a structured, defensible decision about whether an extracted claim can be automatically verified by an AI system or must be routed to a human reviewer. It is not a fact-checking prompt itself; it is a triage prompt that sits between claim extraction and verification execution. Use this prompt when you have a claim, a set of available evidence, and a defined auto-verification capability, and you need to decide the next step in the pipeline. The prompt forces explicit reasoning about evidence sufficiency, claim complexity, risk, and confidence thresholds rather than defaulting to auto-verification for every claim. It is appropriate for operations teams running verification pipelines at scale where cost, latency, and accuracy trade-offs require documented escalation logic.

This prompt is the wrong tool if you need to actually verify a claim against evidence—that is the job of a downstream verification prompt. Do not use this prompt when you lack a defined auto-verification capability to route to, or when every claim must be reviewed by a human regardless of complexity. It is also inappropriate for real-time, user-facing applications where a triage decision would add unacceptable latency without a corresponding review workflow. The ideal user is a verification pipeline engineer or operations lead who controls the routing logic and can instrument the prompt's output to trigger automated workflows, queue assignments, and audit logging. You must provide the claim text, a structured evidence packet, your auto-verification capability description, and your organization's risk tolerance and cost constraints as inputs.

Before deploying this prompt, ensure your upstream claim extraction produces atomic, context-preserving claims and your evidence retrieval system provides source-attributed, query-relevant evidence. A common failure mode is feeding this prompt with vague claims or incomplete evidence, which forces it to conservatively route everything to human review and defeats the purpose of automated triage. After implementing this prompt, you must validate its decisions against human reviewer outcomes to calibrate its routing accuracy. Start by running it in shadow mode alongside existing manual triage, then gradually shift traffic once you've confirmed that low-evidence claims never route to auto-verify and that high-risk claims are consistently escalated.

This is the core prompt template for the verification triage decision. It accepts a single claim, available evidence, and operational constraints, then returns a structured routing decision with a confidence threshold justification and escalation reason. The template is designed to be copied directly into your pipeline code, with each square-bracket placeholder replaced by your application's data before the model call. The output schema is strict JSON so downstream routing logic can parse the decision without additional extraction steps.

textCopy
You are a verification triage classifier. Your job is to decide whether a claim can be auto-verified or requires human review.

INPUT:
- Claim: [CLAIM_TEXT]
- Available Evidence: [EVIDENCE_LIST]
- Source Authority Scores: [AUTHORITY_SCORES]
- Domain: [DOMAIN]
- Risk Level: [RISK_LEVEL]
- SLA Deadline (minutes): [SLA_DEADLINE]
- Auto-Verification Budget ($): [AUTO_BUDGET]
- Human Review Budget ($): [HUMAN_REVIEW_BUDGET]

CONSTRAINTS:
- [CONSTRAINTS]

OUTPUT_SCHEMA:
{
  "route": "auto_verify" | "human_review",
  "confidence_threshold_justification": "string explaining why the evidence sufficiency meets or fails the auto-verification threshold",
  "escalation_reason": "string or null if route is auto_verify",
  "risk_category": "low" | "medium" | "high" | "critical",
  "required_reviewer_skills": ["string"] or [],
  "estimated_review_minutes": number or null,
  "evidence_sufficiency_flags": {
    "evidence_count": number,
    "source_diversity": number,
    "recency_days": number or null,
    "authority_minimum_met": boolean,
    "contradiction_present": boolean
  }
}

RULES:
1. Route to "human_review" if any of the following are true:
   - Evidence count is below [MIN_EVIDENCE_COUNT]
   - Source authority minimum is not met
   - Contradiction is present in the evidence
   - Risk level is "high" or "critical"
   - Domain requires specialized expertise not available in auto-verification
2. Route to "auto_verify" only when all evidence sufficiency thresholds are met AND risk level is "low" or "medium".
3. The escalation_reason must be specific and actionable, not generic.
4. Never route a claim with risk_level "critical" to auto_verify, regardless of evidence quality.
5. If the SLA deadline is less than [MIN_REVIEW_MINUTES], flag the constraint in the justification.

Return ONLY the JSON object. No markdown, no commentary.

To adapt this template for your pipeline, replace each placeholder with data from your upstream systems. [CLAIM_TEXT] comes from your claim extraction step. [EVIDENCE_LIST] should be a serialized array of evidence passages with source identifiers. [AUTHORITY_SCORES] can be a mapping of source IDs to numeric scores from your source authority assessment prompt. [CONSTRAINTS] should include any additional business rules, such as 'claims about financial results require CPA review' or 'medical claims always route to human review.' The [MIN_EVIDENCE_COUNT] and [MIN_REVIEW_MINUTES] values should be tuned based on your operational data and evaluation results. After the model returns the JSON, validate the output against the schema before routing. If validation fails, retry once with the error message appended to the prompt, then escalate to a human operator if the retry also fails.

This prompt is designed as a decision node within a larger verification pipeline, not a standalone chat interaction. It should be called after claim extraction and evidence retrieval, receiving a structured payload of claims, evidence, and source metadata. The output is a machine-readable routing instruction that downstream orchestration logic acts on. Treat this prompt as a critical control point: a false-negative triage decision (routing a high-risk claim to auto-verification) can cause direct harm, while a false-positive (escalating a trivial claim) wastes reviewer time and budget.

The implementation harness must enforce a strict contract around the prompt. Before calling the model, validate that the [CLAIM_EVIDENCE_PACKET] contains all required fields: the claim text, a unique claim ID, a list of evidence objects with source identifiers and retrieval dates, and any pre-computed confidence scores. After receiving the model's response, parse the JSON output and validate it against a schema that requires routing_decision (enum: AUTO_VERIFY, HUMAN_REVIEW), confidence_threshold_justification (string), and escalation_reason (string, required if routing is HUMAN_REVIEW). If parsing or schema validation fails, implement a retry loop with a maximum of 2 attempts, appending the validation error to the prompt context. If retries are exhausted, the claim must default to HUMAN_REVIEW as a safe fallback. Log every decision, including the raw prompt, the model's response, the validation result, and the final routing outcome, to an append-only store for auditability.

For high-risk domains, insert a human-in-the-loop gating step even for AUTO_VERIFY decisions. A sampling strategy can be implemented here: route 100% of claims where the model's internal confidence score (if available) falls below a calibrated threshold to a review queue, and randomly sample 5-10% of high-confidence auto-verifications for human QA. This harness should also monitor for drift by tracking the rate of HUMAN_REVIEW decisions over time. A sudden drop could indicate a model change, a prompt regression, or a shift in incoming claim complexity. Wire these metrics into your observability stack alongside the eval checks for false-positive auto-verifications and missed high-risk escalations. The next step is to run this harness against a golden dataset of pre-labeled claims to calibrate the sampling rate and validate that the safe fallback path is never silently bypassed.