Unsupported Statement Identification Prompt

This prompt is designed for a single, high-stakes job: scanning a block of model-generated text and identifying every factual assertion that cannot be supported by a provided context document. The ideal user is a developer or AI product builder integrating a factuality gate into a RAG pipeline, an editorial workflow, or a compliance review system. The prompt expects two inputs: the generated text to audit and the source context that should contain all supporting evidence. It is not a general-purpose hallucination detector for open-domain chat, nor is it a replacement for human editorial judgment in high-risk domains like medicine or law.

Use this prompt when you have a closed-context problem—meaning the model's output should be entirely derivable from a known set of source documents. Common deployment points include post-generation validation in RAG applications, automated report generation where every claim must be traceable to an input data feed, and content moderation pipelines that need to flag invented statistics or fabricated quotes before publication. The prompt is most effective when the source context is complete and well-structured. If the context is fragmented, contradictory, or incomplete, the prompt will correctly flag many statements as unsupported, but you will need a human review step to distinguish between model hallucination and gaps in your retrieval system.

Do not use this prompt when the model is expected to synthesize novel insights, apply external world knowledge, or generate creative content. It is also unsuitable for real-time chat moderation where latency constraints prevent a full context audit. In regulated industries, always route flagged outputs to a human reviewer and log the prompt's verdict alongside the evidence for auditability. Before deploying, test the prompt against a golden dataset of known supported and unsupported statements to calibrate your confidence thresholds and catch systematic false negatives where the model fails to flag a clear fabrication.

This prompt template is the core of the unsupported statement identification workflow. It instructs the model to act as a strict fact-checker, comparing every assertion in a piece of generated text against a provided evidence context. The output is a structured report that flags unsupported claims, assigns confidence levels, and recommends removal or human review. The template is designed to be copied directly into your prompt management system, with placeholders that you replace with your application's specific inputs, schemas, and constraints.

textCopy
You are a strict fact-checking engine. Your task is to analyze the provided [CONTENT] and identify every factual assertion that lacks sufficient support in the [EVIDENCE_CONTEXT].

For each assertion in the [CONTENT], you must:
1. Determine if the assertion is a factual claim (not an opinion, question, or stylistic flourish).
2. Check if the claim is directly supported, indirectly supported, or unsupported by the [EVIDENCE_CONTEXT].
3. Assign a confidence level: HIGH (clearly supported), MEDIUM (partially supported or requires inference), LOW (no supporting evidence found).
4. For LOW confidence claims, recommend an action: REMOVE or HUMAN_REVIEW.

[CONSTRAINTS]
- Do not flag opinions, subjective statements, or rhetorical questions as unsupported.
- If the [EVIDENCE_CONTEXT] is empty or insufficient to verify any claims, flag all factual assertions as LOW confidence.
- Do not invent evidence. If a claim cannot be verified, it is unsupported.
- Preserve the original text of each flagged claim exactly as it appears in the [CONTENT].

[OUTPUT_SCHEMA]
Return a JSON object with the following structure:
{
  "analysis": {
    "total_claims_identified": <integer>,
    "supported_claims": <integer>,
    "unsupported_claims": <integer>,
    "claims": [
      {
        "claim_text": "<exact text from content>",
        "claim_type": "factual",
        "support_status": "supported|partially_supported|unsupported",
        "confidence": "HIGH|MEDIUM|LOW",
        "evidence_reference": "<quote from evidence or null>",
        "recommended_action": "KEEP|REMOVE|HUMAN_REVIEW",
        "rationale": "<brief explanation of the determination>"
      }
    ]
  }
}

[EVIDENCE_CONTEXT]
[INSERT_EVIDENCE_HERE]

[CONTENT]
[INSERT_CONTENT_TO_ANALYZE_HERE]

To adapt this template for your production system, replace the [EVIDENCE_CONTEXT] placeholder with the source documents, retrieved chunks, or ground-truth data that the model should use for verification. Replace [CONTENT] with the model-generated text you need to audit. If your application requires a different output format, modify the [OUTPUT_SCHEMA] block to match your downstream parser. For high-risk domains like healthcare or legal, add a [RISK_LEVEL] placeholder that adjusts the strictness of the confidence thresholds and always route LOW confidence claims to a human review queue before any automated removal.

This prompt is designed to operate as a post-generation filter in a factuality pipeline. The typical integration pattern is: (1) a primary model generates content from retrieved context, (2) this prompt receives both the generated content and the source context as inputs, and (3) the output is a structured audit report that downstream systems can act on. The prompt expects two required inputs: the model-generated text to audit and the source context that should support all claims. Without both inputs, the prompt cannot perform its core function of cross-referencing assertions against evidence.

Wire this prompt into your application as a synchronous validation step before content reaches users or databases. The implementation harness should include: input validation that rejects requests missing either the generated text or source context fields; structured output parsing that expects a JSON array of statement objects with statement, support_status, confidence, and recommendation fields; retry logic with a maximum of 2 retries if the output fails to parse as valid JSON or if required fields are missing; and logging that captures the prompt version, input hashes, output structure, and any parse failures for observability. For high-risk domains like healthcare, legal, or finance, add a human review gate that routes any statement flagged with support_status: "unsupported" and confidence: "high" to a review queue before the content is published or stored.

Model choice matters for this workflow. Use a model with strong instruction-following and JSON output capabilities, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that may struggle with the dual task of claim decomposition and evidence matching. If your pipeline processes long documents, chunk the generated text and source context into segments that fit within the model's context window, run the prompt on each segment, and merge the results with deduplication logic. Do not use this prompt as a real-time user-facing filter without caching—the latency of a full factuality audit is typically 2-5 seconds per segment, which is unsuitable for sub-second response requirements.

Testing this harness requires a golden dataset of known supported and unsupported claims. Create test cases where the generated text contains: (a) claims fully grounded in the source context, (b) claims partially grounded with missing details, (c) claims entirely fabricated, and (d) claims that are true but not present in the provided context (the hardest case). Measure precision (did the prompt flag only unsupported claims?) and recall (did it catch all unsupported claims?). A common failure mode is the prompt missing claims that are factually true but unsupported by the provided context—this is a false negative that can erode trust in the filter. Calibrate the confidence thresholds in your harness based on your tolerance for false positives versus false negatives.

For production deployment, implement a circuit breaker that stops processing if the prompt returns malformed output on more than 10% of requests in a rolling window. This prevents a prompt degradation event from silently passing unsupported content through. Additionally, store the full prompt input and output for every request where support_status: "unsupported" appears—this audit trail is essential for debugging false positives, refining the prompt, and demonstrating due diligence to compliance reviewers. Never silently discard unsupported statements; always log the removal decision with the statement text, support status, and confidence score for downstream observability.