Factual Claim Extraction and Hallucination Flagging Prompt

This prompt is designed for factuality teams and RAG system builders who need to decompose a model's generated text into a set of discrete, auditable factual claims. The core job-to-be-done is post-generation verification: you have a block of output from an LLM, and before it reaches a user, a database, or a downstream system, you must know exactly which statements are supported by your source material and which are unsupported. The ideal user is an engineer or ML operator building a production pipeline where hallucination detection is a hard requirement, not a nice-to-have. You should use this prompt when you have a clear source context (retrieved chunks, a source document, or a verified knowledge base) and you need a structured, machine-readable report that pairs each extracted claim with its grounding status.

Do not use this prompt when you lack a defined source of truth. If you cannot provide a [SOURCE_CONTEXT] that contains the evidence against which claims will be checked, the prompt will either flag everything as ungrounded or hallucinate its own grounding assessments. This prompt is also not a substitute for a human fact-checker in high-stakes domains like healthcare, legal, or finance; its output is a structured artifact for review, not a final verdict. It is a poor fit for creative or subjective text where 'factuality' is ill-defined, such as marketing copy, brainstorming sessions, or narrative fiction. Finally, avoid this prompt if your primary goal is to fix the output; this prompt extracts and flags claims. Use a separate repair prompt from the Output Repair and Validation pillar to strip or correct the flagged content.

The output of this prompt is a JSON array of claim objects, each containing the claim text, a grounding status, and the matched source excerpt. This structure is designed to be consumed by an automated evaluation harness. You should wire this prompt into a pipeline that logs the grounding report, routes ungrounded claims for human review if they exceed a risk threshold, and blocks fully ungrounded outputs from being published. The next step after reading this section is to copy the prompt template, adapt the [SOURCE_CONTEXT] and [RISK_LEVEL] placeholders to your domain, and integrate it with a validation step that checks for false negatives—claims the model failed to extract at all.

This prompt template is designed to be dropped into a post-generation validation step in your RAG pipeline or content generation system. It takes the model's raw output and the source context that was provided to the model, extracts every discrete factual assertion, and pairs each with a grounding status. The template uses square-bracket placeholders that you replace with your actual inputs, schemas, and constraints before sending it to the model. Copy the block below and adapt the placeholders to match your application's data model and risk tolerance.

textCopy
You are a factuality auditor. Your task is to extract every discrete factual claim from the provided model output and determine whether each claim is supported by the provided source context.

## INPUT

### MODEL OUTPUT
[OUTPUT_TEXT]

### SOURCE CONTEXT
[SOURCE_CONTEXT]

## INSTRUCTIONS

1. Extract every discrete factual assertion from the MODEL OUTPUT. A factual assertion is any statement that could be true or false, including:
   - Claims about events, dates, people, organizations, or locations
   - Quantitative claims, statistics, or measurements
   - Statements about relationships, causes, or effects
   - Attributions to specific sources or authorities

2. For each extracted claim, determine its grounding status by comparing it against the SOURCE CONTEXT:
   - **GROUNDED**: The claim is directly supported by explicit evidence in the source context.
   - **PARTIALLY_GROUNDED**: Some but not all elements of the claim are supported by the source context.
   - **UNGROUNDED**: The claim cannot be verified from the source context. This includes claims that may be true but lack supporting evidence.
   - **CONTRADICTED**: The claim directly conflicts with evidence in the source context.

3. For GROUNDED and PARTIALLY_GROUNDED claims, include the exact source text that supports the claim.

4. For UNGROUNDED claims, indicate whether the claim appears plausible but unverifiable or likely fabricated.

5. Do not evaluate the truth of claims against your own knowledge. Only compare against the provided SOURCE CONTEXT.

## OUTPUT SCHEMA

Return a JSON object with this exact structure:

{
  "claims": [
    {
      "claim_id": "string (unique identifier, e.g., C1, C2)",
      "claim_text": "string (the exact factual assertion from the output)",
      "grounding_status": "GROUNDED | PARTIALLY_GROUNDED | UNGROUNDED | CONTRADICTED",
      "source_evidence": "string or null (exact quote from source context that supports the claim)",
      "fabrication_likelihood": "string or null (for UNGROUNDED claims: LIKELY_FABRICATED | PLAUSIBLE_UNVERIFIABLE | UNKNOWN)",
      "notes": "string or null (brief explanation of the grounding determination)"
    }
  ],
  "summary": {
    "total_claims": "number",
    "grounded_count": "number",
    "partially_grounded_count": "number",
    "ungrounded_count": "number",
    "contradicted_count": "number",
    "overall_reliability": "HIGH | MEDIUM | LOW | CRITICAL"
  }
}

## CONSTRAINTS

- Extract ALL factual claims. Do not skip claims that seem minor or obvious.
- Do not treat opinions, recommendations, or speculative language as factual claims unless they assert something verifiable.
- If the MODEL OUTPUT contains no factual claims, return an empty claims array with appropriate summary counts.
- If the SOURCE CONTEXT is empty or missing, mark all claims as UNGROUNDED with fabrication_likelihood set to UNKNOWN.
- Preserve the exact wording of claims from the MODEL OUTPUT. Do not paraphrase.
- [ADDITIONAL_CONSTRAINTS]

## RISK LEVEL
[RISK_LEVEL]

Adapting the template: Replace [OUTPUT_TEXT] with the model-generated content you need to audit. Replace [SOURCE_CONTEXT] with the retrieved documents, source text, or context chunks that were provided to the generating model. Use [ADDITIONAL_CONSTRAINTS] to add domain-specific rules—for example, in healthcare you might add 'Flag any claim involving a diagnosis, medication, or procedure for mandatory human review regardless of grounding status.' Set [RISK_LEVEL] to HIGH for regulated domains where false negatives are unacceptable, which should trigger stricter grounding requirements and mandatory human review of UNGROUNDED claims. For lower-risk applications, you can relax the fabrication_likelihood field or simplify the output schema. Before deploying, run this prompt against a golden dataset of known grounded and hallucinated outputs to calibrate your grounding thresholds and measure false negative rates.

This prompt is designed to operate as a post-generation validation step in a RAG pipeline or content generation workflow. It should be called after the primary model produces its output, receiving both the generated text and the source context that was provided to the generation step. The harness must supply both [INPUT_TEXT] (the model's output to audit) and [SOURCE_CONTEXT] (the retrieved chunks, documents, or evidence the model was instructed to use). Without both inputs, the extraction prompt cannot perform grounding verification and will produce unreliable results.

Wire the prompt into an asynchronous validation queue rather than blocking the primary response path. After the generation step completes, publish the output and source context to a validation worker that calls this prompt. Configure the worker with a strict JSON schema validator that enforces the expected output structure: an array of claim objects, each containing claim_text, source_reference, grounding_status (one of grounded, partially_grounded, ungrounded), and evidence_excerpt. Reject and retry any response that fails schema validation. Use a maximum of two retry attempts with exponential backoff before escalating to a human review queue. Log every validation run with the prompt version, model used, input lengths, claim count, and grounding status distribution for observability.

For high-risk domains such as healthcare, legal, or financial applications, add a mandatory human review gate for any output where the proportion of ungrounded claims exceeds 5% or where any single claim is flagged as ungrounded with high confidence. Route these cases to a review interface that displays the original output, the extracted claims, and the source context side by side. Do not automatically strip or modify the output based on this prompt's results in regulated workflows—always require human confirmation before the content reaches users or databases. For lower-risk applications, you may configure an automated policy to remove ungrounded claims and regenerate the affected sections, but always log the original output and the removal decisions for auditability.