Evidence Discrepancy Flagging Prompt for RAG Answers

This prompt operates as a post-generation guardrail in a RAG pipeline. After your system generates an answer from retrieved context, this prompt reviews the answer against the original evidence to flag statements where sources disagree. It produces a structured discrepancy report that identifies conflicting claims, cites the disagreeing sources, and distinguishes genuine source conflict from model synthesis errors. The primary job-to-be-done is preventing overconfident or misleading responses caused by contradictory source material before answers reach end users.

Use this prompt when your application surfaces answers to users and you need to add evidence-consistency verification to an existing retrieval and generation pipeline. The ideal user is a RAG application developer or AI engineer who already has a working retrieval step, a generation step, and access to the original retrieved passages. You need the full generated answer text and the complete set of retrieved source passages as inputs. This prompt does not generate answers, rank evidence, or resolve conflicts. It audits an existing answer against its source context and flags discrepancies for downstream handling—whether that means surfacing a warning to users, triggering a human review queue, or regenerating the answer with conflict-aware instructions.

Do not use this prompt as a replacement for retrieval quality checks, as a primary hallucination detector for unsupported claims, or as a conflict resolution engine. It is specifically scoped to detecting disagreements between sources that are cited or reflected in the generated answer. If your pipeline needs to verify that every claim is supported by at least one source, pair this with a grounding verification prompt. If you need to resolve conflicts automatically, route flagged discrepancies to a separate resolution prompt or human review step. Avoid running this prompt on answers where the retrieved context is already known to be homogeneous or single-source, as the discrepancy check adds latency without value in those cases.

The following prompt template is designed to be dropped directly into your RAG evaluation or guardrail pipeline. It instructs the model to act as an evidence auditor, comparing a generated answer against the specific source passages that were provided as context. The core instruction forces the model to isolate each factual claim in the answer and trace it back to the source material, flagging statements that lack support or directly contradict the evidence. This template is structured to produce a machine-readable JSON report, making it suitable for automated checks before an answer is shown to a user.

codeCopy
You are an evidence auditor. Your task is to review a generated answer against the provided source passages and flag any discrepancies.

## INPUT
**Generated Answer:**
[ANSWER]

**Source Passages:**
[SOURCES]

## INSTRUCTIONS
1. Decompose the generated answer into discrete, verifiable factual claims.
2. For each claim, search the source passages for direct supporting or contradicting evidence.
3. Classify each claim as one of:
   - **SUPPORTED**: The claim is directly stated in a source passage.
   - **UNSUPPORTED**: The claim is not found in any source passage.
   - **CONTRADICTED**: A source passage explicitly states the opposite of the claim.
   - **SYNTHESIS_ERROR**: The claim combines information from multiple sources in a way that creates a false or misleading statement not present in any single source.
4. For UNSUPPORTED, CONTRADICTED, or SYNTHESIS_ERROR claims, provide a brief explanation and cite the relevant source passage(s) by their ID.
5. If no discrepancies are found, explicitly state that the answer is fully grounded.

## OUTPUT SCHEMA
Return a single JSON object with the following structure:
{
  "overall_grounding_status": "FULLY_GROUNDED" | "DISCREPANCY_FOUND",
  "claims": [
    {
      "claim_text": "string",
      "status": "SUPPORTED" | "UNSUPPORTED" | "CONTRADICTED" | "SYNTHESIS_ERROR",
      "source_ids": ["string"],
      "explanation": "string"
    }
  ]
}

## CONSTRAINTS
- Only use the provided source passages as evidence. Do not use external knowledge.
- Do not flag stylistic differences or minor paraphrasing as discrepancies.
- A SYNTHESIS_ERROR must involve a logical or factual error created by combining sources, not just a lack of direct support.

To adapt this template, replace the [ANSWER] and [SOURCES] placeholders with your application data. The [SOURCES] input should be a structured list of objects, each containing an id and text field, to enable precise citation in the output. For high-stakes domains like healthcare or finance, add a [RISK_LEVEL] constraint that forces the model to flag any UNSUPPORTED claim as a blocking error. You can also extend the [OUTPUT_SCHEMA] to include a confidence score for each classification, which is useful for routing low-confidence flags to a human review queue. The SYNTHESIS_ERROR category is the most critical for distinguishing a flawed model from conflicting sources; test this heavily with examples where the model incorrectly merges two true statements into a false conclusion.

This prompt operates as a post-generation guardrail, not a user-facing step. After your RAG system produces an answer from retrieved context, pipe both the generated answer and the full set of retrieved passages into this prompt to produce a structured discrepancy report. The report flags statements where sources disagree, distinguishes genuine source conflict from model synthesis errors, and provides source references for each flagged issue. This is a high-stakes verification step: a false negative (missed discrepancy) produces an overconfident, misleading answer, while a false positive (flagged agreement) creates unnecessary review burden and erodes trust in the system.

Wire the prompt into an asynchronous validation queue rather than blocking the user response. The primary RAG answer can be returned to the user immediately with a 'verification in progress' indicator, while the discrepancy report is generated in parallel. If the report flags high-severity conflicts, trigger a correction workflow: either append a conflict notice to the answer, regenerate with conflict-aware synthesis, or escalate to a human review queue. Implement a severity threshold in your harness—for example, only interrupt the user experience for conflicts rated 'high' or 'critical' on a defined scale. Log every discrepancy report with the answer version, retrieval set hash, and model version for audit trails.

Validation must happen at two levels. First, validate the output schema: ensure the JSON contains the required fields (flagged_statements, conflict_type, source_references, severity), that source references map to actual passage indices in the input, and that no hallucinated passage IDs appear. Second, run semantic evals: sample flagged statements and check whether human reviewers agree that a genuine discrepancy exists. Track false positive rate (flagging agreement as conflict) and false negative rate (missing known contradictions in a golden test set). For high-risk domains like healthcare or legal, require human approval on all flagged discrepancies before any correction is surfaced to users. Set a maximum retry count of 2 for malformed outputs; if the model fails to produce valid JSON after two repair attempts, log the failure and escalate to a human operator rather than silently dropping the check.