RAG Hallucination Recovery Prompt Template

This prompt is designed for product teams running Retrieval-Augmented Generation (RAG) systems where a validator has flagged an initial answer for containing unsupported claims. It is not a first-pass answer generator. Use it inside a retry loop after a hallucination detector or citation verifier has identified statements that cannot be grounded in the provided context. The prompt instructs the model to cross-check each claim against the source evidence, regenerate a corrected answer with proper citations, and abstain when evidence is insufficient. It assumes you already have the original query, the flagged answer, and the retrieved context available.

You should not use this prompt when the retrieved context itself is empty, irrelevant, or poisoned. If the retriever failed to surface any useful documents, this prompt will correctly abstain, but the root cause is a retrieval failure, not a hallucination problem. Similarly, do not use this prompt for first-pass answer generation—it is explicitly designed for correction, not creation. The ideal integration point is immediately after a FactualConsistencyValidator or CitationVerifier returns a status: HALLUCINATION_DETECTED signal. Wire the validator's output directly into the [FLAGGED_ANSWER] and [UNSUPPORTED_CLAIMS] placeholders to give the recovery prompt precise targets rather than asking it to re-audit the entire response from scratch.

Before deploying this prompt, ensure your evaluation harness includes tests for both successful correction and appropriate abstention. Common failure modes include the model over-correcting by removing accurate statements, introducing new hallucinations while fixing old ones, or generating citations that point to the wrong passage. Set a retry budget of 2-3 attempts maximum, and if the recovery prompt fails to produce a fully grounded answer within that budget, escalate to a human review queue rather than looping indefinitely. For high-stakes domains like healthcare or legal, always require human approval on the final corrected output, even if automated validation passes.

This prompt template is designed to be called by your application's retry handler after a validator has detected unsupported claims in a RAG-generated answer. It takes the original query, the retrieved context, the flagged hallucinated answer, and the specific validator error details, then instructs the model to regenerate a grounded response. The template uses square-bracket placeholders for all dynamic inputs, making it safe to paste directly into your retry logic and populate from your validator's output payload.

textCopy
You are a fact-correction assistant operating inside a RAG system. Your task is to regenerate an answer after a validator detected unsupported claims in the previous response.

## INPUTS
- USER QUERY: [USER_QUERY]
- RETRIEVED CONTEXT (numbered passages): [RETRIEVED_CONTEXT]
- PREVIOUS ANSWER (flagged for hallucination): [PREVIOUS_ANSWER]
- VALIDATOR ERRORS: [VALIDATOR_ERRORS]

## CONSTRAINTS
1. Every factual claim in your answer MUST be directly supported by at least one passage in RETRIEVED CONTEXT. Cite the passage number in brackets after each claim, like this: [P3].
2. If the RETRIEVED CONTEXT does not contain sufficient evidence to answer the USER QUERY, respond with: "INSUFFICIENT_EVIDENCE: [explain what is missing]" and do not fabricate any answer.
3. Do not repeat or reference the PREVIOUS ANSWER. Generate a completely new response from the RETRIEVED CONTEXT only.
4. Address each VALIDATOR ERROR explicitly. If an error points to a claim that cannot be supported, omit that claim entirely.
5. If multiple passages conflict, note the conflict and explain which passage you relied on and why, based on recency, authority, or specificity indicated in the context.
6. Do not use external knowledge, training data, or assumptions beyond what is explicitly stated in RETRIEVED CONTEXT.

## OUTPUT FORMAT
Return a JSON object with the following schema:
{
  "status": "success" | "insufficient_evidence" | "partial",
  "answer": "The grounded answer text with inline citations [P1] [P2]...",
  "corrections_applied": ["List of specific corrections made for each validator error"],
  "unsupported_claims_removed": ["List of claims from the previous answer that were removed due to lack of evidence"],
  "conflicts_noted": ["Any source conflicts identified and how they were resolved"]
}

## RETRY CONTEXT
This is retry attempt [RETRY_COUNT] of [MAX_RETRIES]. If you cannot produce a fully grounded answer, set status to "insufficient_evidence" or "partial" rather than fabricating.

Adaptation guidance: Populate [RETRIEVED_CONTEXT] with passages formatted as [P1] text..., [P2] text... so the model can reference them by number. [VALIDATOR_ERRORS] should contain structured error objects from your hallucination detector, including the specific unsupported claim and the passage(s) it failed to match. Set [MAX_RETRIES] to your retry budget ceiling (typically 2-3 for hallucination recovery). If [RETRY_COUNT] equals [MAX_RETRIES], consider adding an escalation instruction that routes to human review instead of another retry. For high-stakes domains, always require human review when status is "insufficient_evidence" or "partial" before surfacing the answer to end users.

This prompt template is designed to be called programmatically inside a post-generation validation loop, not as a one-off chat interaction. The typical integration pattern is: (1) the primary RAG pipeline generates an answer with citations, (2) a lightweight validator or LLM-as-judge check flags unsupported claims, and (3) this recovery prompt is invoked with the original context, the flagged output, and the specific validator error. The recovery prompt should run in a retry harness that limits attempts—usually 1–3 correction passes—before escalating to human review or logging the failure for offline analysis. Do not call this prompt unconditionally on every answer; it is a targeted repair tool for outputs that have already failed a factual-consistency check.

Wiring the prompt into your application requires several components. First, maintain a structured [VALIDATION_ERRORS] object that includes the specific claim(s) that failed, the validator's confidence score, and the source passage(s) that contradict the claim. This error payload becomes the primary input to the recovery prompt. Second, implement a retry budget with a counter that increments on each correction attempt. After the recovery prompt returns a regenerated answer, run the same validator again. If the regenerated answer passes, return it to the user. If it fails, increment the counter and retry up to your threshold (typically 2–3 attempts). On budget exhaustion, call your [ESCALATION_HANDLER]—this could be a human review queue, a fallback model, or a structured error response to the user that acknowledges the system cannot answer confidently. Third, log every recovery attempt with the original output, the validator errors, the regenerated output, and the final disposition (passed, failed, escalated). This log is essential for debugging systematic hallucination patterns and tuning your validator thresholds.

Model choice matters. Recovery prompts benefit from models with strong instruction-following and context-adherence capabilities. If your primary generation model is cost-optimized or smaller, consider routing recovery attempts to a more capable model (e.g., from a distilled model to a frontier model) as a first escalation step before human review. Tool integration is optional but powerful: if your RAG system supports re-retrieval, the recovery prompt can include a [RE_RETRIEVE] flag that triggers fresh context fetching before regeneration. This is especially useful when the validator indicates that the original retrieved context was insufficient. Validation gating is critical—never ship a regenerated answer without re-running your factual-consistency check. The recovery prompt includes an [ABSTENTION_INSTRUCTION] placeholder that should be populated with your product's specific abstention language (e.g., 'I couldn't find enough information to answer that question reliably').

What to avoid: Do not use this prompt as a substitute for improving your primary RAG prompt or retrieval quality. If more than 10–15% of answers trigger recovery, invest in upstream fixes—better retrieval, stricter primary-prompt constraints, or context-quality filtering. Do not allow unbounded retry loops; they waste compute and latency budget without improving outcomes. Do not skip logging; without recovery-attempt traces, you cannot distinguish between a prompt problem, a retrieval problem, or a validator calibration issue. Finally, if your use case involves regulated domains (healthcare, legal, finance), the [ESCALATION_HANDLER] must route to a qualified human reviewer, and the recovery log must be preserved as part of your audit trail.