Missing Citation Flagging Prompt

This prompt is a post-generation guardrail, not a content creator. Its job is to sit between your RAG answer generator and the end user, scanning the generated output for factual claims that lack source attribution. The ideal user is an AI engineer or QA pipeline builder who already has a working RAG system and needs automated quality assurance before answers are delivered. You need the generated answer text and the list of source documents or chunks that were provided to the generator as context. Without both, this prompt cannot distinguish between a properly cited claim and an unsupported one.

Use this prompt when every factual assertion must be traceable to evidence—compliance audit trails, legal research, clinical decision support, or financial analysis. It is also valuable when building automated regression tests for your RAG pipeline, where you need to catch citation drift before it reaches users. The prompt distinguishes between verifiable claims that require citations and common knowledge that does not, reducing false positives that would otherwise flood your review queue. It produces structured remediation requests that your system can act on: either requesting attribution from a retriever, flagging for human review, or stripping the unsupported claim before delivery.

Do not use this prompt as a substitute for proper retrieval. If your retriever is returning irrelevant or insufficient context, no amount of post-generation flagging will fix the root cause. This prompt also should not be used on answers that are purely conversational, opinion-based, or creative—it will flag every statement as unsupported. For high-stakes domains, always pair this prompt with a human review step for claims that cannot be automatically resolved. The prompt is a filter, not a fact-checker; it verifies the presence of citations, not the accuracy of the cited evidence against the source document.

This prompt template is designed for a post-generation review pipeline. Its job is to take a previously generated answer and the source documents that were used to produce it, then identify every factual claim that lacks a direct supporting citation. The prompt distinguishes between claims that require evidence and statements of common knowledge that do not, reducing false positives that would otherwise overwhelm a review queue. Use this template as the core instruction for an LLM call that runs after answer generation but before the answer is shown to a user or written to a system of record.

codeCopy
You are an auditor reviewing a generated answer against its source documents. Your task is to identify every factual claim in the answer that lacks a supporting citation from the provided sources.

## INPUT

**Generated Answer:**
[GENERATED_ANSWER]

**Source Documents (with IDs):**
[SOURCE_DOCUMENTS]

## DEFINITIONS

- **Factual Claim:** A verifiable statement about the world, events, people, data, processes, or relationships that could be true or false. Includes specific numbers, dates, names, causal claims, comparisons, and technical assertions.
- **Common Knowledge:** Information that a general audience would accept without needing a specific source. Includes widely known historical dates (e.g., "World War II ended in 1945"), basic scientific facts (e.g., "water freezes at 0°C"), and idiomatic expressions. Do not flag these.
- **Supporting Citation:** A direct, unambiguous link between a claim and a source document. The source must contain the information stated in the claim. Inference or interpretation beyond what the source explicitly states does not count as support.

## INSTRUCTIONS

1. Extract every factual claim from the generated answer. Number them sequentially.
2. For each claim, determine if it requires a citation or qualifies as common knowledge.
3. For claims requiring a citation, check if any provided source document supports it. Record the source ID if found.
4. Flag every claim that requires a citation but has no supporting source.
5. Assign a severity rating to each flagged claim:
   - **CRITICAL:** The claim is central to the answer's main point and is likely false or misleading without evidence.
   - **HIGH:** The claim is specific and verifiable but not the core thesis.
   - **MEDIUM:** The claim adds detail but the answer remains substantially correct without it.
   - **LOW:** The claim is peripheral or the missing citation is likely an oversight with minimal impact.
6. For each flagged claim, suggest a remediation action:
   - **CITE:** Locate the supporting source and add a citation.
   - **REMOVE:** Delete the unsupported claim from the answer.
   - **REWRITE:** Rephrase the claim to reflect only what the sources support.
   - **ESCALATE:** Flag for human review because the claim is high-stakes and sources are ambiguous.

## OUTPUT FORMAT

Return a JSON object with this exact structure:

{
  "total_claims_extracted": <integer>,
  "common_knowledge_exemptions": <integer>,
  "claims_requiring_citation": <integer>,
  "flagged_claims": [
    {
      "claim_id": <integer>,
      "claim_text": "<exact text from answer>",
      "severity": "CRITICAL|HIGH|MEDIUM|LOW",
      "remediation": "CITE|REMOVE|REWRITE|ESCALATE",
      "reasoning": "<one sentence explaining why this claim is flagged and why no source supports it>"
    }
  ],
  "false_positive_notes": [
    "<any claim that appeared unsupported but was confirmed as common knowledge, with brief justification>"
  ]
}

## CONSTRAINTS

- Do not flag claims that are directly supported by at least one source document.
- Do not flag common knowledge. If uncertain whether something is common knowledge, flag it and note the uncertainty in false_positive_notes.
- If a claim is partially supported but adds unsupported detail, flag the unsupported portion and explain the gap.
- If the generated answer contains no factual claims, return an empty flagged_claims array with total_claims_extracted set to 0.
- Preserve the exact claim text in claim_text. Do not paraphrase or truncate.

To adapt this template, replace [GENERATED_ANSWER] with the full text output from your RAG pipeline and [SOURCE_DOCUMENTS] with the retrieved chunks, each prefixed by a stable identifier. The source format should include both the document ID and the chunk text so the auditor can trace claims to specific passages. If your system uses page numbers or section headers, include those in the source representation. The output schema is designed for programmatic consumption: parse the JSON, route CRITICAL and HIGH severity flags to a human review queue, and automatically apply REMOVE or REWRITE remediations for MEDIUM and LOW flags if your risk tolerance permits. Always log the full audit result alongside the answer for downstream governance and eval analysis.

The Missing Citation Flagging Prompt operates as a post-generation guardrail, not a real-time generation constraint. It should be called after the primary RAG answer is produced but before the answer is returned to the user. This separation keeps the generation prompt focused on synthesis while the flagging prompt handles audit. The typical call pattern is: generate answer with citations → run flagging prompt on the answer → if flags are returned, decide whether to repair, retry, or escalate. This prompt is stateless and idempotent; running it twice on the same input should produce the same flags.

Integration points: Wire this prompt into your answer pipeline as a conditional step. After the primary LLM generates an answer, extract all factual claims (sentences or clauses that assert something about the world) and pass them alongside the source context and the full answer to the flagging prompt. The prompt returns a structured list of claims that lack citations. Use a JSON output contract with fields like claim_text, claim_position (character offset or sentence index), missing_citation_reason (e.g., no_source_found, common_knowledge_exempt, ambiguous_attribution), and suggested_action (e.g., add_citation, remove_claim, mark_as_opinion). Validate the JSON schema before acting on the results. If the output fails schema validation, retry once with a stricter format instruction; if it fails again, log the raw output and escalate for manual review.

Retry and repair logic: When flags are returned, do not automatically regenerate the entire answer. Instead, route the flagged claims back to the generation model with a targeted repair prompt: 'The following claims in your answer lack citations. For each claim, either add a citation from the provided sources or rewrite the claim to remove the unsupported assertion.' This preserves the rest of the answer and avoids introducing new errors. Set a maximum of two repair attempts per answer. If flags persist after two repairs, escalate to a human reviewer with the original answer, the flagged claims, and the source context. Log every flagging result, repair attempt, and escalation decision with timestamps and model versions for audit trails.

Model choice and latency: This prompt works well with fast, instruction-following models (e.g., GPT-4o-mini, Claude 3.5 Haiku) because the task is classification-oriented rather than generative. Run it in parallel with other post-generation checks (hallucination detection, toxicity screening) to minimize added latency. For high-throughput systems, batch multiple answers into a single flagging call if your context window permits. Common knowledge exemptions are the primary source of false positives; maintain a configurable allowlist of claim patterns (e.g., widely known dates, definitions of field-specific terms) that should bypass flagging. Review this allowlist monthly against production logs to catch over-exemption drift. Always log when a claim is exempted and why, so reviewers can audit the decision.