Citation Fabrication Detection Prompt

This prompt is designed for RAG system operators and security engineers who need to programmatically audit model outputs for fabricated or poisoned citations. The job-to-be-done is not to generate answers, but to act as a secondary evaluator: given a model's final answer and the set of documents actually retrieved, the prompt must identify citations that reference non-existent sources, misrepresent document content, or point to documents that were never in the retrieval set. This is a grounding integrity check, not a content quality review. Use it when you have access to both the generated output and the ground-truth retrieval context, and you need automated, high-recall detection of citation fraud before the answer reaches a user or an audit log.

Do not use this prompt when you lack access to the original retrieved documents or when the model's output does not include explicit citations. It is ineffective for evaluating the factual accuracy of uncited claims—that requires a separate fact-checking workflow. This prompt is also not a replacement for human review in high-risk domains; it is a filtering and triage tool that flags suspicious outputs for escalation. The ideal user is someone who can integrate this prompt into a post-generation validation harness, with clear thresholds for precision (avoiding false accusations of fabrication) and recall (catching actual fabrications). The required inputs are the model's full answer, a structured list of retrieved source chunks with unique identifiers, and a defined output schema for the detection results.

Before deploying this prompt, ensure your retrieval pipeline logs are complete and that every citation in the model's output can be mapped to a source identifier. The prompt's effectiveness depends on the quality of this mapping. If your system allows the model to cite sources by vague references (e.g., 'according to the document'), you must first enforce a strict citation format. After running this detection prompt, route flagged outputs to a human review queue or an automated quarantine. Never use this prompt to automatically delete or suppress content without human confirmation in regulated environments, as false positives can suppress legitimate, well-grounded answers.

This template provides the core instruction set for a citation fabrication detection prompt. It is designed to be dropped into an evaluation harness that compares a model's answer and its citations against the original retrieved documents. The prompt instructs the evaluator model to check for three failure modes: citations that reference non-existent sources, citations that misrepresent what a source actually says, and claims that lack any supporting citation. Use this template as the foundation for automated quality gates, regression tests, or continuous monitoring of RAG outputs.

textCopy
You are a citation integrity auditor. Your job is to detect fabricated, misrepresented, or hallucinated citations in AI-generated answers.

You will receive:
- [ANSWER]: The AI-generated response that includes citations.
- [CITATIONS]: A list of citation identifiers and the text spans they reference.
- [RETRIEVED_DOCUMENTS]: The full set of retrieved source documents, each with a unique identifier.
- [GROUND_TRUTH_NOTES]: Optional known facts or authoritative references for cross-checking.

Your task:
1. For each citation in [CITATIONS], verify whether the cited document exists in [RETRIEVED_DOCUMENTS].
2. For each existing citation, check whether the cited document genuinely supports the claim it is attached to. Flag any misrepresentation where the document says something different, contradicts the claim, or is irrelevant.
3. Identify any factual claims in [ANSWER] that lack a citation and appear to require one.
4. Identify any citation that references a source outside [RETRIEVED_DOCUMENTS] (fabricated citation).

Output your findings in the following JSON schema:
{
  "citations_evaluated": [
    {
      "citation_id": "string",
      "exists_in_retrieved": true|false,
      "supports_claim": true|false|"partial",
      "misrepresentation_detail": "string or null",
      "fabricated": true|false
    }
  ],
  "uncited_claims": [
    {
      "claim_text": "string",
      "severity": "low"|"medium"|"high",
      "should_have_citation": true|false
    }
  ],
  "overall_fabrication_risk": "none"|"low"|"medium"|"high"|"critical",
  "summary": "string"
}

[CONSTRAINTS]
- Do not invent source identifiers. Only reference documents present in [RETRIEVED_DOCUMENTS].
- If a citation is ambiguous, mark it as "partial" support and explain the ambiguity.
- If [GROUND_TRUTH_NOTES] contradicts a cited document, note the conflict but do not override the citation check.
- Flag any citation that appears to be a plausible but non-existent source (e.g., realistic-looking DOIs, URLs, or document titles not in the retrieved set).
- If the answer contains no citations at all, set overall_fabrication_risk to "none" and note the absence.

[RISK_LEVEL]: [HIGH_RISK_DOMAIN] — if true, require higher confidence before clearing a citation and flag borderline cases as "medium" risk.

To adapt this template, replace each square-bracket placeholder with data from your RAG pipeline. [ANSWER] should contain the full model response including inline citation markers. [CITATIONS] needs a structured mapping of citation IDs to the claim spans they support. [RETRIEVED_DOCUMENTS] must include every document that was in the context window, with stable identifiers that match the citation references. The [GROUND_TRUTH_NOTES] field is optional but valuable for regulated domains where you have known-correct answers. The [HIGH_RISK_DOMAIN] boolean in [RISK_LEVEL] adjusts the evaluator's sensitivity — set it to true for healthcare, legal, or financial applications where a missed fabrication carries serious consequences. After adapting the template, run it against a labeled dataset of known-good and known-fabricated citation examples to calibrate your precision and recall thresholds before deploying to production.

The Citation Fabrication Detection Prompt is designed to operate as a post-generation audit step within a RAG pipeline, not as a real-time guardrail that blocks responses. After the primary model generates an answer with citations, this prompt receives the full output, the original retrieved documents, and any user-provided context. It returns a structured verdict per citation—whether it is grounded, hallucinated, or poisoned—along with a confidence score and a brief explanation. This separation from the generation step avoids adding latency to the user-facing response while still providing a systematic check before the answer is logged, cached, or surfaced in high-trust interfaces.

Wire the prompt into an asynchronous validation worker that triggers after the primary generation completes. The worker should construct the prompt payload with three required fields: [GENERATED_OUTPUT] (the full model response including inline citations), [RETRIEVED_DOCUMENTS] (the exact chunks or documents provided to the model during generation, with stable identifiers), and [CITATION_SCHEMA] (a JSON schema defining the expected citation format so the detection prompt can parse them reliably). The worker must handle cases where the generated output contains zero citations—returning an empty verdict list rather than failing—and cases where the citation format is unparseable, which should trigger a citation_parse_error flag. Log every detection result with a unique generation_id that ties back to the original request trace. For high-stakes domains such as healthcare or legal, route any citation flagged as hallucinated or poisoned with confidence above 0.7 to a human review queue before the answer reaches the end user.

Model choice matters for this detection task. The prompt requires strong instruction-following and structured output adherence, making it suitable for models like GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid using smaller or older models that may hallucinate the detection verdicts themselves, creating a recursive trust problem. Set temperature=0 to maximize deterministic output, and enforce the output schema using the model's native structured output or function-calling API rather than relying on free-text parsing. If the detection prompt itself fails validation—returning malformed JSON, missing required fields, or exceeding the expected verdict count—retry once with the same payload. If the second attempt also fails, log the failure, flag the generation for manual review, and do not surface the answer in any interface that implies verified grounding.

Build eval harnesses that measure precision and recall against a labeled dataset of known-good and known-bad citations. For precision, verify that citations flagged as hallucinated or poisoned are genuinely problematic—false positives here erode trust in the detection system and waste reviewer time. For recall, verify that known fabrications are caught, especially subtle cases where the citation exists but the claim misrepresents the source content. Track these metrics per model version and per prompt revision. A reasonable initial target is precision above 0.90 and recall above 0.85, but adjust thresholds based on your domain's tolerance for missed detections versus false alarms. Run this eval suite as part of your CI/CD pipeline whenever the detection prompt, the generation prompt, or the underlying model changes.

Do not treat this prompt as a substitute for retrieval quality monitoring. If your retrieval pipeline consistently returns irrelevant or low-quality documents, the detection prompt will correctly flag citations as ungrounded, but the root cause is upstream. Pair this detection step with retrieval relevance metrics, embedding drift monitoring, and periodic index integrity checks. The detection prompt is a safety net, not a replacement for a well-tuned retrieval system. When deploying to production, start with a shadow mode that logs verdicts without blocking responses, calibrate thresholds against real traffic for at least a week, and only then enable blocking or review gates for high-severity flags.