RAG Citation Audit Prompt for Fabricated Sources

This prompt is designed for RAG product teams who need a systematic, automated way to verify citation integrity after a model generates a response. The core job-to-be-done is cross-referencing every in-text citation, footnote, and reference list entry against a ground-truth source manifest to catch fabricated DOIs, URLs, author names, or source IDs. The ideal user is an engineering lead or ML engineer integrating this as a post-generation validation step in a RAG pipeline, where the model output and the complete source list are already available. This is not a prompt for generating answers or retrieving sources; it is a targeted audit and repair tool for citation hygiene.

You should use this prompt when your application surfaces citations to end users, stores them in a database, or submits them for compliance review. The prompt requires two concrete inputs: the full model-generated text containing citations and a structured source manifest listing every valid source with its identifier, title, URL, DOI, and author fields. The output is a corrected citation map that flags fabricated references, provides a reason for each flag, and suggests corrections when a close match exists in the source manifest. This is particularly valuable in regulated domains like legal tech, healthcare, and finance, where a hallucinated citation can carry compliance risk. In such cases, the prompt's output should always route to a human review queue before publication.

Do not use this prompt as a substitute for proper retrieval hygiene or source manifest management. It cannot fix systemic RAG failures where the retriever consistently returns irrelevant or insufficient context. It also cannot verify the factual accuracy of a claim against a source—only whether the citation itself is real. For factual consistency checks, pair this with a separate claim-grounding verification prompt. Before deploying, test the prompt against a golden dataset of known fabricated citations to calibrate its false positive and false negative rates, and log every audit result for observability into your citation quality over time.

This prompt template is the core of the RAG Citation Audit workflow. It is designed to be inserted into your application's post-generation validation layer. The model receives the original generated text and the complete list of valid sources that were available during retrieval. Its sole job is to act as a skeptical auditor: it must assume every citation is fabricated until it finds a match in the provided source list. The output is a structured report that flags invented DOIs, URLs, author names, or dangling reference IDs, and provides a corrected citation map your application can use to strip or replace bad citations before the response reaches the user.

textCopy
You are a citation auditor for a RAG system. Your task is to verify every citation, reference, and source claim in the provided model output against the official source list. You must assume any citation not found in the source list is fabricated.

## INPUT

**Model Output to Audit:**

[OUTPUT_TEXT]

codeCopy

**Official Source List (the only valid sources):**

[SOURCE_LIST]

codeCopy

Each source in the list includes a unique [SOURCE_ID] and its full citation details (title, authors, DOI, URL, year, etc.).

## INSTRUCTIONS

1. Extract every citation, reference marker, source mention, DOI, URL, and author-name claim from the Model Output.
2. For each extracted item, attempt to match it to exactly one source in the Official Source List. Match on [SOURCE_ID] first, then on DOI, URL, or author+title combinations.
3. Classify each extracted item as:
   - **GROUNDED**: Found a clear match in the source list.
   - **FABRICATED**: No match found. The model invented this source.
   - **MISMATCHED**: A source was cited but the surrounding claim does not align with what the source actually says (if source content is provided).
4. For FABRICATED items, note what was invented (e.g., fake DOI, hallucinated author, non-existent URL).
5. Produce the output in the exact JSON schema specified below.

## CONSTRAINTS

- Do not use any external knowledge. Only the Official Source List is authoritative.
- If a citation is ambiguous, flag it as FABRICATED. Err on the side of caution.
- Do not modify the original Model Output text in the audit report; only produce the audit map.
- If no citations are found in the output, return an empty audit list with `citation_count: 0`.

## OUTPUT SCHEMA

Return a single JSON object with this structure:
{
  "citation_count": <total number of citations found in the output>,
  "grounded_count": <number of citations matched to the source list>,
  "fabricated_count": <number of citations with no match>,
  "mismatched_count": <number of citations where the source exists but the claim doesn't align>,
  "audit_entries": [
    {
      "extracted_citation": <the citation text as it appears in the output>,
      "citation_type": "in-text-reference" | "DOI" | "URL" | "author-name" | "source-id" | "other",
      "status": "GROUNDED" | "FABRICATED" | "MISMATCHED",
      "matched_source_id": <the [SOURCE_ID] from the source list if GROUNDED, otherwise null>,
      "fabrication_detail": <if FABRICATED, describe what was invented; otherwise null>,
      "correction": <if FABRICATED, suggest removal or a valid alternative source ID if one exists; otherwise null>,
      "claim_context": <the sentence or paragraph surrounding the citation for human review>
    }
  ],
  "corrected_output": <the original Model Output with fabricated citations removed and replaced with [CITATION REMOVED] markers, or null if no changes needed>
}

EXAMPLES

Fabricated DOI: Extracted: "doi:10.1234/fake.2024" Status: FABRICATED Fabrication Detail: "DOI not present in any source in the official list." Correction: "Remove citation. No valid alternative found."

Mismatched Source: Extracted: "[Smith et al., 2023]" Status: MISMATCHED Fabrication Detail: "Source exists but the claim 'AI reduces costs by 90%' is not supported by the abstract provided." Correction: "Replace claim with verifiable statement from source or remove."

RISK LEVEL

[HIGH_RISK] if the output will be shown to users, stored in a database, or used in regulated workflows. Always log the full audit report before any automated correction is applied.

To adapt this template for your system, replace [OUTPUT_TEXT] with the raw model response you need to audit, and [SOURCE_LIST] with your retrieved context formatted as a structured list. Each source must have a unique [SOURCE_ID] that your application can use to trace corrections. If your source list includes full text or abstracts, include them to enable mismatch detection. For high-throughput systems, consider batching multiple outputs into a single audit call, but keep each audit entry traceable to its original request. Before deploying, run this prompt against a golden dataset of known fabricated citations to calibrate your model's false-positive and false-negative rates.

The RAG Citation Audit Prompt is designed to sit as a post-generation validation step in a retrieval-augmented generation pipeline. After your primary model produces a response with citations, this audit prompt receives the generated text and the original source list as inputs, then cross-references every citation against the provided sources. The output is a corrected citation map that flags invented DOIs, URLs, author names, or reference IDs. This prompt should not be used as the primary generation step—it is a repair and verification harness that catches fabricated sources before they reach users or databases.

Wire this prompt into your application as a synchronous validation call immediately after the primary generation step and before the response is returned to the user or written to storage. The input harness requires two fields: [GENERATED_TEXT] containing the full model response with inline citations, and [SOURCE_LIST] containing the complete set of retrieved sources with their identifiers, titles, authors, DOIs, and URLs. Parse the audit prompt's output into a structured CitationAuditResult object with fields for valid_citations, fabricated_citations, unverifiable_citations, and corrected_citation_map. Implement a retry loop with a maximum of two attempts if the audit output fails to parse as valid JSON—on the second failure, escalate to a human review queue rather than silently passing unverified citations downstream. For high-risk domains such as legal, medical, or financial applications, always route outputs containing any fabricated_citations to a human approval step before publication, regardless of parse success.

Choose a model with strong instruction-following and structured output capabilities for the audit step—GPT-4o, Claude 3.5 Sonnet, or equivalent. Avoid running citation audits on low-cost or small models that may hallucinate during the audit itself, creating a recursive trust problem. Log every audit result with the original generated text, source list, and audit verdict for downstream observability. Set up an eval harness that periodically tests the audit prompt against known fabricated citations to measure recall (did it catch the fake?) and precision (did it falsely flag real citations?). The most common production failure mode is the audit model accepting a fabricated citation because the source list is incomplete or poorly structured—ensure your retrieval pipeline passes complete source metadata, not just chunk text, into the audit harness.