Hallucinated Citation Removal Prompt Template

This prompt is a post-generation repair tool for RAG systems, documentation assistants, and any AI product where a model output contains citations that cannot be traced back to a provided evidence set. Its job is to identify invented references, remove or replace them with abstention markers, and produce a structured audit log so downstream systems and human reviewers know exactly what was fabricated. Use this prompt when citation integrity is a hard requirement and hallucinated references would erode user trust, break compliance, or corrupt downstream databases.

This is not a prompt for generating citations. It assumes a model has already produced a response with inline citation markers and that you have a ground-truth set of valid source identifiers to check against. The prompt works best when you can provide both the original model output and the complete list of authorized source IDs as input. It is designed for batch repair workflows, real-time guardrails before user-facing display, and compliance audit pipelines where every citation must be traceable to an approved source. Do not use this prompt when you lack a definitive source list—without ground truth, the repair itself becomes unreliable.

Before deploying this prompt, ensure your pipeline captures the raw model output, the authorized source ID list, and any citation format conventions your system enforces. Wire the prompt into a post-processing step that runs after generation but before the response reaches the user or database. For high-risk domains, always route outputs with unresolved hallucinations to human review rather than silently removing citations. The audit log this prompt produces should be persisted alongside the repaired output for downstream traceability and debugging.

This prompt template is designed to be dropped into a post-generation repair loop. It receives the original model output and the authoritative evidence set, then produces a cleaned version with hallucinated citations removed or replaced by abstention markers. The template uses square-bracket placeholders so you can wire it into your application's context assembly layer without manual editing.

textCopy
You are a citation auditor. Your job is to identify and remove citations in the provided [OUTPUT_TEXT] that have no matching source in the [EVIDENCE_SET].

[EVIDENCE_SET]
[SOURCE_METADATA]

[OUTPUT_TEXT]

[CONSTRAINTS]
- Do not modify any text that is not a citation.
- For each citation, check whether the cited source exists in [EVIDENCE_SET] by matching on [MATCH_FIELD].
- If a citation has no match, replace it with [ABSTENTION_MARKER].
- If a citation has a partial match, flag it as [PARTIAL_MATCH_ACTION].
- Preserve the original text structure, formatting, and non-citation content exactly.

[OUTPUT_SCHEMA]
{
  "cleaned_output": "string (the full output with hallucinated citations removed or replaced)",
  "audit_log": [
    {
      "original_citation": "string",
      "match_status": "MATCH | NO_MATCH | PARTIAL_MATCH",
      "matched_source_id": "string | null",
      "action_taken": "RETAINED | REMOVED | REPLACED_WITH_ABSTENTION | FLAGGED",
      "confidence": "HIGH | MEDIUM | LOW"
    }
  ],
  "summary": {
    "total_citations_found": "integer",
    "hallucinated_citations_removed": "integer",
    "partial_matches_flagged": "integer",
    "retained_citations": "integer"
  }
}

[EXAMPLES]
Example 1:
Input: "The sky is blue (Smith, 2023)."
Evidence set contains: Smith, 2023
Output: RETAINED, no change.

Example 2:
Input: "Water is wet (Johnson, 2019)."
Evidence set does not contain: Johnson, 2019
Output: "Water is wet [citation not found]."
Audit log entry: NO_MATCH, REMOVED.

[RISK_LEVEL]
HIGH - Incorrect citation removal can delete valid references. Always log every removal decision. Route outputs with more than [REMOVAL_THRESHOLD]% citations removed for human review before publishing.

To adapt this template, replace each square-bracket placeholder with your application's actual values. [EVIDENCE_SET] should contain the full text or structured records of all authorized sources. [MATCH_FIELD] defines which identifier to use for matching—typically a source ID, author-date pair, or URL. Set [ABSTENTION_MARKER] to your preferred placeholder text, such as [citation not found] or [source unavailable]. The [OUTPUT_SCHEMA] is designed for programmatic consumption: your repair harness should parse the JSON response, validate the audit log against the original output, and route high-removal cases to a human review queue. Always test with known hallucinated citations before deploying to production.

This prompt is designed to sit as a post-generation safety net in your RAG pipeline, immediately after the model produces an answer with citations and before that answer reaches the user or a database. The harness should receive three inputs: the original model-generated text containing citations, the complete set of retrieved source chunks with their identifiers, and a configurable similarity threshold for matching claims to evidence. The prompt is stateless and idempotent—running it twice on the same input should produce the same hallucination audit log and cleaned output, which makes it safe for retry loops.

Wire the prompt into an application function that first extracts all citation markers from the model output using a regex pattern appropriate to your citation format (e.g., \[\d+\] for numeric, \(Author, Year\) for parenthetical). Pass the extracted citations, the full text, and the source evidence set into the prompt template. On the response, parse the JSON output into two structures: a cleaned text with hallucinated citations removed or replaced with [CITATION NOT FOUND] markers, and a hallucination audit log array containing each flagged citation, its claimed source text, the match score against the evidence set, and the action taken (removed, replaced, or retained). Validate that every citation in the cleaned output has a corresponding entry in the audit log before proceeding.

Implement a retry strategy with a maximum of two additional attempts if the model returns malformed JSON or if the audit log contains citations marked as uncertain. On retry, include the previous failure reason in the [CONSTRAINTS] block to guide the model toward a correctable output. Log every invocation—including the original text, the evidence set hash, the cleaned output, and the full audit log—to an append-only store for debugging and compliance. For high-risk domains such as legal or medical applications, route any output where more than 5% of citations were removed to a human review queue before publication. Avoid running this prompt on streaming outputs mid-stream; always accumulate the full response first to prevent partial citation analysis that could miss context-dependent attributions.