Citation Completeness Audit Prompt

This prompt is designed for quality assurance teams, compliance reviewers, and AI operations engineers who need to systematically verify that AI-generated answers are properly supported by their cited sources. The primary job-to-be-done is a post-generation audit: you have an AI-generated answer, the set of sources it claims to cite, and the full text of those sources, and you need a structured report that flags unsupported claims, missing citations, fabricated references, and citation-to-claim misalignment. This is not a real-time generation guardrail. It belongs in review queues, batch QA pipelines, and compliance verification workflows where human reviewers need a systematic first-pass analysis before signing off on AI outputs.

The ideal user has access to the complete source text for every cited reference. Without full source text, the audit cannot verify whether a claim is actually supported by the cited passage. The prompt expects explicit citation markers in the AI-generated answer—inline numbers, footnote references, or structured citation arrays. If your answer format lacks explicit citation markers, you should first apply a citation marker prompt or restructure the output before running this audit. The prompt also assumes that the set of sources provided is the complete set the model was allowed to cite; missing sources in the audit input will produce false positives for fabrication detection.

Do not use this prompt when you lack access to the full source text, when the answer format does not include explicit citation markers, or when you need real-time intervention during generation. For real-time hallucination prevention, use a mandatory citation prompt that enforces citation requirements during generation. For cases where you need to verify claims against a broader knowledge base rather than a fixed set of provided sources, use a fact-checking and claims verification prompt instead. After running this audit, route outputs with high-severity findings to human review and consider using the findings to improve your retrieval pipeline or citation instructions.

This prompt template is designed to be dropped directly into your quality assurance workflow. It instructs the model to act as an auditor, systematically comparing an AI-generated answer against a set of provided source documents. The goal is not to re-answer the question, but to produce a structured, pass/fail audit report that flags unsupported claims, verifies citation accuracy, and identifies potential fabrication. The placeholders allow you to inject the original user question, the final AI-generated answer, and the raw retrieved source context that was available to the generation model.

textCopy
You are a citation completeness auditor. Your task is to audit an AI-generated answer against a set of provided source documents. You must determine if every factual claim in the answer is supported by at least one source, if the citations are accurate, and if there is any risk of fabrication.

## INPUTS
**User Question:** [USER_QUESTION]
**AI-Generated Answer:** [AI_ANSWER]
**Source Documents (with IDs):** [SOURCE_DOCUMENTS]

## AUDIT INSTRUCTIONS
1.  **Claim Extraction:** Decompose the AI-Generated Answer into a list of discrete, verifiable factual claims.
2.  **Evidence Matching:** For each claim, search the Source Documents for direct supporting evidence. A claim is supported only if a source explicitly states the fact.
3.  **Citation Verification:** Check every citation marker in the AI-Generated Answer (e.g., [1], [Doc A]). Verify that the cited source exists and that its content directly supports the specific claim it is attached to.
4.  **Fabrication Risk Flagging:** Flag any claim that appears factual but has no supporting evidence in the Source Documents. Also flag any citation that points to a non-existent source or a source that does not support the claim.

## OUTPUT FORMAT
You must produce a valid JSON object conforming to this exact schema:
{
  "audit_result": "PASS" | "FAIL",
  "overall_score": <number 0-100>,
  "claims": [
    {
      "claim_text": "string",
      "support_status": "SUPPORTED" | "UNSUPPORTED" | "CONTRADICTED",
      "source_ids": ["string"],
      "auditor_notes": "string"
    }
  ],
  "citation_issues": [
    {
      "citation_marker": "string",
      "issue_type": "FABRICATED_SOURCE" | "MISMATCHED_CLAIM" | "MISSING_SOURCE",
      "details": "string"
    }
  ],
  "fabrication_risk_flags": ["string"],
  "summary": "string"
}

## CONSTRAINTS
- The `overall_score` must be 100 if `audit_result` is PASS. Any UNSUPPORTED claim or citation issue must result in a FAIL and a score less than 100.
- `source_ids` must only contain IDs that exist in the provided Source Documents.
- If no factual claims are made in the answer, `audit_result` is PASS with a score of 100.
- Do not add any commentary outside the JSON object.

To adapt this template, replace the placeholders with your actual data. The [SOURCE_DOCUMENTS] placeholder should be filled with a pre-formatted string that clearly separates each document and assigns it a unique ID (e.g., [Doc 1] ..., [Doc 2] ...). This prompt is most effective when used as a post-generation validation step in a RAG pipeline. For high-stakes domains, the structured JSON output should be logged and reviewed before the answer is shown to an end-user. A common failure mode is the auditor model hallucinating source IDs; mitigate this by strictly enforcing the source_ids constraint in your output validator and retrying the audit if a violation is detected.

The Citation Completeness Audit Prompt is designed to sit inside a quality assurance pipeline, not as a one-off manual check. In production, it typically runs after an answer generation step, receiving the original user query, the generated answer, and the set of retrieved source passages as input. The prompt's structured audit report becomes the decision point for whether an answer passes automated QA, requires human review, or triggers a regeneration loop. This means the implementation harness must handle input assembly, output validation, retry logic, and routing based on the audit verdict.

Start by constructing the prompt input from three pipeline artifacts: the user's original query, the generated answer text, and the full set of retrieved passages with their source metadata. The prompt expects these as structured inputs. Before calling the model, validate that all required fields are present and non-empty. An empty source list should immediately fail the audit without calling the model, since citation completeness is undefined when no sources were retrieved. After receiving the model's response, parse the structured audit report and validate it against an expected schema. At minimum, check that the verdict field contains a valid enum value such as PASS, FAIL, or REVIEW, and that any flagged claims include the specific passage IDs they reference. If the output fails schema validation, retry once with a repair prompt that includes the validation error message. If the retry also fails, escalate to a human review queue with the raw output attached.

For high-trust domains such as legal, healthcare, or finance, the audit verdict should never be the final word without additional safeguards. Route PASS verdicts to a sampling-based human review queue where a configurable percentage of passing audits are spot-checked. Route FAIL verdicts to either an automated regeneration loop that re-queries the RAG system with expanded retrieval, or directly to a human reviewer if the failure count exceeds a threshold. Route REVIEW verdicts to a dedicated human review queue with the full audit report, source passages, and generated answer presented side by side. Log every audit run with a trace ID, the prompt version, the model used, the raw audit output, and the final routing decision. This trace data is essential for debugging citation quality trends and for producing audit evidence if downstream consumers question the system's outputs.

Model choice matters here. The audit task requires careful comparison between claims and source text, which benefits from models with strong instruction-following and long-context reasoning. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are all suitable. Avoid smaller or faster models that may miss subtle mismatches between claims and evidence. If latency is critical, consider running the audit asynchronously after the answer is delivered to the user, flagging problematic answers for retrospective correction rather than blocking the response. Finally, treat the audit prompt itself as a product artifact: version it, test it against a golden dataset of known citation failures and correct citations, and run regression tests before deploying changes. A broken audit prompt is worse than no audit at all, because it creates a false sense of safety.