Citation Presence and Format Validator Prompt

This prompt is for RAG system builders and QA engineers who need to enforce citation discipline in LLM-generated answers. The job-to-be-done is automated verification that every factual claim in an output is backed by a corresponding citation, that citations match a required format (e.g., [Source: doc_id, §section]), and that source references are resolvable against a provided context set. The ideal user is an AI platform engineer embedding this validator into a CI/CD pipeline or a post-generation review step before an answer reaches the user. Required context includes the original LLM output, the set of source documents or chunks that were provided as grounding context, and a citation format specification.

Do not use this prompt when the underlying RAG pipeline does not provide retrievable source identifiers, when citations are optional by design, or when the output format is unstructured prose without explicit claim boundaries. This validator is not a fact-checker—it checks citation presence and format, not whether the cited source actually supports the claim. For factual accuracy verification, pair this with a separate Factual Claim Grounding Check Prompt. The validator also assumes citations are inline or footnoted in a parseable pattern; if your system uses implicit attribution or conversational references, you will need a pre-processing step to extract claim-citation pairs before validation.

High-risk domains such as healthcare, legal, or finance should never rely solely on automated citation validation. In these contexts, this prompt serves as a first-pass filter that flags orphan claims and malformed references for mandatory human review. Configure the prompt's severity thresholds to block outputs with missing citations rather than merely warning, and log every validation result with the original output, source context, and violation details for audit trails. The next step after reading this section is to review the prompt template, adapt the citation format pattern to your system's convention, and wire the validator into your output post-processing harness with clear pass/fail gating logic.

The following prompt template is designed to be dropped into a validation step after your primary RAG answer generation. It takes the generated answer and the retrieved source context as inputs, then systematically checks that every factual claim is backed by a citation, that citations follow your required format, and that source references are resolvable. Use this as a gating step before returning answers to users or as part of a CI/CD eval harness.

codeCopy
You are a citation validator for a RAG system. Your job is to check an AI-generated answer against its source context and flag citation problems.

## INPUT

**Generated Answer:**
[GENERATED_ANSWER]

**Source Context (with source IDs):**
[SOURCE_CONTEXT]

## REQUIRED CITATION FORMAT

Citations must follow this pattern:
- Inline: `[Source: {source_id}]` immediately after the claim it supports
- Source IDs must match exactly one ID from the Source Context above
- Every factual claim, statistic, date, name, or quoted material must have at least one citation
- Opinions, summaries, and transitional text do not require citations

## OUTPUT SCHEMA

Return a JSON object with this exact structure:
{
  "overall_pass": boolean,
  "total_claims_checked": integer,
  "orphan_claims": [
    {
      "claim_text": string,
      "claim_position": string,
      "severity": "blocking" | "warning",
      "reason": string
    }
  ],
  "malformed_citations": [
    {
      "citation_text": string,
      "position": string,
      "issue": string,
      "suggested_fix": string
    }
  ],
  "unresolvable_references": [
    {
      "citation_text": string,
      "source_id_used": string,
      "issue": string
    }
  ],
  "summary": string
}

## VALIDATION RULES

1. Extract every factual claim from the Generated Answer.
2. For each claim, check if it has a corresponding citation in the required format.
3. If a claim lacks any citation, add it to `orphan_claims` with severity "blocking".
4. If a claim has a citation but the source ID does not appear in Source Context, add it to `unresolvable_references`.
5. If a citation exists but does not match the required format pattern, add it to `malformed_citations`.
6. Set `overall_pass` to `true` only if all three arrays are empty.
7. In `summary`, describe the overall result and the most critical issue if any.

## CONSTRAINTS

- Do not fabricate source IDs or claims.
- If the Generated Answer contains no factual claims, `total_claims_checked` should be 0 and `overall_pass` should be `true`.
- Flag ambiguous formatting as malformed rather than assuming intent.
- If the same source ID appears multiple times in Source Context, treat it as valid for any citation using that ID.

Adaptation guidance: Replace [GENERATED_ANSWER] with the raw text output from your RAG pipeline. Replace [SOURCE_CONTEXT] with the retrieved passages, each prefixed with a unique source ID that your system can resolve back to a document, chunk, or URL. If your citation format differs—for example, you use footnote-style markers or bracketed numbers—update the REQUIRED CITATION FORMAT section accordingly. The output schema is designed for machine consumption in a validation pipeline; you can extend it with additional fields like confidence_score or requires_human_review if your workflow routes borderline cases to a review queue.

What to do next: Wire this prompt into a post-generation validation step. If overall_pass is false, either route the answer for human review, trigger a retry with the violation details fed back as correction instructions, or block the response entirely depending on your risk tolerance. For high-stakes domains such as healthcare or legal, always require human review when orphan_claims contains any blocking-severity items. Before deploying, run this validator against a golden dataset of known good and bad citation examples to calibrate your pass/fail thresholds and catch false positives on stylistic text that the model may incorrectly flag as factual claims.

This prompt is designed to be called as a post-processing step after your primary RAG answer generation. It should not be the final user-facing output. Instead, treat it as a validation gate: the primary model generates an answer with citations, and this validator prompt inspects that answer against the provided source context. The validator returns a structured report that your application can use to decide whether to return the answer to the user, trigger a retry, or escalate for human review. The prompt expects three inputs: the original user question, the generated answer with citations, and the full set of retrieved source documents that were provided to the generation model.

Wire this prompt into your application as a synchronous validation step. After receiving the generated answer, immediately call the validator with the same context. Parse the JSON output and check the pass field. If pass is true, route the answer to the user. If pass is false, inspect the violations array. For orphan claims (claims with no citation), trigger a retry with explicit instructions to cite every factual statement. For malformed citations (wrong format, unresolvable references), attempt a single repair retry before escalating. Log every validation result—including the validator's raw output, the decision made, and the final answer shown to the user—for auditability and prompt improvement over time. In high-stakes domains like healthcare or legal, a pass result should still route to a human review queue before reaching the end user.

For CI/CD integration, build a test harness that runs this validator against a golden dataset of known good and bad answers. A prompt change should only pass the release gate if the validator's precision and recall on your golden set remain above your defined thresholds. Watch for false positives where the validator flags correctly formatted citations as malformed, and false negatives where it misses orphan claims embedded in complex sentence structures. The most common production failure mode is the validator hallucinating violations when the source context is long or contains citation-like patterns in the body text. Mitigate this by keeping source chunks focused and by adding a confidence threshold check: if the validator's own confidence score for a violation is below 0.7, route the answer for human review rather than auto-rejecting.