Citation Policy Enforcement Prompt

This prompt is for RAG-augmented assistant deployments where every factual claim must be traceable to a provided source, and citation discipline is a non-negotiable product requirement. The primary job-to-be-done is automated compliance surveillance: you need a secondary evaluator that reads recent assistant responses and checks whether they adhere to the session's citation policy. The ideal user is an AI engineer or platform operator who already has a working RAG pipeline but is seeing citation fatigue, unsupported claims, or source drift in long-running sessions. You should use this prompt when your assistant operates in a domain where unsupported statements carry regulatory, clinical, financial, or brand risk, and where human review of every turn does not scale.

This prompt is not a replacement for a well-structured system prompt or retrieval pipeline. It is a guardrail that sits in the evaluation layer, typically invoked every N turns or when a confidence threshold is crossed. You should not use this prompt for single-turn Q&A where citation requirements are simple enough to enforce in the generation prompt itself. It is also not suitable for creative or open-ended conversational use cases where factual grounding is not expected. The prompt requires access to the original retrieved sources, the assistant's recent responses, and the session's citation policy definition. Without these three inputs, the compliance check will produce unreliable results.

Before deploying this prompt, ensure you have defined a concrete citation policy that specifies what requires a citation, acceptable citation formats, and how to handle conflicting sources. The prompt works best when paired with a re-prompting workflow: if the compliance check fails, the flagged response should be regenerated with the violation details injected as corrective context. Start by running this prompt on a sample of historical sessions to calibrate your failure thresholds, then integrate it into your production evaluation loop with logging and human spot-checking for edge cases.

This template is designed to be inserted into a multi-turn RAG-augmented assistant workflow as a compliance check. It evaluates the last N assistant responses against the session's citation policy and produces a structured compliance report. The prompt is model-agnostic and expects the conversation history, citation policy, and retrieved sources as inputs. Use it as a guardrail before delivering a response to the user or as a periodic audit step in long-running sessions.

textCopy
You are a citation compliance auditor. Your task is to evaluate whether the assistant's recent responses comply with the session's citation policy.

## CITATION POLICY
[CITATION_POLICY]

## CONVERSATION HISTORY (last [TURN_COUNT] assistant turns)
[CONVERSATION_HISTORY]

## RETRIEVED SOURCES (available during these turns)
[RETRIEVED_SOURCES]

## INSTRUCTIONS
1. For each assistant response, identify every factual claim, data point, or quoted passage.
2. Check whether each claim is backed by a citation to a specific source from the retrieved sources.
3. Flag claims that are:
   - **MISSING_CITATION**: No source provided for a factual statement.
   - **WEAK_CITATION**: Citation exists but does not clearly support the claim.
   - **HALLUCINATED_SOURCE**: Citation references a source ID or quote not present in the retrieved sources.
   - **CITATION_FATIGUE**: Earlier turns cited properly but later turns dropped citations.
4. For each flagged issue, provide the exact assistant text, the turn number, and the specific violation type.
5. If the assistant correctly abstained or expressed uncertainty where sources were insufficient, mark those as **APPROPRIATE_ABSTENTION**.

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "overall_compliance": "COMPLIANT" | "PARTIALLY_COMPLIANT" | "NON_COMPLIANT",
  "turns_evaluated": <number>,
  "violations": [
    {
      "turn_number": <number>,
      "assistant_text": "<excerpt>",
      "violation_type": "MISSING_CITATION" | "WEAK_CITATION" | "HALLUCINATED_SOURCE" | "CITATION_FATIGUE",
      "claim": "<the unsupported or weakly supported claim>",
      "available_sources": ["<source_id>"] | null,
      "severity": "CRITICAL" | "MAJOR" | "MINOR"
    }
  ],
  "appropriate_abstentions": [
    {
      "turn_number": <number>,
      "assistant_text": "<excerpt>",
      "reason": "<why abstention was correct>"
    }
  ],
  "corrective_action": "<if non-compliant, what should be re-prompted or corrected>"
}

## CONSTRAINTS
- Do not flag stylistic choices or tone unless they affect citation accuracy.
- If a claim is common knowledge within the domain and the policy allows it, do not flag it.
- If the citation policy explicitly permits uncited summaries, respect that boundary.
- Return ONLY the JSON object. No preamble, no commentary.

To adapt this template, replace [CITATION_POLICY] with the session's specific citation rules—such as 'every factual claim must cite a source ID' or 'direct quotes require inline citations, summaries require source attribution.' The [TURN_COUNT] variable controls how far back the audit looks; set it based on your context window budget and session length. [CONVERSATION_HISTORY] should contain only the assistant's responses, not user messages, to keep the evaluation focused. [RETRIEVED_SOURCES] must include the source IDs and content that were available to the assistant during those turns. If your RAG pipeline uses source metadata, include it here to enable more precise hallucination detection.

Before deploying this prompt, test it against known violation cases: a response with no citations, a response citing a non-existent source ID, and a response where citations gradually disappear across turns. Validate that the JSON output parses correctly and that violation types match your operational definitions. For high-stakes domains such as healthcare or legal, always route CRITICAL severity violations to human review before the response reaches the user. Do not use this prompt as the sole enforcement mechanism—pair it with a re-prompting step that injects the corrective action back into the assistant's context.

The Citation Policy Enforcement Prompt is designed to run as a post-generation guardrail, not as part of the primary answer generation step. After your RAG pipeline produces an assistant response, pass that response—along with the original retrieved sources and the session's citation policy—to this prompt. The output is a structured compliance check that your application can act on: pass the response through to the user, re-prompt the generator with corrective instructions, or escalate for human review. This separation keeps citation enforcement decoupled from answer generation, making it easier to tune thresholds, swap models, or add new citation rules without touching the core RAG logic.

Wire the prompt into an async post-generation hook in your orchestration layer. The hook receives three inputs: the candidate assistant response, the list of retrieved source chunks with their identifiers, and the session's citation policy contract. Call the enforcement prompt with a fast, cost-efficient model (such as GPT-4o-mini or Claude Haiku) since the task is structured classification, not generation. Parse the JSON output and check the compliance_status field. If compliant, release the response. If non_compliant with corrective_re_prompt populated, feed that re-prompt back to your generator model for a single correction attempt. If non_compliant after the retry, or if the initial check returns requires_human_review, route to a review queue. Log every enforcement decision—including the original response, sources, policy version, compliance verdict, and any corrective re-prompts—for auditability and prompt iteration. This log becomes your dataset for tuning citation thresholds and detecting systemic failure patterns.

Validation and failure handling must be explicit. Validate that the enforcement prompt's JSON output parses correctly and contains the required fields (compliance_status, violations, corrective_re_prompt). If parsing fails, retry once with a stricter output format instruction. If the retry also fails, treat it as a requires_human_review escalation. Common production failure modes include: the enforcement model hallucinating violations for correctly cited claims, missing violations when sources are long or dense, and producing corrective re-prompts that are too vague to fix the issue. Mitigate these by setting a confidence threshold—if the enforcement model's violations array is empty but the response contains factual claims without inline citations, flag for review anyway. For high-stakes domains, run a periodic spot-check eval where a human reviewer compares enforcement decisions against ground truth for a sample of turns, and use discrepancies to refine the citation policy contract or adjust escalation thresholds.

Model choice and latency budgeting are critical because this prompt adds a synchronous dependency in your response pipeline. Target under 500ms for the enforcement check by using a small, fast model and keeping the input context tight—send only the candidate response and the specific source chunks that were used, not the entire retrieval set. If latency is a hard constraint, consider running enforcement as a near-real-time async check that flags violations after the response is sent, with a mechanism to issue a correction follow-up message. For RAG systems using streaming responses, buffer the complete response before running enforcement, or implement a sentence-level streaming check that accumulates citations and flags gaps mid-stream. Avoid the temptation to skip enforcement on low-risk queries—citation fatigue is most likely to appear exactly when the system has been running smoothly for many turns and operators stop watching.