Hallucination Root Cause Classification Prompt Template

This prompt is for AI observability and debugging teams who need to move beyond detecting hallucinations to understanding why they occur. When a hallucination has already been flagged by a detection guardrail, this prompt classifies the root cause into one of six actionable categories: insufficient context, source contradiction, outdated evidence, over-generalization, prompt ambiguity, or model knowledge intrusion. The output feeds directly into prompt improvement, retrieval tuning, and model selection loops. Use this prompt after a hallucination is detected but before you decide how to fix it.

The ideal user is an engineer or AI operator who has a specific, flagged hallucination in hand and needs to route the fix to the right team or system. Required inputs include the original user query, the retrieved context or evidence provided to the model, the full model output containing the hallucination, and the specific hallucinated span that was flagged. Without all four inputs, the classification will be unreliable. This prompt is not a real-time safety guardrail—it is a post-detection diagnostic tool that assumes a hallucination has already been confirmed by a separate detection step.

Do not use this prompt for real-time hallucination detection, for end-user-facing responses, or as a substitute for a grounding verification step. It will not tell you if a hallucination exists; it will only tell you why one that has already been found likely occurred. Avoid using this prompt when the hallucination is ambiguous or when multiple root causes are equally plausible without additional evidence—in those cases, log the ambiguity and escalate for human review rather than forcing a single classification.

This prompt template is the core diagnostic tool for your hallucination debugging loop. It takes a flagged hallucination—a specific claim that has already been identified as unsupported or contradictory—and forces a structured classification of why it occurred. The output is not a yes/no flag but a root cause label with supporting evidence, which feeds directly into prompt improvement, retrieval tuning, and model behavior analysis. Use this when you have a confirmed hallucination and need to decide whether to fix the retrieval pipeline, clarify the system prompt, update stale data, or adjust model temperature.

codeCopy
SYSTEM:
You are an AI observability classifier. Your job is to analyze a detected hallucination and determine its most likely root cause. You will receive the full model output, the specific hallucinated claim, the provided source context, and the original user query. Classify the hallucination into exactly one of the following categories and provide a concise evidence-based justification.

CLASSIFICATION CATEGORIES:
- INSUFFICIENT_CONTEXT: The source context lacked the necessary information to support the claim. The model should have abstained or expressed uncertainty.
- SOURCE_CONTRADICTION: The claim directly contradicts information present in the provided source context.
- OUTDATED_EVIDENCE: The source context contained information that was once correct but is now stale or superseded, leading to a factually incorrect claim.
- OVER_GENERALIZATION: The model extrapolated beyond what the evidence supports, applying a general rule or pattern to a specific case without grounding.
- PROMPT_AMBIGUITY: The user query or system instructions were ambiguous, leading the model to make an incorrect assumption about what was being asked or how to respond.
- MODEL_KNOWLEDGE_INTRUSION: The claim appears to come from the model's pre-training data rather than the provided source context, overriding the grounded evidence.
- PARAPHRASE_DRIFT: The model's rephrasing of source material introduced a meaning change, exaggeration, or omission that altered the factual content.

INPUTS:
- User Query: [USER_QUERY]
- Full Model Output: [FULL_OUTPUT]
- Hallucinated Claim: [HALLUCINATED_CLAIM]
- Source Context Provided: [SOURCE_CONTEXT]
- Hallucination Detection Method: [DETECTION_METHOD]

OUTPUT_SCHEMA:
{
  "root_cause": "INSUFFICIENT_CONTEXT | SOURCE_CONTRADICTION | OUTDATED_EVIDENCE | OVER_GENERALIZATION | PROMPT_AMBIGUITY | MODEL_KNOWLEDGE_INTRUSION | PARAPHRASE_DRIFT",
  "confidence": 0.0-1.0,
  "evidence_for_classification": "Quote the specific parts of the source context, output, or query that support this classification.",
  "alternative_causes_considered": ["List other categories briefly considered and why they were ruled out."],
  "recommended_action": "One sentence recommending the next step: fix retrieval, update context, clarify prompt, adjust model params, or escalate for human review."
}

CONSTRAINTS:
- Classify only the provided hallucinated claim, not other parts of the output.
- If multiple categories could apply, choose the most direct and actionable cause.
- Base your classification on the evidence provided, not on speculation about model internals.
- If the source context is empty or clearly insufficient, default to INSUFFICIENT_CONTEXT.

Adapt this template by adjusting the classification categories to match your specific failure taxonomy. If your system frequently encounters a particular failure mode—such as entity confusion or temporal misalignment—add a dedicated category. The output schema is designed for direct ingestion into your observability database; keep the JSON structure stable so that dashboards and alerting rules can query root cause distributions over time. For high-risk domains, add a [RISK_LEVEL] field to the output schema and route HIGH classifications to a human review queue before the recommended action is applied.

This prompt is designed to operate as a post-detection analysis step inside an AI observability or debugging pipeline. It assumes a hallucination has already been flagged by an upstream detector—such as the Unsupported Statement Detection or Source-Output Contradiction Scanner prompts in this pillar. The harness receives the flagged output span, the source evidence set, and the original user query or instruction. Its job is to produce a structured root cause classification that feeds directly into prompt improvement, retrieval tuning, or model behavior debugging workflows. Do not use this prompt as a real-time guardrail; it is an offline or async diagnostic tool that runs after a hallucination event is captured.

Wire the prompt into your application with a validation layer that enforces the expected output schema before the classification is accepted. The prompt returns a JSON object with root_cause (a controlled enum: insufficient_context, source_contradiction, outdated_evidence, over_generalization, prompt_ambiguity, model_knowledge_intrusion), a confidence score, and an evidence_rationale string. Your harness should validate that root_cause matches the allowed enum, that confidence is a float between 0.0 and 1.0, and that evidence_rationale is non-empty. On validation failure, implement a single retry with the validation error message appended to the prompt as additional context. If the retry also fails, log the raw output and escalate for human review rather than silently accepting an invalid classification. For model choice, use a capable instruction-following model (GPT-4, Claude 3.5, or equivalent) rather than a smaller or faster model—root cause analysis requires nuanced reasoning that weaker models frequently get wrong. Set temperature=0 for deterministic outputs.

Integrate the classification results into a feedback database that tracks hallucination patterns over time. Each classification should be stored alongside the prompt version, retrieval configuration, model ID, and timestamp. This lets you run aggregate queries: 'Are 60% of our hallucinations caused by insufficient context?' or 'Did the new chunking strategy reduce source_contradiction classifications?' Build a dashboard or alert that surfaces clusters of the same root cause, which signals where to focus engineering effort. Avoid the trap of treating each classification in isolation—the value of this prompt comes from pattern detection across many incidents. For high-risk domains (medical, legal, financial), always require a human reviewer to confirm the root cause classification before it triggers a change to production prompts or retrieval pipelines. The model's classification is a strong signal, not a final decision.