Model Hallucination Detection and Escalation Prompt

This prompt is designed for AI reliability engineers and operations teams who need to systematically detect and escalate model hallucinations in production outputs. The primary job-to-be-done is comparing a model's generated claims against a provided ground-truth context or a set of known facts, then producing a structured severity score and a list of specific, unsupported claims that require human review. The ideal user is someone monitoring an AI pipeline—such as a RAG system, a summarization service, or a customer-facing chatbot—who needs an automated, auditable signal to trigger a human-in-the-loop intervention before a hallucinated output reaches an end user or is written to a system of record.

Use this prompt when you have a reliable source of truth available at inference time, such as a retrieved document chunk, a database record, or a verified knowledge base entry. It is most effective as a post-generation guardrail: the model first generates a response, and then this prompt acts as a critic, comparing the response to the evidence. The output is not a corrected response but a structured escalation payload containing a severity_score (e.g., LOW, MEDIUM, HIGH, CRITICAL), a list of unsupported_claims with direct quotes from the output and the conflicting or missing evidence, and a recommended_action (e.g., PUBLISH, FLAG_FOR_REVIEW, BLOCK). This design keeps the verification step separate from generation, making the hallucination detection logic easier to test, version, and calibrate independently.

Do not use this prompt as a substitute for source-grounding in the initial generation step. If your primary model is already instructed to answer only from provided context, this prompt serves as a second-layer audit, not the first line of defense. Avoid using it on creative or open-ended tasks where there is no single factual ground truth, as it will over-flag valid stylistic or inferential output. The biggest production risk is over-escalation: flagging accurate but paraphrased statements as hallucinations. To mitigate this, you must calibrate the prompt with clear rules distinguishing semantic equivalence from contradiction, and you should pair it with an evaluation harness that measures the false-positive rate on a golden dataset of known-good outputs. The next section provides the exact prompt template you can adapt and deploy.

This prompt template is designed to be dropped into a hallucination detection and escalation workflow. It compares a model-generated claim against a provided ground-truth context and produces a structured output that includes a severity score, a list of unsupported claims, and a clear escalation decision. The template uses square-bracket placeholders for all dynamic inputs, making it straightforward to parameterize in application code. Before integrating this prompt, ensure you have a reliable source of ground-truth context—this prompt is only as effective as the evidence it can verify against. For high-stakes domains like healthcare or finance, always route CRITICAL severity findings to a human review queue rather than taking automated action.

textCopy
You are an AI reliability auditor. Your task is to compare a model-generated CLAIM against a provided GROUND_TRUTH_CONTEXT and determine whether the claim is fully supported, partially supported, or unsupported (hallucinated).

## INPUTS

**GROUND_TRUTH_CONTEXT:**
[GROUND_TRUTH_CONTEXT]

**MODEL_CLAIM:**
[MODEL_CLAIM]

**OUTPUT_SCHEMA:**
[OUTPUT_SCHEMA]

**CONSTRAINTS:**
[CONSTRAINTS]

## INSTRUCTIONS

1. Extract every factual assertion from the MODEL_CLAIM.
2. For each assertion, check whether it is explicitly supported by the GROUND_TRUTH_CONTEXT.
3. Classify each assertion as:
   - `SUPPORTED`: Directly stated in the context.
   - `PARTIALLY_SUPPORTED`: Partially stated but missing key details or contains minor inaccuracies.
   - `UNSUPPORTED`: Not stated in the context or directly contradicted by it.
4. Assign an overall `severity_score` from 1 to 5:
   - 1: All claims fully supported.
   - 2: Minor unsupported details, no material impact.
   - 3: One or more partially supported claims that could mislead.
   - 4: At least one fully unsupported claim with potential harm.
   - 5: Multiple unsupported claims or a single critically false claim.
5. If severity_score is 4 or 5, set `escalation_required` to `true` and populate the `escalation_payload`.
6. If the MODEL_CLAIM is creative, subjective, or opinion-based rather than factual, note this in `notes` and do not flag as hallucination unless it contradicts explicit context.

## OUTPUT FORMAT

Respond with a single JSON object matching the OUTPUT_SCHEMA exactly. Do not include any text outside the JSON.

To adapt this template for your system, replace the placeholders with your specific inputs. The [GROUND_TRUTH_CONTEXT] should contain the verified source material—this could be a retrieved document chunk, a database record, or a human-verified fact set. The [MODEL_CLAIM] is the output you're auditing, typically from a prior model call. The [OUTPUT_SCHEMA] placeholder should be replaced with your exact expected JSON schema, including fields like assertions (array of objects with claim_text, support_status, and evidence_excerpt), severity_score (integer 1-5), escalation_required (boolean), and escalation_payload (object with reason, unsupported_claims, and recommended_action). The [CONSTRAINTS] placeholder lets you inject domain-specific rules, such as "do not flag marketing language as hallucination" or "treat any medical dosage discrepancy as severity 5." After adapting the template, run it against a golden dataset of known hallucinations and accurate outputs to calibrate your severity thresholds before production deployment.

This prompt is designed to sit inside a post-generation monitoring step, not as a real-time guardrail blocking every model response. In a typical deployment, every output from a primary generative model passes through an asynchronous evaluation queue. The hallucination detection prompt receives the original user query, the retrieved or provided context, and the model's final response. It returns a structured JSON payload with a severity score, a list of unsupported claims, and a recommended action. The application layer reads the action field: pass responses are logged and forwarded; flag responses are routed to a human review queue; block responses may trigger an automatic retry or fallback response before human review.

Validation and retry logic is critical because the detection prompt itself can produce malformed JSON or hallucinate about the presence of hallucinations. Implement a JSON schema validator that checks for required fields (severity_score, unsupported_claims, action, reasoning). If validation fails, retry up to two times with the same input but append the validation error message to the prompt as [PREVIOUS_VALIDATION_ERROR]. If the third attempt still fails validation, escalate the entire evaluation attempt to a human reviewer with the raw model output and detection prompt response attached. Log every validation failure for later analysis—frequent failures may indicate prompt drift or a need to switch to a more reliable model for the detection step.

Model selection matters more here than in many other prompts. The detection prompt requires careful reading comprehension and restraint—it must not flag creative but factually-grounded statements as hallucinations. GPT-4o and Claude 3.5 Sonnet perform well on this task, while smaller or faster models tend to over-flag. If latency is critical, use a fast model for the primary generation and a slower, more capable model for the detection step. For high-volume pipelines, consider batching detection requests and running them on a separate compute budget. Always log the model version alongside each evaluation result so that changes in detection behavior can be correlated with model updates.

Human review integration should present reviewers with a clean diff view: the original context on the left, the model's claims in the center, and the detection prompt's flagged items on the right. Include the severity score and reasoning. Reviewers should be able to confirm, dismiss, or reclassify each flagged claim. Track reviewer decisions over time to measure the detection prompt's precision and recall. If more than 30% of flagged claims are dismissed by reviewers, recalibrate the prompt's threshold language or adjust the severity score boundaries. For regulated domains, require a second reviewer for any block action before the response is permanently discarded or the user is notified.

Avoid wiring this prompt as a synchronous blocker on every user-facing response unless your latency budget can absorb an additional model call and your false-positive rate is well-characterized. A better pattern for user-facing applications is to let responses through but flag them for asynchronous review, with the ability to retroactively correct or retract responses if hallucinations are confirmed. For internal analytics or content generation pipelines where downstream decisions depend on factual accuracy, synchronous blocking with a fast fallback (e.g., 'I couldn't verify that claim') is appropriate. In all cases, instrument the pipeline with metrics: detection latency, validation failure rate, flag rate, reviewer confirmation rate, and time-to-review. These metrics will tell you whether the harness is working before a user complaint does.