Hallucinated Field Detection Assertion Prompt

This prompt is a precision instrument for QA leads and platform engineers who need to programmatically verify that a structured LLM output is factually anchored in a provided source document. The core job-to-be-done is not general fact-checking; it is a targeted, field-level audit that compares every value in a JSON or structured output against a single, authoritative [SOURCE_CONTEXT]. It detects fabricated names, invented statistics, unsupported dates, and claims that sound plausible but have no evidence in the source material. Use it as a deterministic gate in a CI/CD pipeline, a post-processing assertion in a RAG system, or a quality checkpoint before extracted data is ingested into a database.

The ideal user is an engineer who already has both the [SOURCE_CONTEXT] and the [TARGET_OUTPUT] in hand. This prompt assumes you are not asking the model to search the web or use external knowledge; it operates in a strict 'closed-book' mode where the source context is the only acceptable evidence. A typical implementation involves feeding a JSON object with field-value pairs and receiving a structured grounding report that flags each field as supported, unsupported, or contradicted, along with the specific evidence fragment or a null evidence marker. This is particularly valuable in regulated workflows where an unsupported claim in a generated summary or an extracted record can have compliance consequences.

Do not use this prompt when you need to evaluate the quality, style, or helpfulness of an output. It is not a general evaluator or a substitute for a human review of tone and nuance. It is also not a replacement for a retrieval system's own confidence scores; it is a post-hoc assertion on the final payload. If your source context is too large for a single context window, you must chunk it and run the assertion per chunk, or use a retrieval step to pre-select the most relevant evidence before invoking this prompt. In high-risk domains like healthcare or finance, always route unsupported findings to a human review queue rather than auto-rejecting or auto-correcting the output.

This template is designed to be pasted directly into your assertion harness. It instructs the model to act as a strict, evidence-based auditor. The prompt takes a structured output and the source context that was supposed to ground it, then produces a field-by-field report. Each field is classified as GROUNDED, HALLUCINATED, or UNCERTAIN, with a direct evidence quote or a clear explanation of the gap. This is not a general-purpose fact-checker; it is a targeted tool for catching fabricated entities, invented numbers, and unsupported claims in structured data extracted by another LLM call.

textCopy
You are an output validation auditor. Your task is to compare each field in the [OUTPUT_SCHEMA] of a generated response against the provided [SOURCE_CONTEXT]. You must produce a strict, field-level grounding report.

## INPUT
**Source Context:**

[SOURCE_CONTEXT]

codeCopy

**Generated Output:**
```json
[GENERATED_OUTPUT]

OUTPUT_SCHEMA

Your response must be a valid JSON object with a single key "field_checks" containing an array of objects. Each object must have the following keys:

• "field_path": The JSON path to the field being checked (e.g., $.user.name).
• "field_value": The exact value from the generated output.
• "verdict": One of "GROUNDED", "HALLUCINATED", or "UNCERTAIN".
• "evidence": If GROUNDED, a direct quote from the source context. If HALLUCINATED, a statement that the value is not present in the source. If UNCERTAIN, explain the ambiguity.

CONSTRAINTS

• Do not infer or assume any information not explicitly present in the [SOURCE_CONTEXT].
• A field is HALLUCINATED if its value is not directly supported by the source, even if it seems plausible.
• A field is UNCERTAIN if the source implies the information but does not state it explicitly.
• If the [GENERATED_OUTPUT] is not valid JSON, return a single error object: {"error": "Invalid JSON input", "detail": "..."}.
• Do not add any text outside the JSON response.

To adapt this template, replace the placeholders with your specific data. [SOURCE_CONTEXT] should contain the raw text the original LLM was supposed to use. [GENERATED_OUTPUT] is the structured JSON you are validating. The [OUTPUT_SCHEMA] description can be a simple list of field paths to check or a full JSON Schema, depending on how you want to scope the audit. For high-risk domains like healthcare or finance, always route HALLUCINATED verdicts to a human review queue before taking any action. You can harden this further by adding a [RISK_LEVEL] parameter that adjusts the strictness of the UNCERTAIN classification.

This prompt is designed to be called programmatically as a validation gate, not as a one-off chat interaction. The typical integration point is immediately after an LLM generates a structured output that must be grounded in source context. You pass the original source context, the generated output, and a field-level schema into this assertion prompt, and it returns a structured grounding report. The report is machine-readable and should be parsed to decide whether to accept, flag, or reject the upstream output.

In a CI/CD pipeline, wrap this prompt in a test harness that calls your primary generation endpoint, collects the output, and then calls this assertion prompt with the same source context. Parse the JSON response and check the overall_verdict field. If overall_verdict is FAIL, block the pipeline or route the output for human review. For production systems, implement a retry loop: if the assertion fails, feed the violation details back to the generation model as correction instructions, regenerate, and re-assert. Limit retries to 3 attempts before escalating. Log every assertion result—including the source context hash, output hash, verdict, and per-field evidence gaps—for auditability and drift detection over time.

Model choice matters here. Use a model with strong instruction-following and JSON output reliability, such as gpt-4o or claude-3-5-sonnet. Avoid smaller or older models that may hallucinate the assertion itself. Set temperature=0 to maximize determinism. Validate the assertion output against a JSON Schema before trusting the verdict; if the assertion prompt itself returns malformed JSON, treat it as an infrastructure failure and escalate. For high-stakes domains like healthcare or finance, never auto-accept a PASS verdict without sampling a percentage of assertions for human review. The assertion prompt reduces hallucination risk but does not eliminate it—treat it as a strong filter, not a guarantee.