Hallucinated Field Detection and Removal Prompt

This playbook is for teams running extraction pipelines where models invent plausible but unsourced fields. When a model is asked to extract structured data from a document, it may add fields that sound correct but have no basis in the source text. This prompt uses few-shot counterexamples showing hallucinated versus grounded output pairs to teach field-level skepticism. Use it as a post-extraction repair step or as a guardrail before extracted data enters a database, CRM, or downstream system. It is not a replacement for a well-designed extraction schema, but it catches the most common class of fabrication errors that pass basic format validation.

Deploy this prompt when your extraction pipeline produces structured outputs that pass schema validation but contain fields that cannot be traced back to the source document. Common triggers include: models adding plausible dates, names, or amounts to incomplete records; filling in missing enum values with likely defaults; or expanding abbreviations into full terms without evidence. The prompt works best when paired with a source document reference and an extraction schema that defines which fields are required versus optional. For high-stakes domains like healthcare, legal, or finance, always route flagged outputs to human review rather than auto-deleting fields, as over-aggressive removal can drop critical information that was merely phrased differently than expected.

Do not use this prompt as your primary extraction instruction. It is a repair and guardrail step, not a replacement for clear schema definitions, explicit null-handling rules, and source-grounding instructions in your main extraction prompt. Also avoid using it when the extraction task requires inferential reasoning that legitimately goes beyond the literal source text—for example, sentiment analysis, risk scoring, or summarization tasks where the output is expected to synthesize rather than transcribe. In those cases, field-level hallucination detection will produce false positives by flagging valid inferences as unsourced. Start with this prompt after you have a working extraction pipeline and need to reduce the fabrication rate before data reaches production systems.

This prompt template is designed to be pasted directly into your system. It uses few-shot counterexamples to teach the model the difference between grounded fields (derived from the source text) and hallucinated fields (plausible but unsupported inventions). The counterexamples are the critical teaching mechanism; keep them domain-specific and update them when you observe new hallucination patterns in production. Replace the square-bracket placeholders with your actual source text, extraction schema, and domain-relevant counterexamples before use.

textCopy
You are an extraction auditor. Your job is to review a structured output and remove any field whose value cannot be directly supported by the source text. A field is hallucinated if it invents a fact, name, date, number, or relationship that does not appear in the source.

Do not remove a field just because the source uses different wording. If the information is present and can be clearly inferred without adding new facts, keep it. If you are unsure, flag the field as [UNCERTAIN] instead of removing it.

---

SOURCE TEXT:
[SOURCE_TEXT]

EXTRACTED OUTPUT TO AUDIT:
[EXTRACTED_OUTPUT]

---

COUNTEREXAMPLES:

Example 1 - Hallucinated Field (REMOVE):
Source: "Acme Corp reported Q3 revenue of $2.1 million."
Extracted: {"company_name": "Acme Corp", "quarter": "Q3", "revenue": 2100000, "ceo": "Jane Smith"}
Audit Decision: Remove "ceo". The source never mentions a CEO.

Example 2 - Grounded Field (KEEP):
Source: "The defendant, a 34-year-old engineer, was cited for speeding."
Extracted: {"age": 34, "occupation": "engineer", "charge": "speeding", "gender": "male"}
Audit Decision: Remove "gender". The source does not state the defendant's gender. "Engineer" does not imply gender.

Example 3 - Uncertain Field (FLAG):
Source: "The server crashed at 03:00 UTC."
Extracted: {"event": "server crash", "timestamp": "03:00 UTC", "root_cause": "memory leak"}
Audit Decision: Flag "root_cause" as [UNCERTAIN]. The source states a crash occurred but does not state the cause.

---

TASK:
Review the extracted output above. For each field, decide whether it is GROUNDED (directly supported by the source), HALLUCINATED (invents unsupported information), or UNCERTAIN (plausible but not confirmed).

Return a JSON object with the following structure:
{
  "audit_result": {
    "grounded_fields": { ... },
    "removed_fields": { "field_name": "reason for removal" },
    "uncertain_fields": { "field_name": "reason for uncertainty" }
  },
  "cleaned_output": { ... },
  "audit_notes": "Brief explanation of decisions"
}

The "cleaned_output" must contain only grounded and uncertain fields. Hallucinated fields must be excluded entirely.

CONSTRAINTS:
- Do not add any fields that were not in the original extracted output.
- Do not modify field values unless correcting an obvious extraction error that is supported by the source.
- If the entire output appears fabricated, return an empty "cleaned_output" and explain in "audit_notes".

To adapt this prompt for your domain, replace the counterexamples with real hallucination patterns you've observed in production. The three examples above cover removal, keeping, and flagging—maintain this structure. Update the [SOURCE_TEXT] and [EXTRACTED_OUTPUT] placeholders with your actual data. If your schema is deeply nested, add a counterexample showing nested field removal. For high-stakes domains like healthcare or finance, add a fourth counterexample demonstrating when to escalate to human review rather than making an automated removal decision. Test the prompt against a golden set of known hallucinated outputs before deploying, and monitor the uncertain_fields rate in production—a rising rate may indicate source quality issues or schema drift.

This prompt is designed to sit between your initial extraction step and downstream consumers of structured data. After the model produces a JSON object from source text, pass both the raw output and the source evidence into this detection prompt. The prompt's job is to return a cleaned object with hallucinated fields removed and a removed_fields audit trail. Do not use this prompt as the primary extractor—it is a post-extraction safety filter that assumes the input is already structured JSON.

Wire the prompt into a validation pipeline with these stages: (1) Run the primary extraction prompt and parse the JSON. (2) If parsing fails, route to a JSON repair prompt first. (3) Pass the valid JSON and the original source text into this hallucination detection prompt. (4) Parse the cleaned output and compare removed_fields against the original object. (5) If removed_fields is non-empty, log the full audit record including source text, original output, cleaned output, and removal reasons. (6) Route outputs with high-confidence removals in critical fields to a human review queue before ingestion. For low-risk fields, you can auto-accept the cleaned output but still log for monitoring.

Model choice matters. This prompt relies on the model's ability to compare structured claims against unstructured evidence. Use a model with strong instruction-following and long-context handling, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that struggle with multi-field comparison tasks—they tend to either over-remove (killing correct fields) or under-remove (missing subtle hallucinations). If latency is critical, consider running this check asynchronously after the primary extraction returns to the user, flagging corrections for later reconciliation rather than blocking the response.

Retry logic should be minimal. If the detection prompt itself returns malformed JSON or fails to include the removed_fields key, retry once with a stricter output format instruction appended. If the second attempt fails, log the failure and fall back to the original extraction output with a hallucination_check_failed flag. Never silently drop data because the safety check itself broke. For high-stakes pipelines (healthcare, legal, finance), require human review on any output where the hallucination check could not complete successfully.

Evaluation and monitoring are essential. Track precision (did we remove genuinely hallucinated fields?) and recall (did we catch all hallucinations?) on a golden test set of source-output pairs with known hallucinated fields. Common failure modes include: the model removing correct but surprising fields (low precision), missing hallucinations that paraphrase source text (low recall), and inconsistent removal decisions across similar inputs. Run this eval weekly and whenever you change the primary extraction prompt, the source document type, or the model version. If precision drops below 90% or recall below 80%, escalate to human review for all outputs until the prompt is tuned.