Entity Hallucination Detection Prompt for Summaries

This prompt is designed for summarization pipeline operators who need to verify that a generated summary does not introduce fabricated people, organizations, locations, or products. The core job-to-be-done is automated factuality gating: before a summary reaches a user, database, or downstream system, you compare its named entities against a provided source document and receive a structured diff report. The ideal user is an engineering lead or ML engineer integrating this check into a post-generation validation harness for RAG systems, meeting intelligence tools, or document processing pipelines where entity-level accuracy is a hard requirement.

Use this prompt when you have a single source document and a single summary, and you need a machine-readable report of which entities are grounded, which are hallucinated, and which are partial matches. It is not suitable for multi-document synthesis without adaptation—you would need to provide a merged source context or run the check per source. It is also not a replacement for human review in high-risk domains like healthcare or legal; the output should be treated as a triage signal that flags potential fabrications for human verification. The prompt includes threshold controls for fuzzy matching, allowing you to tune how aggressively it flags near-matches as hallucinations.

Before wiring this into production, define your tolerance for false positives and false negatives. A strict threshold may flag legitimate paraphrases or alternate names as hallucinations; a loose threshold may let invented entities slip through. Start by running this prompt against a golden dataset of known-good and known-bad summaries to calibrate the threshold. The output schema is designed to be consumed by a validation harness that can block, flag, or route summaries based on the hallucination count and severity. Do not use this prompt as the sole defense in regulated workflows—always pair it with human review for high-severity flags.

The prompt below is designed to be dropped into a post-summarization validation step. It takes a source document and a generated summary as inputs, extracts all named entities from both, and produces a diff that identifies entities present in the summary but absent from the source. Use this template when you need automated hallucination detection before summaries reach end users, databases, or downstream systems. The prompt expects square-bracket placeholders to be filled by your application harness before each call.

textCopy
You are an entity validation auditor. Your task is to compare named entities in a summary against the source document and identify any entities the summary invented.

[INPUT]

SOURCE DOCUMENT:
"""
[SOURCE_TEXT]
"""

GENERATED SUMMARY:
"""
[SUMMARY_TEXT]
"""

[CONTEXT]

ENTITY TYPES TO CHECK: [ENTITY_TYPES]

PARTIAL MATCH THRESHOLD: [MATCH_THRESHOLD]

[OUTPUT_SCHEMA]

Return a JSON object with this exact structure:
{
  "source_entities": [
    {
      "entity_text": "string",
      "entity_type": "string",
      "first_occurrence_offset": number
    }
  ],
  "summary_entities": [
    {
      "entity_text": "string",
      "entity_type": "string",
      "first_occurrence_offset": number
    }
  ],
  "hallucinated_entities": [
    {
      "entity_text": "string",
      "entity_type": "string",
      "summary_offset": number,
      "closest_source_match": "string or null",
      "match_similarity": number,
      "confidence": "high" | "medium" | "low"
    }
  ],
  "partial_matches": [
    {
      "summary_entity": "string",
      "source_entity": "string",
      "similarity_score": number,
      "match_type": "fuzzy" | "alias" | "abbreviation"
    }
  ],
  "audit_summary": {
    "total_source_entities": number,
    "total_summary_entities": number,
    "hallucinated_count": number,
    "partial_match_count": number,
    "hallucination_rate": number
  }
}

[CONSTRAINTS]

1. Extract entities only of the types specified in [ENTITY_TYPES]. If [ENTITY_TYPES] is empty or "all", extract PERSON, ORGANIZATION, LOCATION, PRODUCT, DATE, MONEY, and EVENT entities.
2. For each entity in the summary, search for an exact or partial match in the source. Use the [MATCH_THRESHOLD] value (0.0 to 1.0) to determine when a partial match is close enough to consider grounded.
3. If [MATCH_THRESHOLD] is not provided, default to 0.85.
4. Flag any summary entity with no match above the threshold as hallucinated.
5. For hallucinated entities, set confidence to "high" if no source entity is remotely similar, "medium" if a weak partial match exists below threshold, and "low" if the entity type is ambiguous.
6. Include partial matches in the partial_matches array even when they fall below the threshold, so reviewers can assess borderline cases.
7. Do not flag entities that appear in the source under a different name if the match similarity exceeds the threshold.
8. Preserve all offsets as character positions from the start of the respective text.
9. If the summary contains no entities of the specified types, return empty arrays with audit_summary counts set to zero.
10. Do not invent entities. Only report what is present in the texts.

[EXAMPLES]

[FEW_SHOT_EXAMPLES]

[RISK_LEVEL]

Risk level: [RISK_LEVEL]

If risk level is "high", include a "requires_human_review" boolean field set to true in the audit_summary when hallucination_rate exceeds 0.1 or any hallucinated entity has confidence "high".

To adapt this template, replace each square-bracket placeholder with values from your application context. [SOURCE_TEXT] and [SUMMARY_TEXT] are the two required text inputs. [ENTITY_TYPES] accepts a comma-separated list like "PERSON, ORGANIZATION, LOCATION" or the keyword "all". [MATCH_THRESHOLD] controls how aggressively the prompt flags near-matches as grounded versus hallucinated—lower values are more permissive. [FEW_SHOT_EXAMPLES] should contain one or two example input-output pairs showing correct entity extraction and flagging behavior for your domain. [RISK_LEVEL] accepts "low", "medium", or "high" and gates whether the output includes a mandatory human-review flag. Wire this prompt into a post-summarization pipeline where the source document is always available, and never run it without providing both texts—the prompt cannot detect hallucinations from the summary alone.

This prompt is designed to operate as a post-generation validation step within a summarization pipeline. It should be called immediately after a summary is produced, receiving both the generated summary and the source document as inputs. The prompt's output—a structured diff report of matched, unmatched, and partial-match entities—should be parsed by your application to determine whether the summary passes quality gates or requires remediation. Do not treat this as a one-shot check; integrate it as a mandatory stage before any summary reaches a user-facing surface, database, or downstream system.

Wire the prompt into your application with a clear harness: define [SUMMARY] and [SOURCE_DOCUMENT] as required string inputs, and set [MATCH_THRESHOLD] to a float between 0.0 and 1.0 (start at 0.8 for strict matching). Parse the JSON output into a structured object with fields for matched_entities, unmatched_entities, and partial_match_entities. Implement a validation layer that rejects malformed JSON and triggers a retry with the same inputs. If unmatched entities exceed a configurable threshold (e.g., more than 2 invented entities or any invented entity in a high-risk domain), route the summary to a human review queue rather than auto-publishing. Log every detection result—including the summary ID, source document ID, entity counts, and the full diff—for audit trails and prompt performance monitoring over time.

For model selection, use a model with strong instruction-following and structured output capabilities. Avoid models prone to hallucination in their own right for this verification task. Set temperature to 0 or a very low value to maximize deterministic entity extraction. If your pipeline processes high-volume summaries, consider batching multiple summary-source pairs into a single call with clear delimiters, but test thoroughly for cross-contamination between batches. Implement a circuit breaker: if the detection prompt itself fails or returns invalid JSON three consecutive times, escalate to on-call engineering and halt the affected pipeline segment. Never allow a summary with unverified entity provenance to bypass this check silently—the cost of a hallucinated entity in a customer-facing summary far exceeds the latency of a verification step.