Entity Extraction with Source Attribution Prompt

This prompt is designed for regulated-domain RAG systems where every entity mapping must be traceable to a source of truth. It extracts domain-specific entities from a user query and attributes each mapping to a specific glossary entry, taxonomy node, or canonical record. Use this when your retrieval pipeline requires auditability, compliance reviewers need to verify entity resolution, or downstream systems consume entity mappings that must be defensible. The core job-to-be-done is producing a structured, evidence-backed mapping that answers the question: 'Why did the system resolve this term to this entity?' without requiring a human to reverse-engineer the model's reasoning.

This is not a general entity extraction prompt. It assumes you have a curated source glossary or taxonomy and that fabricated mappings are unacceptable. If your application needs fast, high-recall extraction without source grounding—for example, a fuzzy search autocomplete that tolerates occasional mismatches—use a simpler extraction prompt and add a separate hallucination check later. Similarly, if your domain lacks a maintained glossary and you expect the model to infer canonical terms from its training data, this prompt is the wrong tool. The required inputs are a user query and a structured glossary or taxonomy provided in the prompt context. Without that curated reference, source attribution collapses into guesswork, and the audit trail becomes meaningless.

Before adopting this prompt, verify that your retrieval architecture can consume attributed entity mappings. Downstream systems must be able to log the source citation alongside each resolved entity, and your evaluation framework must include checks for hallucinated attributions—mappings that point to a glossary entry that doesn't exist or misrepresent what the entry actually says. If your pipeline treats entity resolution as a black box, start by instrumenting it for traceability. Then use this prompt as the resolution step, paired with a validation harness that flags unattributed or fabricated mappings before they reach retrieval.

This prompt template is designed for regulated domain RAG systems where every entity mapping must be traceable to an authoritative source. Unlike a simple entity extraction prompt, this template requires the model to cite the exact glossary entry, taxonomy path, or canonical record that justifies each mapping. The output includes both the extracted entity and its source attribution, creating an audit trail that downstream systems and human reviewers can verify. Use this prompt when your retrieval pipeline must prove that a query term was correctly mapped to an internal controlled vocabulary before search execution.

textCopy
You are an entity extraction system operating in a regulated domain. Your task is to extract domain-specific entities from a user query and attribute each mapping to a specific source entry in the provided glossary, taxonomy, or canonical record set. Every entity you return must be traceable to an explicit source. Do not infer entities that are not supported by the provided reference material.

## INPUT
User Query: [USER_QUERY]

## REFERENCE SOURCES
Below are the authoritative sources you may use for entity mapping. You must cite the exact source ID for each mapping.

Glossary:
[GLOSSARY_ENTRIES]

Taxonomy:
[TAXONOMY_NODES]

Canonical Records:
[CANONICAL_RECORDS]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "query": "[USER_QUERY]",
  "extracted_entities": [
    {
      "mention": "exact text from query",
      "entity_type": "[ENTITY_TYPE]",
      "mapped_term": "canonical term or identifier",
      "source_attribution": {
        "source_type": "glossary | taxonomy | canonical_record",
        "source_id": "exact identifier from the reference source",
        "source_entry": "the full entry text or path that supports this mapping",
        "match_type": "exact | synonym | broader | narrower | related"
      },
      "confidence": "high | medium | low"
    }
  ],
  "unmapped_mentions": [
    {
      "mention": "exact text from query",
      "reason": "why no source mapping was found"
    }
  ]
}

## CONSTRAINTS
1. Only map entities that appear in the provided reference sources. If a query term has no matching entry, list it under unmapped_mentions.
2. For each mapping, cite the exact source_id as it appears in the reference material. Do not fabricate identifiers.
3. If multiple source entries could match a mention, return the best match and note alternatives in a disambiguation_notes field.
4. Set confidence to "high" only when the mention and source entry match exactly or through a documented synonym. Use "medium" for broader/narrower matches. Use "low" for inferred matches with weak evidence.
5. Do not expand acronyms unless the expansion is explicitly listed in the provided reference sources.
6. Preserve the original mention text exactly as it appears in the query.

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## RISK LEVEL
[RISK_LEVEL: high | medium | low]

If RISK_LEVEL is "high", include a review_required flag for each entity and add a human_review_notes field explaining what a reviewer should verify.

To adapt this template for your system, replace the square-bracket placeholders with your actual data. The [GLOSSARY_ENTRIES], [TAXONOMY_NODES], and [CANONICAL_RECORDS] placeholders should contain your domain's authoritative reference material, formatted as structured entries with clear identifiers. The [FEW_SHOT_EXAMPLES] placeholder should include 2-4 examples showing correct mappings with source attributions, including at least one example of an unmapped mention and one example of a medium-confidence match. The [RISK_LEVEL] placeholder controls whether the output includes human review flags; set it to "high" for clinical, legal, or financial domains where incorrect entity mappings could cause harm. Before deploying, validate the prompt against a golden dataset of queries with known entity mappings and verify that source_ids in the output match your reference material exactly. Common failure modes include the model fabricating plausible-looking source identifiers, mapping entities to sources that contain related but not matching terms, and failing to populate the unmapped_mentions array when no match exists.

The Entity Extraction with Source Attribution prompt is designed to sit between the user's raw query and your retrieval system. In a production RAG pipeline, this prompt is called synchronously before any vector or keyword search executes. The application sends the user query along with a domain glossary or taxonomy snippet as [CONTEXT], receives a structured JSON array of entity mappings with source attributions, and then uses those canonical terms and IDs to construct precise retrieval queries. This is not a conversational prompt—it should be treated as a deterministic extraction step with strict output validation before the results are allowed to influence downstream retrieval.

Wire the prompt into your application with a pre-retrieval hook that: (1) loads the relevant glossary or taxonomy slice based on the user's session context, tenant, or domain; (2) constructs the prompt with [QUERY] and [GLOSSARY_OR_TAXONOMY] placeholders populated; (3) calls the model with temperature=0 and a low top_p value to maximize deterministic extraction; (4) validates the output JSON against a schema that requires entity, canonical_term, source_entry, and confidence fields per extracted entity; (5) rejects any mapping where the source_entry does not appear verbatim in the provided glossary—this is your primary hallucination guard. For high-compliance domains, add a human review queue for any extraction where confidence falls below a configurable threshold or where the model returns entities not found in the glossary. Log every extraction with the original query, the glossary version used, the full model response, and the validation result for audit trails.

On model choice, prefer models with strong instruction-following and JSON mode support. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro all perform well on this task when given a clear output schema. For latency-sensitive pipelines, consider a smaller fine-tuned model trained on your specific glossary and entity types—this reduces both cost and the risk of hallucinated source entries. Implement retry logic with a maximum of two retries: if the first response fails schema validation, send the validation errors back in a retry prompt asking the model to correct only the invalid fields. If the second attempt still fails, fall back to using the original user query without entity mapping and flag the interaction for glossary gap analysis. Never silently drop failed extractions—they are signals that your glossary needs updating.

For evaluation, build a golden dataset of 100-200 queries with hand-labeled entity mappings and source attributions. Run this prompt against the dataset and measure: (1) entity recall—did the prompt find all entities that should be mapped? (2) mapping precision—were any terms mapped to the wrong canonical entry? (3) source attribution accuracy—does every source_entry exactly match a glossary entry? (4) hallucination rate—what percentage of responses contain fabricated source entries? Track these metrics per glossary version and per model version. When you update your domain glossary, re-run the eval suite before deploying the new glossary to production. This prompt's value depends entirely on the quality and coverage of the glossary you provide—a sparse or outdated glossary will produce low-confidence mappings and increase the hallucination risk. Invest in glossary maintenance as a first-class engineering task, not an afterthought.