Cross-Lingual Entity Normalization Prompt Template

This prompt is for enterprise RAG teams that operate multilingual knowledge bases where the same real-world entity—a person, organization, product, or location—appears under different names, spellings, or translations across languages. The job-to-be-done is mapping a named entity mentioned in a user's source-language query to a single, language-independent canonical identifier before retrieval executes. Without this normalization step, a query for 'International Business Machines' in English might miss German documents that reference 'IBM' or Japanese documents that use 'アイ・ビー・エム,' causing retrieval gaps that no amount of synonym expansion or query translation can fix on its own.

The ideal user is a search engineer, RAG architect, or platform developer who already has a canonical entity store (a knowledge graph, entity resolution service, or authoritative ID system) and needs the LLM to perform the disambiguation and mapping step. Required context includes the source query text, the detected source language, a list of candidate entities extracted from the query, and access to entity metadata such as known aliases, language variants, and canonical IDs. This prompt is not a replacement for a production entity resolution system; it is a pre-retrieval normalization layer that reduces the problem space before the query hits your indexes. Do not use this prompt when you lack a canonical entity store to map into, when the query contains no named entities, or when your retrieval backend already performs fuzzy entity matching natively.

This prompt is also inappropriate for real-time, sub-50ms retrieval paths where an LLM call would add unacceptable latency. In those cases, precompute entity mappings offline or use a fast, rules-based alias dictionary. Additionally, if your entity store is incomplete or untrusted, the prompt will produce mappings that look confident but are wrong—always validate the canonical ID against your ground-truth system after the LLM returns. For high-stakes domains like legal or healthcare, require human review of any entity mapping where the LLM's confidence is below a defined threshold or where multiple candidate entities have similar scores. The next section provides the copy-ready prompt template you can adapt and wire into your retrieval pipeline.

This prompt template normalizes named entities—people, organizations, products, locations, and domain-specific terms—found in a user's query into their canonical identifiers. The goal is to bridge the gap between how users refer to entities in their own language and how those entities are indexed in a multilingual knowledge base. The prompt expects a source query, a target retrieval language, and access to a canonical entity catalog (provided inline or via a tool). It outputs a structured mapping that can be fed directly into a retrieval system's filter or query rewriter.

textCopy
You are an entity normalization engine for a multilingual retrieval system. Your task is to identify named entities in a user query and map them to their canonical, language-independent identifiers.

## INPUT
User Query: [USER_QUERY]
Source Language: [SOURCE_LANGUAGE]
Target Retrieval Language: [TARGET_LANGUAGE]

## CANONICAL ENTITY CATALOG
[ENTITY_CATALOG]

## INSTRUCTIONS
1. Extract every named entity from the User Query. Entities include people, organizations, products, brands, locations, technical terms, and domain-specific concepts.
2. For each extracted entity, find its canonical identifier in the Entity Catalog. Match on:
   - Exact name in any language variant listed.
   - Known aliases, acronyms, or abbreviations.
   - Common translations or transliterations between the source and target languages.
3. If an entity is not found in the catalog, mark it as `unmatched` and provide the best-guess normalized form in the target language.
4. Do not invent canonical IDs. If the catalog does not contain a match, the `canonical_id` field must be `null`.

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "entities": [
    {
      "mention": "string (exact text from query)",
      "entity_type": "PER | ORG | PROD | LOC | TERM | OTHER",
      "canonical_id": "string | null",
      "canonical_name": "string (preferred name in target language)",
      "match_method": "exact | alias | translation | transliteration | guess | none",
      "confidence": 0.0-1.0
    }
  ],
  "unmatched_count": 0,
  "normalized_query_hint": "string (query with mentions replaced by canonical names in target language, for retrieval fallback)"
}

## CONSTRAINTS
- Preserve the original mention text exactly as it appears in the query.
- If multiple catalog entries are possible candidates, return the highest-confidence match and list alternatives in a `candidates` array per entity.
- Set confidence below 0.5 for transliteration-only or guess-based matches.
- The `normalized_query_hint` is a fallback string for keyword retrieval, not a replacement for structured entity filters.

Adaptation notes: Replace [ENTITY_CATALOG] with your actual canonical entity list, formatted as a JSON array of objects with id, canonical_name, language_variants, and aliases fields. For large catalogs, replace the inline catalog with a tool call—define a search_entity_catalog function that accepts a mention string and language, and returns candidate matches. If your retrieval system uses structured filters, add a filter_clause field to the output schema that generates the exact filter syntax your backend expects (e.g., entity_id: "canonical-123" OR entity_id: "canonical-456"). For high-stakes domains like legal or medical, route unmatched entities with confidence below 0.7 to a human review queue before retrieval proceeds.

What to do next: Test this prompt against a golden dataset of 50–100 queries with known entity mappings across your supported language pairs. Measure precision and recall of canonical ID assignment, and pay special attention to false positives where an ambiguous mention is incorrectly matched. Wire the output into your retrieval pipeline as a pre-retrieval filter, and log every unmatched entity for catalog expansion.

This prompt is designed to be called as a pre-retrieval microservice. When a user query enters your RAG system, intercept it before the retrieval step and send the raw query text, the detected source language, and your canonical entity catalog to the model. The model returns a structured JSON payload containing normalized entity identifiers and the original query with entity placeholders replaced. This normalized query string becomes the input to your downstream vector, keyword, or hybrid retrieval indexes. Do not pass the raw user query directly to retrieval if entity normalization is required for cross-lingual recall.

Implement a strict validation layer around the model's JSON output. The response must be parsed and checked before the normalized query is used for retrieval. Validate that every entity_id returned exists in your canonical entity catalog; reject or flag any hallucinated identifiers. Confirm that the normalized_query string contains the expected placeholder tokens (e.g., [ENT:canonical_id]) and that the original entity surface forms have been removed. If validation fails, log the raw response and the failure reason, then fall back to a literal translation of the original query without entity normalization. This prevents a bad normalization from silently degrading retrieval quality. For high-stakes domains like legal or medical knowledge bases, route validation failures to a human review queue instead of auto-falling back.

Cache normalized entity mappings aggressively. Entity normalization is idempotent for a given surface form and language pair. Store the mapping of (source_language, surface_form) -> canonical_entity_id in a fast key-value store (e.g., Redis) with a TTL that matches your entity catalog update cadence. Before calling the LLM, check the cache. This reduces latency and cost for frequently mentioned entities like company names, product lines, or well-known people. Invalidate the cache when your entity catalog is updated. For entity disambiguation cases where the model must consider surrounding context, include a context hash in the cache key to avoid incorrect cache hits.

Choose a model with strong multilingual instruction-following and JSON mode support. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are all suitable. Set temperature=0 to maximize deterministic entity mapping. Enable structured output mode (JSON schema constraint) if the model provider supports it; otherwise, include a retry loop that detects JSON parse errors and re-prompts with the error message appended. Limit retries to 2 attempts before falling back to the un-normalized query path. Log every retry event for observability into model reliability.

Wire this prompt into an evaluation harness before production deployment. Build a golden dataset of 100-200 queries spanning your supported languages, each with hand-labeled canonical entity IDs and expected normalized query strings. Run the prompt against this dataset and measure exact-match accuracy on entity IDs, recall of all entities present, and precision (no false entity insertions). Track semantic drift by comparing retrieval results from the normalized query against results from a human-translated ideal query. Gate deployment on passing a minimum threshold (e.g., 95% entity ID accuracy) and set up periodic re-evaluation when the entity catalog changes or the model is updated.