Personalized Entity Resolution Prompt for Query Disambiguation

This prompt is designed for search relevance engineers and RAG developers who need to resolve ambiguous entity mentions in user queries before retrieval. The job-to-be-done is mapping a user's vague or overloaded reference—such as a project codename, an internal acronym, or a person's first name—to a specific, disambiguated entity identifier grounded in that user's organizational context. The ideal user is an engineer integrating this prompt into a retrieval pipeline where the same surface query from different users must resolve to different entities based on role, department, recent interactions, or access scope.

Use this prompt when your system has access to a structured user profile containing attributes like department, recent_document_ids, project_memberships, or team_entity_aliases. The prompt requires these inputs to perform resolution. It is not suitable for anonymous or first-time users with no interaction history, nor for queries where the ambiguous term has no organizational context to disambiguate it. Do not use this prompt for general synonym expansion or query relaxation; it is specifically for entity resolution against a known set of organizational entities. In high-stakes domains like healthcare or legal search, always route the disambiguated output for human review before retrieval execution to prevent misidentification of patient records, legal matters, or regulated entities.

Before wiring this into production, define your evaluation criteria: measure entity resolution accuracy across a golden set of ambiguous queries paired with different user profiles, and test for over-resolution where the prompt forces a specific entity when the query was intentionally broad. Common failure modes include resolving to a recently accessed but incorrect entity due to recency bias in the user history, or failing to resolve when the user's profile lacks sufficient context. If your entity catalog changes frequently, pair this prompt with a retrieval step that fetches the current entity list rather than hardcoding it. Next, review the prompt template and implementation harness to see how to embed this into your query rewriting pipeline.

This prompt template is designed to be the core instruction set for an LLM tasked with disambiguating entity references. It takes a raw user query, a structured user profile, and a list of candidate entities as input. The model's job is to select the most probable entity based on the user's context and produce a rewritten query that replaces the ambiguous term with a canonical, unambiguous identifier. The template uses square-bracket placeholders for all dynamic inputs, making it ready to be populated by your application logic before each inference call.

textCopy
You are an entity resolution specialist for an enterprise search system. Your task is to disambiguate an ambiguous entity mention in a user's query.

You will be provided with:
- [USER_QUERY]: The original query containing an ambiguous entity reference.
- [AMBIGUOUS_TERM]: The specific word or phrase in the query that is ambiguous.
- [USER_PROFILE]: A JSON object containing the user's department, role, location, and active projects.
- [CANDIDATE_ENTITIES]: A JSON array of potential entity objects, each with an 'id', 'name', 'type', and 'associations' list.
- [USER_HISTORY]: A JSON array of the user's recent interactions with entities (e.g., viewed documents, assigned tickets).

# Steps
1.  **Analyze the Query:** Identify the [AMBIGUOUS_TERM] in the [USER_QUERY].
2.  **Review Candidates:** Examine the [CANDIDATE_ENTITIES] list.
3.  **Contextual Match:** For each candidate, calculate a relevance score based on its overlap with the [USER_PROFILE] and [USER_HISTORY]. Prioritize candidates that share the user's department, are part of their active projects, or appear in their recent history.
4.  **Select Best Entity:** Choose the single candidate with the highest relevance score. If no candidate has a meaningful connection, the best entity is 'null'.
5.  **Rewrite Query:** Generate a new query string where the [AMBIGUOUS_TERM] is replaced with the selected entity's canonical 'id' in the format `entity:{id}`. If the best entity is 'null', return the original query unchanged.

# Constraints
- [CONSTRAINTS]: You must not invent entity IDs. Only use IDs from the provided [CANDIDATE_ENTITIES] list. If the connection is weak, default to 'null' rather than guessing. Do not alter any other part of the query.

# Output Format
You must respond with a single, valid JSON object without any markdown formatting. The object must conform to this schema:
[OUTPUT_SCHEMA]

To adapt this template, start by defining the structure of your [USER_PROFILE] and [CANDIDATE_ENTITIES] objects to match your internal systems. The [OUTPUT_SCHEMA] placeholder should be replaced with a strict JSON schema definition, including required fields like disambiguated_query and selected_entity_id. The [CONSTRAINTS] section is critical for high-stakes environments; you should expand it to include rules about handling tied scores or entities with expired access. After populating the placeholders, test the prompt with a golden dataset of queries and user profiles to calibrate the model's matching logic before production use.

Integrating this prompt into a production system requires treating it as a deterministic function within a broader retrieval pipeline. The prompt expects a user query, a structured user profile, and an entity catalog as inputs. Before calling the LLM, the application layer must assemble these inputs: the raw query string from the user, a JSON profile object fetched from the identity service containing role, department, recent_interactions, and permissions, and a pre-indexed entity catalog mapping canonical entity names to their unique identifiers and aliases. The prompt should be invoked as a pre-retrieval step, meaning its output—a disambiguated query string with resolved entity IDs—is passed directly to the downstream retrieval system, whether that's a vector store, a keyword index, or a hybrid search endpoint. Do not expose the raw LLM output to the user; it is an internal signal for retrieval accuracy.

The implementation must include a validation layer that parses the model's output against a strict schema before the disambiguated query is used for retrieval. The expected output is a JSON object with at minimum a disambiguated_query string and a resolved_entities array of objects, each containing mention, resolved_name, entity_id, and confidence. If the model returns malformed JSON, missing required fields, or an entity ID that does not exist in the source catalog, the system should reject the output and fall back to the original raw query. A retry loop with a maximum of two attempts can be added, using a more explicit error message in the retry prompt, but if validation continues to fail, the system must log the failure and proceed with the unmodified query to avoid increased latency. For high-risk domains where entity misresolution could surface unauthorized documents, implement a human review queue for low-confidence resolutions (e.g., confidence < 0.85) or for queries where the resolved entity's access scope conflicts with the user's permission profile.

Model choice and latency are critical considerations. This prompt performs a classification and mapping task that requires strong instruction-following and structured output capabilities. Faster, cheaper models (e.g., GPT-4o-mini, Claude Haiku) are usually sufficient for this task, provided they are tested against your specific entity catalog and user profile schemas. Implement structured output modes (JSON mode or tool calling with a defined schema) rather than relying on parsing free-text responses. Log every invocation with the input profile hash, the raw query, the resolved entities, the confidence scores, and the validation status. This audit trail is essential for debugging retrieval failures and for tuning the entity catalog. Finally, implement a cache keyed on the combination of user_id and query_string with a short TTL (e.g., 5 minutes) to avoid redundant LLM calls when users repeat or refine queries in quick succession.