Synonym Expansion Prompt Template for Keyword Search

This prompt is designed for search engineers and RAG developers who are diagnosing a specific failure mode in their retrieval system: vocabulary mismatch. This occurs when users consistently use different terminology than the language found in your indexed documents. For example, a user searching a hardware support knowledge base for 'laptop repair' may get zero results if the documents exclusively use the terms 'notebook servicing' or 'computer maintenance.' The core job-to-be-done is to bridge this lexical gap by programmatically generating a weighted list of synonyms, related terms, and conceptual alternatives from the original query. The output is not a final answer; it is a structured artifact designed to be fed directly into a keyword search engine's query builder (e.g., Elasticsearch's should clauses) or the sparse retrieval component of a hybrid RAG pipeline.

You should reach for this prompt when you have concrete evidence of a recall problem in a keyword-based index, such as BM25, and you cannot solve it through document-side enrichment alone. The ideal input is a short, focused user query where the core concepts are clear but the specific terminology is narrow. The prompt respects part-of-speech constraints and domain context, meaning you can supply a [DOMAIN_CONTEXT] like 'enterprise IT support' to prevent the model from generating synonyms that are technically valid but contextually wrong (e.g., expanding 'server' to 'waiter'). Do not use this prompt when your query already contains precise technical identifiers like error codes, serial numbers, or UUIDs that must be matched exactly and should never be expanded. It is also the wrong tool if your primary goal is to improve dense vector recall, where semantic similarity is already handled by the embedding model; for that, see the 'Concept Expansion Prompt for Vector Retrieval' playbook.

Before integrating this prompt, establish a clear evaluation baseline. Measure your current recall@k on a representative set of queries where you know relevant documents exist but are not being surfaced. After implementing the expansion, re-run the same evaluation to quantify the improvement. Be vigilant for the most common failure mode: over-expansion, where adding too many low-precision synonyms floods the results with irrelevant documents, destroying precision. To mitigate this, always use the [CONFIDENCE_THRESHOLD] and [MAX_TERMS] constraints in the prompt template, and consider pairing the expanded query with a strict re-ranker. If the workflow involves regulated content or high-stakes decisions, the expanded terms must be logged and made auditable, and a human-in-the-loop review step should be added before the query is executed against production data.

This template is the core instruction set for a model to act as a query expansion engine. It takes a raw user query and a domain context description and produces a structured list of synonyms and related terms. The prompt is designed to be dropped into an API call, a LangChain node, or a custom proxy. Before copying, identify the placeholders you must fill at runtime: the user's original query, the domain context that constrains term meaning, and the output format you intend to parse downstream. The prompt instructs the model to weight terms by relevance, filter by part of speech, and reject terms that would introduce ambiguity or drift outside the specified domain.

textCopy
You are a search query expansion assistant. Your task is to improve keyword search recall by generating a weighted list of synonyms, related terms, and conceptual alternatives for a given user query.

INPUT:
- User Query: [USER_QUERY]
- Domain Context: [DOMAIN_CONTEXT]

CONSTRAINTS:
- Generate terms that are direct synonyms, hyponyms, hypernyms, or commonly co-occurring domain terms.
- Preserve the original query's intent. Do not introduce terms that change the meaning or negate the original query.
- Assign a relevance weight from 0.0 to 1.0 for each term, where 1.0 is a perfect synonym.
- Exclude terms that are proper nouns unless they are known aliases for entities in the domain context.
- If the query contains a negation (e.g., 'without', 'excluding'), do not generate synonyms for the negated terms.
- Limit output to a maximum of [MAX_TERMS] terms.
- If no relevant expansion is possible, return an empty list.

OUTPUT_SCHEMA:
Return a valid JSON object with the following structure:
{
  "original_query": "string",
  "domain": "string",
  "expanded_terms": [
    {
      "term": "string",
      "weight": number,
      "part_of_speech": "noun|verb|adjective|adverb",
      "rationale": "short explanation of relationship to original query"
    }
  ]
}

To adapt this template, start by replacing [USER_QUERY] and [DOMAIN_CONTEXT] with runtime variables from your application. The [DOMAIN_CONTEXT] field is critical: a query for 'cells' in a biology context should produce different expansions than in a telecommunications or prison context. Set [MAX_TERMS] based on your search engine's query length limits and performance budget—start with 5 to 10 and measure recall improvement before increasing. If your application requires a different output format, such as a plain list for a keyword index that cannot parse JSON, modify the OUTPUT_SCHEMA section accordingly, but keep the weighting and rationale fields for debugging and evaluation. Always validate the model's output against the schema before sending terms to your search index; malformed JSON or missing fields will cause downstream failures. For high-stakes domains like legal or medical search, add a human review step before expanded terms are committed to a production index.

This prompt is designed to sit between the user's raw query and your keyword search index. The typical integration point is a pre-retrieval expansion step in a search service or RAG pipeline. The application receives a user query, calls the LLM with this prompt template, parses the structured output, and appends the expanded terms to the original query before executing the search. The prompt expects a [QUERY] string and an optional [DOMAIN_CONTEXT] description. The output contract should be a strict JSON schema containing an array of term objects, each with a term, weight, and optional rationale field. Define this schema in your application code and validate the LLM response against it before using any terms in retrieval.

Validation is the first line of defense. After parsing the JSON, check that no expanded term is identical to the original query token, that weights are within a defined range (e.g., 0.0 to 1.0), and that the total number of terms does not exceed your search engine's query clause limit. Implement a confidence threshold: discard any term with a weight below a configurable minimum (start with 0.3 and tune based on eval). For high-recall use cases, log all discarded terms for offline analysis. If the LLM response fails JSON parsing or schema validation, implement a single retry with a repair prompt that includes the raw output and the validation error message. After one retry, fall back to the original query without expansion to avoid latency spikes. Never retry more than twice for this non-critical path.

Model choice matters for cost and latency. This task is classification-adjacent and does not require a frontier model. A fast, cheaper model like GPT-4o-mini, Claude Haiku, or a fine-tuned open-weight model works well. If you are processing queries at high volume, consider batching expansion requests where latency allows, or caching frequent query expansions in a key-value store with a TTL. Instrument the expansion step with metrics: expansion latency, term count, discard rate, and cache hit rate. Log a sample of expanded queries alongside the original query and the final search result count to monitor for term drift or over-expansion that floods the index with low-quality clauses. The next step after implementing this harness is to run the Synonym Expansion Eval Check Prompt against a golden query set to calibrate your weight threshold and measure recall impact before production rollout.