Synonym Expansion with Re-Ranking Integration Prompt

Use this prompt when you need to expand a user query with synonyms and related terms while preserving explicit provenance for each added term. This is critical for retrieval pipelines where a downstream re-ranker must distinguish between original query terms and expanded terms to apply different match weights. Standard synonym expansion prompts produce flat term lists that re-rankers cannot interpret. This prompt solves that by attaching a source field to every expanded term, marking it as original, synonym, hypernym, or concept. The ideal user is a retrieval pipeline architect or search engineer who controls both the expansion step and the re-ranking configuration, and who needs the re-ranker to boost exact original-term matches while giving lower but non-zero weight to conceptually related expansions.

Concretely, deploy this prompt when your retrieval stack includes a learned re-ranker, a cross-encoder, or a fusion layer that benefits from provenance-aware scoring. For example, if you use a two-stage pipeline where a vector or keyword index retrieves candidate documents and a cross-encoder re-ranks them, this prompt's output lets you configure the re-ranker to assign a higher relevance boost to documents matching original terms than to those matching hypernym expansions. Without provenance labels, the re-ranker treats all query terms identically, losing the signal that the user's exact words are stronger relevance indicators than automatically generated alternatives. The prompt's structured output contract also enables automated validation: you can assert that every term in the expansion output has a non-null source field and that at least one term is marked original.

Do not use this prompt if your retrieval backend is a pure vector database with no re-ranking step; a simpler expansion prompt that produces a flat list of terms will suffice and cost fewer tokens. Similarly, avoid this prompt if your re-ranker is a simple lexical scorer like BM25 that cannot consume per-term provenance metadata—the added structure provides no benefit and increases output parsing complexity. If you are unsure whether your re-ranker can exploit provenance labels, start by inspecting its configuration or documentation for term-weighting hooks, then adopt this prompt only once you confirm the downstream component can act on the source field. For teams evaluating whether to introduce re-ranking into a previously single-stage retrieval pipeline, this prompt serves as the expansion-side contract that makes provenance-aware re-ranking possible.

This prompt template is designed to be pasted directly into your orchestration layer. It instructs the model to produce a structured expansion of a user query, where each generated term includes a provenance label (original or expanded) and a confidence score. This explicit provenance is the critical integration contract with your re-ranking stage, allowing it to boost documents that match original user terms while still considering documents that match high-confidence synonyms. Replace every square-bracket placeholder with values from your application context before sending the request.

textCopy
System: You are a query expansion engine for a retrieval system. Your output will be consumed by a downstream re-ranker that weights term matches by provenance. Follow the output schema exactly.

User:
Expand the following user query with synonyms, related terms, and conceptual alternatives to improve retrieval recall.

[QUERY]

Domain context: [DOMAIN_CONTEXT]

Expansion constraints:
- Maximum total terms: [MAX_TERMS]
- Minimum confidence threshold for inclusion: [MIN_CONFIDENCE]
- Preserve these protected entities without expansion: [PROTECTED_ENTITIES]
- Negation scope to preserve: [NEGATION_SCOPE]

Output a JSON object with this exact schema:
{
  "original_query": "string",
  "expanded_terms": [
    {
      "term": "string",
      "provenance": "original" | "expanded",
      "confidence": 0.0-1.0,
      "rationale": "string"
    }
  ],
  "negation_preserved": true | false
}

[EXAMPLES]

Adaptation notes: Replace [QUERY] with the user's raw input. [DOMAIN_CONTEXT] should describe the knowledge base domain (e.g., 'enterprise cloud infrastructure documentation'). [MAX_TERMS] prevents query bloat that degrades retrieval latency. [PROTECTED_ENTITIES] is a list of product names, person names, or codes that must not be synonymized. [NEGATION_SCOPE] captures phrases like 'without' or 'excluding' to prevent semantic inversion. The [EXAMPLES] placeholder should contain 2-3 few-shot demonstrations of correct provenance labeling. Before deploying, validate outputs against the schema and run a regression suite using the Synonym Expansion Eval Check Prompt Template to detect term drift.

The synonym expansion prompt is not a standalone component—it sits between the user's original query and the retrieval engine, feeding expanded terms into both sparse and dense retrieval backends before results reach the re-ranker. The implementation harness must capture the original query terms alongside each expanded term with explicit provenance markers so the downstream re-ranker can apply differential weighting. Without this harness, the re-ranker cannot distinguish between a match on the user's exact term and a match on a model-generated synonym, which undermines relevance tuning and makes expansion quality opaque in production traces.

To wire this into a retrieval pipeline, wrap the prompt call in a function that accepts the user query, optional session context, and a domain glossary or controlled vocabulary. The function should call the LLM with the prompt template, parse the structured output (a JSON array of objects with fields: term, weight, source, original_term, and reject), and validate each entry against the provided glossary if one exists. Terms with reject: true or confidence below a configurable threshold should be dropped before injection. The harness should then produce two parallel query structures: a keyword query string with boosted original terms and weighted expanded terms, and a dense vector query constructed from the concatenation of original and accepted expanded terms. Both structures carry provenance metadata into the retrieval layer. Log every expansion decision—accepted terms, rejected terms, weights, and the prompt version—so that retrieval failures can be traced to specific expansion choices.

Re-ranking integration requires that the retrieval response preserves which terms matched each document. The harness should annotate retrieved documents with match provenance: whether the match came from an original query term, a synonym, or a concept expansion. The re-ranker prompt then receives these annotations and can apply a scoring policy—for example, boosting exact original-term matches by 1.0×, synonym matches by 0.7×, and concept-expansion matches by 0.5×. Implement a validation step that checks whether the expansion output conforms to the expected schema before retrieval executes; if validation fails, fall back to the original unexpanded query and log the failure. For high-stakes domains, add a human-review gate on expansion outputs when the prompt's confidence scores fall below a defined threshold or when the expansion introduces terms not present in the approved glossary. Avoid running expansion on every query unconditionally—cache frequent query patterns and skip expansion for queries that already match controlled vocabulary terms exactly.