Query Expansion

Leverages lexical databases like WordNet to add terms with identical or highly similar meanings. This ensures that a search for 'automobile' also retrieves documents containing 'car'.

• Mechanism: Dictionary-based lookup or embedding similarity.
• Example: Query 'CEO salary' expands to 'chief executive officer compensation'.
• Risk: Can introduce noise if synonyms have domain-specific meanings.

A blind feedback loop that assumes the top-k documents from an initial retrieval are relevant. Key terms are extracted from these documents and added to the query for a second-pass retrieval.

• Mechanism: Term frequency analysis on initial results.
• Example: A query for 'jaguar' returns top documents about cars; PRF adds terms like 'luxury', 'sedan', and 'horsepower'.
• Risk: Query drift if the initial top documents are not actually relevant.

Builds a statistical thesaurus from the entire corpus before any query is issued. Terms that frequently co-occur in documents are linked, allowing the system to expand queries with contextually associated words.

• Mechanism: Pre-computed term correlation matrix.
• Example: In a medical corpus, 'myocardial infarction' is strongly associated with 'heart attack' and 'troponin'.
• Advantage: Fast at query time since the associations are pre-calculated.

Uses generative LLMs to produce additional context, keywords, or a hypothetical answer. The generated text is then used to augment the original search terms for dense or sparse retrieval.

• Mechanism: Prompting an LLM to generate related terms or a Hypothetical Document Embedding (HyDE).
• Example: A query 'how to fix a flat tire' generates a mini-essay on tire repair, whose dense embedding is used to find similar real documents.
• Advantage: Captures deep semantic intent beyond simple synonyms.

Identifies the specific meaning of a polysemous word in context before expanding. This prevents the addition of terms from the wrong sense of the word.

• Mechanism: Contextual analysis to determine the correct synset.
• Example: For the query 'apple health benefits', WSD identifies 'apple' as the fruit, not the company, and expands with 'malus domestica' and 'fiber'.
• Risk: High computational cost for real-time disambiguation.

Maps query terms to abstract concepts in a structured knowledge base or ontology, then expands with all terms linked to that concept. This moves beyond surface-form words to canonical entity identifiers.

• Mechanism: Entity linking to a knowledge graph.
• Example: A query for 'New York' is linked to the entity Q60 in Wikidata, expanding with 'NYC', 'The Big Apple', and 'City of New York'.
• Advantage: Highly precise for named entities and domain-specific jargon.

The most direct form of query expansion. Adds words with identical or near-identical meanings to the original query terms. For example, a query for 'laptop repair' expands to include 'notebook repair' and 'portable computer repair'. Implementation relies on:

• Curated thesauri and domain-specific synonym rings.
• Word embeddings to discover distributionally similar terms.
• Word Sense Disambiguation (WSD) to prevent adding synonyms for the wrong sense of a polysemous word. Without WSD, expanding 'bank' with 'riverbank' synonyms for a financial query introduces catastrophic noise.

A classic blind relevance feedback technique. PRF assumes the top-k documents from an initial retrieval are relevant, extracts the most discriminative terms from those documents, and appends them to the original query for a second retrieval pass. Key considerations:

• k is critical: too small and the expansion terms are noisy; too large and the topic drifts.
• Works well for high-precision initial retrievals; fails catastrophically if the top-k documents are off-topic.
• Modern variants use neural PRF, where a language model reads the top-k documents and generates a refined query rather than simply extracting keywords.

Hypothetical Document Embeddings (HyDE) flips the expansion paradigm. Instead of adding terms to the query, a language model generates a hypothetical ideal document that would answer the query. The dense embedding of this synthetic document is then used to perform a vector similarity search against the real corpus. This technique:

• Excels when queries are short and the corpus contains verbose documents.
• Leverages the language model's world knowledge to hallucinate a plausible answer structure.
• Is inherently unsupervised and requires no relevance judgments.
• Risks embedding bias from the generative model into the retrieval process.

Generative LLMs can expand queries by producing additional context, keywords, or clarifying questions. Common patterns include:

• Keyword generation: Prompt the model to list related terms and concepts for a given query.
• Contextual expansion: Ask the model to elaborate on the query's domain, generating a paragraph that enriches the dense retrieval embedding.
• Chain-of-thought decomposition: For complex queries, the model breaks the question into sub-queries, each of which can be independently expanded and resolved. This approach is powerful but introduces latency and cost; it is typically reserved for queries where simpler expansion methods fail.

Expansion without constraint leads to topic drift. Query scoping analyzes a query to determine its domain, temporal range, and authority constraints before expansion occurs. For example:

• A query containing '2024' triggers a temporal scope that prevents expansion terms from pulling in outdated documents.
• A query with 'FDA guidelines' scopes the search to regulatory domains, blocking expansion into general medical advice.
• Scoping often uses entity recognition to identify constraining facets and applies metadata filters during retrieval to enforce the detected boundaries.