Query Expansion

Two primary paradigms govern how queries are enriched:

• Lexical Expansion: Adds synonyms, stemming variants, and spelling corrections using a thesaurus like WordNet. It directly addresses surface-form mismatch.
• Semantic Expansion: Uses dense vector embeddings or knowledge graphs to add conceptually related terms that don't share lexical overlap. For example, expanding 'heart attack' to 'myocardial infarction'.
• Hybrid Approaches: Modern systems often combine both, using lexical methods for high-precision recall and semantic methods for conceptual breadth.

A classic, unsupervised expansion method that assumes the top-k documents from an initial retrieval are relevant.

• Mechanism: The system extracts key terms from these pseudo-relevant documents and appends them to the original query.
• Risk: Can suffer from query drift if the initial top documents are not actually relevant, introducing noise.
• Modern Adaptation: Contextual PRF uses dense embeddings to select expansion terms that are semantically coherent with the entire initial result set, improving robustness.

Leverages large language models to generate expanded queries or hypothetical documents.

• HyDE (Hypothetical Document Embeddings): The LLM generates a fictional, ideal document answering the query. The embedding of this synthetic document is then used to retrieve real, similar documents.
• Query2Doc: Generates a pseudo-document directly from the query, concatenating it with the original query for a richer representation.
• Multi-Query Generation: The LLM produces several diverse rephrasings of the original query, and retrieval is performed in parallel for each, with results fused via reciprocal rank fusion.

Uses structured entity relationships to expand queries with logically connected concepts.

• Entity Linking: The query is parsed to identify and disambiguate entities, linking them to a knowledge graph like Wikidata.
• Graph Traversal: The system traverses one-hop or two-hop relationships to add attributes, aliases, and related entities. A query for 'Tesla' might be expanded to include 'Elon Musk', 'electric vehicle', and 'Palo Alto'.
• Benefit: Provides highly precise, deterministic expansion that avoids the hallucination risks of generative methods.

Tailors expansion terms based on the user's session history, profile, or domain.

• Session Context: If a user previously searched for 'Java', a subsequent query for 'garbage collection' is expanded with programming terms, not waste management terms.
• Domain-Specific Vocabularies: In specialized fields like medicine or law, expansion uses curated ontologies (e.g., MeSH, SNOMED CT) to ensure added terms are professionally valid.
• Vector-Based Context: The embedding of the user's recent interaction history is used to bias the expansion model toward terms relevant to the ongoing task.

Expansion directly impacts the computational cost and quality of retrieval.

• Increased Recall: Adding terms reliably increases recall, often by 10-20%, by matching documents that use different terminology.
• Precision Risk: Over-expansion, especially with generic terms, can dilute the query's intent and reduce precision.
• Latency Budget: Each expansion term adds to the retrieval cost. Techniques like term weighting and pruning low-weight expansion terms are critical to maintain sub-second latency in production systems.