Query Rewriting

Automatically detects and corrects typographical errors before the query hits the index. This prevents zero-result searches caused by simple misspellings.

• Levenshtein Distance: Calculates the minimum single-character edits (insertions, deletions, substitutions) required to transform the query into a known term.
• Phonetic Algorithms: Uses Soundex or Metaphone to match words that sound alike but are spelled differently (e.g., 'fone' vs 'phone').
• Context-Sensitive Correction: Modern systems use language models to disambiguate corrections based on surrounding terms (e.g., 'apple' vs 'apply' in 'apple for insurance').

Resolves abbreviations and shorthand into their canonical full forms to match indexed documents that use the expanded terminology.

• Static Dictionary Mapping: Maintains a domain-specific lexicon (e.g., 'LTV' → 'Loan-to-Value Ratio').
• Contextual Disambiguation: Uses Named Entity Recognition to determine if 'ML' means 'Machine Learning' or 'Major League' based on surrounding tokens.
• Bidirectional Expansion: Adds both the acronym and the full form to the rewritten query to ensure maximum recall against documents using either variant.

Enriches the query with semantically equivalent or broader terms to overcome vocabulary mismatches between the user's language and the document's terminology.

• WordNet Integration: Leverages lexical databases to inject synonyms (e.g., 'automobile' for 'car') and hypernyms (e.g., 'vehicle' for 'car').
• Embedding-Based Mining: Uses dense vector similarity to discover domain-specific synonyms that don't exist in general-purpose thesauri.
• Weighted Expansion: Applies lower boost weights to synonym terms to prevent them from dominating the relevance score over the original query terms.

Strategically removes or segments overly restrictive query clauses that would otherwise yield zero results, ensuring a graceful degradation of precision for the sake of recall.

• Chunking: Splits long, complex natural language queries into smaller, independent sub-queries for parallel retrieval.
• Stop Word Handling: Removes high-frequency noise words (e.g., 'the', 'is') for sparse retrieval while preserving them for dense semantic encoding.
• Mandatory Term Dropping: If a Boolean AND query returns empty, the system iteratively drops the least frequent term to find a partial match.

Classifies the user's underlying goal—navigational, informational, or transactional—and restructures the query syntax to match the retrieval strategy best suited for that intent.

• Navigational Queries: Rewrites to prioritize exact title and anchor text matches to find a specific page.
• Informational Queries: Expands the query with definitional phrases (e.g., 'what is') to trigger dense passage retrieval.
• Transactional Queries: Adds action-oriented terms (e.g., 'buy', 'download', 'pricing') to surface product pages.

Utilizes sequence-to-sequence Transformer models to generate an optimal rewritten query, moving beyond rule-based heuristics to learn complex paraphrasing patterns from click-through data.

• T5-Based Rewriters: Fine-tunes encoder-decoder models on pairs of raw queries and their high-performing reformulations.
• Reinforcement Learning: Trains the rewriter to maximize downstream retrieval metrics like NDCG rather than just lexical similarity.
• Zero-Shot Generalization: Applies the model to unseen query patterns without requiring manual rule creation.

The foundational preprocessing step that standardizes raw text before any rewriting or retrieval occurs. Normalization ensures that variant forms of the same term are treated identically by the search index.

• Case Folding: Converting all characters to lowercase to eliminate case mismatches
• Stemming: Crudely reducing words to their root form by stripping suffixes (e.g., "running" → "run")
• Lemmatization: Morphologically precise reduction using a dictionary (e.g., "better" → "good")
• Stop Word Removal: Filtering out high-frequency function words that carry little semantic weight
• Unicode Normalization: Standardizing character encodings (e.g., NFC vs NFD forms) to prevent invisible mismatches

A preprocessing step that analyzes the user's raw query to determine its underlying goal before rewriting or retrieval begins. The classified intent drives downstream decisions about fusion weights and rewriting strategies.

• Navigational: User wants a specific document or page (e.g., "login page")
• Informational: User seeks knowledge on a topic (e.g., "how does RRF work")
• Transactional: User intends to perform an action (e.g., "download whitepaper")
• Intent-aware rewriting dynamically selects expansion or relaxation strategies based on the classified intent, preventing inappropriate transformations that would harm precision

A predefined retrieval logic path that activates when the primary query rewriting and search pipeline returns zero or low-confidence results. Fallback strategies prevent empty result pages by progressively relaxing constraints.

• Query Relaxation: Removing optional clauses or filters that may be overly restrictive
• Spelling Correction Fallback: Applying fuzzy matching or edit-distance-based correction when exact match fails
• Pure Lexical Fallback: Reverting to BM25-only retrieval when dense vector search produces low similarity scores
• Chaining: Executing a sequence of increasingly permissive rewrites until a minimum result threshold is met

The fundamental disconnect between keyword-based retrieval, which matches exact character sequences, and semantic retrieval, which understands conceptual intent. Query rewriting is the primary mechanism for bridging this gap before retrieval begins.

• Vocabulary Mismatch Problem: Users and authors use different words for the same concept (e.g., "car" vs "automobile")
• Acronym Expansion: Rewriting "LTV" to "lifetime value" bridges the gap when the index contains only the expanded form
• Entity Normalization: Mapping informal mentions to canonical entity identifiers (e.g., "NYC" → "New York City")
• Effective rewriting reduces the burden on hybrid fusion by aligning the query's surface form with the index's vocabulary before scoring occurs

Techniques that detect and repair typographical errors in user queries before they reach the retrieval pipeline. These are often the first stage in a query rewriting workflow.

• Edit Distance (Levenshtein): Measures the minimum number of single-character insertions, deletions, or substitutions required to transform the query into a valid term
• Phonetic Matching: Algorithms like Soundex or Metaphone match words by their pronunciation, useful for "sounds-like" errors
• Context-Sensitive Correction: Uses language models to select the most probable correction given surrounding terms (e.g., "apple pie" vs "apple phone")
• Did You Mean?: A user-facing suggestion generated when the corrected query differs significantly from the original input