Sparse-Dense Hybrid

The sparse leg of the hybrid relies on high-dimensional vectors where most values are zero, corresponding to specific vocabulary terms. This is typically implemented using the BM25 algorithm, which excels at exact term matching and handling rare keywords.

• Mechanism: Builds an inverted index mapping terms to documents.
• Strength: Unmatched for queries containing unique identifiers, product codes, or proper nouns.
• Limitation: Fails on vocabulary mismatch; searching for 'physician' will miss documents containing only 'doctor'.

The dense leg encodes queries and documents into fixed-size embedding vectors (e.g., 768 dimensions) using a transformer model. These vectors capture conceptual similarity, allowing the system to retrieve 'automobile' documents for a 'car' query.

• Mechanism: Approximate Nearest Neighbor (ANN) search over a vector index.
• Strength: Handles paraphrasing and conceptual overlap robustly.
• Limitation: Can be distracted by high-level topic similarity, missing the specific, narrow intent of a keyword query.

The critical algorithmic step that merges two disjoint result lists into a single, unified ranking. Reciprocal Rank Fusion scores a document based on its reciprocal rank in each individual list, effectively prioritizing documents that appear high in both retrieval streams.

• Formula: score(d) = Σ 1 / (k + rank_i(d)) where k is a constant (often 60).
• Advantage: Non-parametric and robust; requires no score calibration between the sparse and dense systems.
• Outcome: A consensus ranking that mitigates the failure modes of either retriever alone.

An alternative to RRF that combines normalized scores from both retrievers using a weighted linear combination. This allows for dynamic tuning of the retrieval bias.

• Formula: final_score = α * sparse_score + (1 - α) * dense_score
• Tuning: The α parameter can be adjusted per query domain; a higher α favors keyword matches for technical documentation, while a lower α favors semantic understanding for conversational queries.
• Requirement: Demands careful score calibration (e.g., min-max normalization) to ensure the magnitudes are comparable.

A modern evolution that bridges the gap between the two paradigms. Models like SPLADE use neural networks to learn sparse, term-weighted vectors, effectively performing lexical matching with contextualized term expansion.

• Mechanism: Uses a log-saturation activation and FLOPs regularization to induce sparsity in a BERT-based model.
• Advantage: Automatically performs query and document expansion, mapping 'laptop' to 'notebook' and 'portable computer' in the sparse space.
• Architecture: Can serve as a drop-in replacement for BM25 in a hybrid setup, providing the efficiency of an inverted index with the intelligence of a neural model.

Sparse-dense hybrids are typically deployed in a cascading architecture to balance latency and accuracy. The first stage uses a fast, coarse filter, while the second stage applies a precise, computationally intensive re-ranker.

• Stage 1 (Candidate Generation): The hybrid retriever fetches a broad set of candidates (e.g., Top-100) from the combined sparse and dense indexes.
• Stage 2 (Re-Ranking): A powerful Cross-Encoder processes the query concatenated with each candidate document, applying full self-attention to produce a highly accurate relevance score.
• Result: This architecture achieves the speed of dual-encoder retrieval with the accuracy of a cross-encoder, making it viable for low-latency production search.