Hybrid Retrieval

The sparse retriever handles exact keyword and term matching. It relies on traditional information retrieval algorithms like BM25 or TF-IDF and uses an inverted index data structure for fast lookups.

• Primary Function: High precision for queries containing specific named entities, technical terms, or acronyms.
• Key Technology: Often implemented using search libraries like Apache Lucene (powering Elasticsearch or OpenSearch).
• Limitation: Suffers from the vocabulary mismatch problem; synonyms or paraphrases of query terms are not recognized.

The dense retriever captures semantic meaning. It uses an embedding model (e.g., Sentence-BERT, OpenAI embeddings) to convert text into high-dimensional vector embeddings. Similarity is measured using metrics like cosine similarity.

• Primary Function: High recall for conceptual queries, understanding user intent beyond keywords.
• Key Technology: Requires a vector index (e.g., HNSW, IVF) built using libraries like FAISS, Weaviate, or Pinecone for Approximate Nearest Neighbor (ANN) search.
• Limitation: Can be less precise for rare or out-of-domain vocabulary not well-represented in the embedding model's training data.

This component is the 'hybrid' orchestrator. It combines ranked lists from the sparse and dense retrievers into a single, optimized list.

• Fusion Techniques: Simple methods include weighted sum of scores or Reciprocal Rank Fusion (RRF), which is robust to different score distributions.
• Reranking: The fused list can be passed to a computationally intensive cross-encoder model (e.g., a BERT-based reranker) for precise pairwise relevance scoring, a process known as retrieve-and-rerank.
• Output: A final, unified ranking of documents intended to maximize both recall and precision.

This pre-retrieval component processes the raw user query to improve its effectiveness for both retrieval paths.

• For Sparse Search: May involve query expansion (adding synonyms via a thesaurus), spelling correction, and stemming/lemmatization.
• For Dense Search: Involves generating the query embedding. For complex queries, it may perform query decomposition (breaking a multi-part question into sub-queries) or hypothetical document embedding (HyDE).
• Goal: To bridge the gap between the user's natural language expression and the system's retrieval mechanisms.

The specialized storage and retrieval backend for the dense retriever. It is not a traditional relational database.

• Core Function: Stores millions of document embeddings and enables sub-second ANN search.
• Index Types: Common algorithms include Hierarchical Navigable Small World (HNSW) graphs for high recall/speed and Inverted File (IVF) indexes with Product Quantization (PQ) for memory efficiency.
• Systems: Dedicated vector databases like Pinecone, Weaviate, and Qdrant, or vector search extensions to existing databases (e.g., pgvector for PostgreSQL).

The offline system that prepares raw source data for efficient retrieval. Quality here directly determines retrieval quality.

• Steps: Includes extraction (from PDFs, databases), cleaning, chunking (into optimal-sized segments), metadata attachment, and embedding generation for the dense index.
• Chunking Strategies: Critical for balancing context richness with precision. Methods include fixed-size, semantic, or recursive splitting.
• Output: A prepared corpus with both an inverted index (for sparse) and a vector index (for dense) populated and ready for querying.

This pattern executes sparse and dense retrievers in parallel, then combines their relevance scores into a single ranked list. It is the most common and straightforward implementation.

Key Techniques:

• Reciprocal Rank Fusion (RRF): Combines ranked lists by summing the reciprocal of the ranks from each list. Robust as it doesn't require score normalization.
• Weighted Linear Combination: Assigns a tunable weight (e.g., α=0.5) to balance the normalized BM25 score and the cosine similarity score: final_score = α * sparse_score + (1-α) * dense_score.

Example: A query for "machine learning model training" retrieves documents via BM25 (matching "model training") and a dense retriever (matching the semantic concept). RRF fuses these two result sets.

This pattern uses a fast, high-recall first-stage retriever (often hybrid) to fetch a large candidate set, which is then precision-filtered by a computationally intensive second-stage model.

Typical Pipeline:

1. First-Stage (Recall): A hybrid retriever (BM25 + lightweight dual-encoder) fetches 100-200 candidate documents.
2. Second-Stage (Precision): A cross-encoder model jointly processes the query and each candidate to compute a highly accurate relevance score, reordering the final top-k results (e.g., top 10).

Use Case: Enterprise RAG systems where answer quality is paramount and latency budgets allow for a more expensive reranking step (~50-100ms).

This pattern replaces traditional sparse methods like BM25 with a neural model that generates learned sparse representations. It combines the interpretability of lexical matching with the adaptability of learned models.

How it works: Models like SPLADE or uniCOIL generate a sparse, weighted bag-of-words vector for queries and documents. The weights are learned from data, allowing the model to expand queries with related terms and assign importance scores.

Advantages:

• Can be combined with dense vectors via score fusion.
• Often outperforms BM25 by learning domain-specific vocabulary and term importance.
• Maintains the efficiency of inverted index lookups.

This pattern uses the ColBERT model architecture, which employs a late interaction mechanism. It provides a middle ground between the efficiency of dual encoders and the expressiveness of cross-encoders.

Key Mechanism:

• Query and documents are encoded independently into fine-grained, contextualized token embeddings.
• Relevance is scored via a "sum-of-max" operation: for each query token, find its maximum similarity with any document token, then sum these maxima.

Hybrid Application: ColBERT's token-level embeddings can be augmented with traditional sparse term matching signals, or its ranked list can be fused with a BM25 list. Its design inherently captures both lexical and semantic matching.

This intelligent pattern uses a classifier or router to decide, per query, which retrieval method(s) to use. It optimizes for cost and latency by avoiding unnecessary searches.

Routing Logic:

• Keyword-Heavy Queries: Route to sparse retrieval (BM25). Example: "Python API documentation version 3.11".
• Semantic / Conceptual Queries: Route to dense retrieval. Example: "methods for preventing overfitting in neural networks".
• Ambiguous Queries: Execute both paths and fuse results.

The router can be a simple heuristic (e.g., query length, presence of rare terms) or a trained lightweight model.

This advanced pattern stores both dense vectors and sparse lexical signals within a single, unified index structure. This allows for a joint similarity computation in a single retrieval pass.

Implementation:

• Systems like PISA or modern extensions to Faiss and Elasticsearch allow indexing of multi-vector representations.
• A document can be represented by both its dense embedding and a sparse vector (e.g., BM25 term weights or a learned sparse representation).
• The search combines distances/scores from both representations using a unified scoring function during the ANN search traversal.

Benefit: Reduces system complexity and can improve latency by performing a single, combined search operation.

A neural search method that uses vector embeddings to find documents based on semantic similarity. A query and documents are encoded into a shared high-dimensional space by an embedding model (e.g., a transformer). Relevance is determined by proximity in this vector space, typically measured by cosine similarity. This approach excels at understanding conceptual intent and synonyms but can struggle with exact keyword matching and rare terms.

• Key Architecture: Dual Encoder (Bi-Encoder)
• Primary Use: Capturing semantic meaning and paraphrasing.
• Example: The query "ways to save money" retrieves documents about "budgeting techniques" even without keyword overlap.

A traditional, lexical search method that matches documents based on the presence of exact query terms. It relies on statistical models like BM25 or TF-IDF and a core data structure called an inverted index. This index maps each unique token to a list of documents containing it (a postings list), enabling extremely fast Boolean and ranked keyword search.

• Key Algorithm: BM25 (Okapi BM25), which improves upon TF-IDF with term frequency saturation and document length normalization.
• Primary Use: High-precision matching of specific keywords, names, and technical jargon.
• Limitation: Fails on vocabulary mismatch; "automobile" won't match "car" without explicit synonym expansion.

A precision-oriented component in a two-stage retrieval pipeline. After a fast first-stage retriever (like BM25 or a dual encoder) fetches a broad set of candidate documents, a cross-encoder model jointly processes the query and each candidate document pair to produce a refined relevance score. This architecture is computationally expensive but provides the highest ranking accuracy.

• Function: Reorders and rescores an initial candidate list (e.g., top 100 documents).
• Trade-off: High accuracy for significantly higher latency; not used for the initial large-scale search.
• Example Models: MonoT5, RankT5, or fine-tuned BERT models used exclusively for reranking.

A robust score fusion technique used to combine ranked lists from different retrievers (e.g., sparse and dense) into a single, unified ranking. It operates on the ranks of documents, not their raw similarity scores, which avoids complex score normalization. For each document, its reciprocal ranks from each list are summed.

• Formula: score = sum(1 / (k + rank)) for each list where k is a constant (often 60).
• Advantage: Stateless and simple; does not require the underlying systems to produce comparable scores.
• Result: Promotes documents that rank highly across multiple, diverse retrieval methods, improving consensus relevance.

A class of algorithms that enable efficient similarity search in high-dimensional vector spaces. Exact nearest neighbor search is computationally prohibitive for large indexes. ANN algorithms trade a small amount of recall for massive speed and memory improvements by searching an approximate, indexed version of the space.

• Common Algorithms:
  • HNSW (Hierarchical Navigable Small World): A graph-based method offering an excellent speed/recall trade-off.
  • IVF (Inverted File Index): Clusters vectors and searches only the nearest clusters.
  • Product Quantization (PQ): Compresses vectors for reduced memory footprint.
• Libraries: Faiss (Facebook AI Research) and HNSWlib provide optimized implementations.

An umbrella term for retrieval systems where the ranking function or representation is trained end-to-end on relevance data, moving beyond hand-crafted functions like BM25. This includes training the embedding models for dense retrieval and end-to-end systems like Retrieval-Augmented Fine-Tuning.

• Goal: Optimize the retriever directly for the downstream task (e.g., question answering).
• Architectures:
  • Dense Passage Retrieval (DPR): Trains a dual encoder using positive and negative passage pairs.
  • ColBERT: Employs late interaction for more expressive, token-level matching.
• Challenge: Requires substantial labeled query-document relevance data for training.