Dense Passage Retrieval (DPR)

A fundamental efficiency feature of DPR is the decoupling of query and passage encoding. Since passages are encoded independently of any query, their embeddings can be computed offline and indexed once. During retrieval, only the query needs to be encoded in real-time. This results in:

• Millisecond-level latency for search over millions of passages.
• Scalability that is linear with the number of passages for indexing, but constant for query processing.
• A clear separation from cross-encoder re-rankers, which are more accurate but far slower, as they process the query and passage together.

DPR excels at semantic matching, understanding conceptual relevance beyond keyword overlap. This solves key limitations of sparse retrievers like BM25:

• Vocabulary mismatch: It can match "automobile" to "car".
• Contextual understanding: It distinguishes between "Apple the company" and "apple the fruit" based on surrounding context in the query and passage.
• Phrasal and relational queries: It can handle queries like "Who invented the light bulb?" even if the passage states "Thomas Edison's invention of the incandescent lamp." However, for precise keyword or entity-name matching, hybrid search combining DPR with BM25 often yields the best results.

DPR is the de facto standard retriever in the first stage of modern Retrieval-Augmented Generation (RAG) pipelines. Its role is to efficiently filter a massive knowledge corpus (e.g., millions of documents) down to a handful (e.g., 10-100) of the most semantically relevant passages or context chunks. These are then fed into a large language model for answer generation or synthesis. The quality of DPR's retrieval directly determines the upper bound of accuracy for the entire RAG system, as the LLM can only reason over the provided context.

DPR occupies a specific point in the retrieval accuracy/efficiency trade-off spectrum:

• vs. Sparse Retrievers (BM25): More semantically accurate but requires pre-computed index and model inference.
• vs. Late-Interaction Models (ColBERT): ColBERT stores per-token embeddings and uses a more expressive MaxSim operator, achieving higher accuracy than DPR but with a significantly larger index size and slightly slower retrieval.
• vs. Cross-Encoders: Cross-encoders (like MonoT5) process the query and passage together through the transformer, achieving the highest accuracy for re-ranking but are far too slow for scanning a large corpus. DPR is typically used for first-stage retrieval, followed by a cross-encoder re-ranker.

An information retrieval strategy that combines the results of sparse retrieval (e.g., keyword-based BM25) and dense retrieval (e.g., DPR or SBERT vector similarity). This approach leverages the complementary strengths of both methods: sparse retrieval's precision for exact term matching and dense retrieval's ability to capture semantic meaning and synonyms. The scores from each method are normalized and fused, often using a weighted sum, to produce a final ranked list.

• Key Benefit: Mitigates the weaknesses of pure semantic search, such as missing on precise keywords or technical jargon.
• Implementation: Requires maintaining both a traditional inverted index (for sparse search) and a dense vector index.
• Use Case: Enterprise search where documents contain a mix of precise terminology and natural language descriptions.

A neural retrieval model that provides a efficient and effective middle ground between traditional sparse retrieval and dense vector retrieval like DPR. ColBERT computes contextualized embeddings for every token in both the query and the document using BERT. Relevance is scored via a late interaction mechanism (specifically, MaxSim), which allows every query token to interact with every document token. This provides deeper interaction than DPR's single-vector comparison but remains more scalable than cross-encoders that require full pairwise processing.

• Architecture: Encodes queries and documents independently (like DPR) but produces per-token vectors.
• Late Interaction: Similarity is computed as the sum of maximum similarity scores between each query token embedding and all document token embeddings.
• Trade-off: Offers higher accuracy than single-vector DPR but with increased storage and computational cost for scoring.

A probabilistic ranking function and the de facto standard algorithm for keyword-based (sparse) information retrieval. It ranks documents based on the presence of query terms, incorporating term frequency (TF) and inverse document frequency (IDF) with built-in normalization for document length. BM25 does not require machine learning and is based on statistical properties of the text corpus.

• Contrast with DPR: BM25 relies on exact lexical matches, while DPR captures semantic similarity. BM25 cannot match on synonyms or related concepts without explicit query expansion.
• Role in Hybrid Systems: Serves as the sparse component in hybrid search architectures, complementing DPR's semantic capabilities.
• Characteristics: Highly efficient, explainable (results are tied to specific terms), and effective for keyword-centric queries.

The fundamental data structure that enables fast full-text search in traditional search engines and forms the backbone of sparse retrieval methods like BM25. It maps each unique term (or token) in a corpus to a postings list—a record of all documents containing that term and often the positions within those documents. Retrieval involves looking up query terms in this index and intersecting their postings lists to find candidate documents.

• Core Mechanism: Enables sub-linear search time relative to corpus size.
• Comparison to Vector Index: An inverted index is optimized for exact term lookup, while a dense vector index is optimized for similarity search in a continuous vector space.
• Modern Relevance: Remains critical for metadata filtering, keyword boosting, and as a component in multi-stage retrieval pipelines where it provides a fast initial candidate set for more expensive re-ranking models.