Dense Passage Retrieval (DPR)

DPR employs a bi-encoder design with two independent BERT-based encoders: one for queries and one for passages. This asymmetric search setup allows passage embeddings to be pre-computed and indexed offline, while query embeddings are generated at runtime. The final representation is derived by applying mean pooling over the output token vectors, producing a fixed-size dense vector. This separation is critical for low-latency retrieval, as it reduces the online computation to a single forward pass through the query encoder followed by a fast Maximum Inner Product Search (MIPS).

DPR is trained using a contrastive loss function, specifically a negative log-likelihood over a mini-batch. The key innovation is the use of in-batch negatives: for a batch of B query-passage pairs, the other B-1 passages are treated as negative examples for each query. This dramatically increases training efficiency without requiring explicit negative mining. The model is optimized to maximize the similarity of the correct pair while minimizing it for all others, effectively pulling relevant pairs together and pushing irrelevant ones apart in the embedding space.

While in-batch negatives provide a strong training signal, DPR's accuracy is significantly boosted by incorporating hard negatives. These are passages that are superficially similar to the query—often retrieved by a sparse model like BM25—but do not contain the answer. Training with hard negatives forces the model to learn finer-grained semantic distinctions, improving its ability to reject near-matches. Without them, the retriever may default to simple lexical overlap rather than true semantic understanding.

To further refine retrieval quality, DPR can be enhanced through cross-encoder distillation. A computationally expensive cross-encoder—which processes query-passage pairs jointly with full attention—acts as a teacher. Its relevance scores are used to train the faster bi-encoder student model. This transfers the deep semantic reasoning capability of the cross-encoder into the efficient dual-encoder architecture, yielding a retriever that is both fast and highly accurate. The student learns to mimic the teacher's ranking without the prohibitive inference cost.

During training, DPR often uses a momentum encoder—a slowly updating copy of the query encoder—to maintain a consistent representation space for the passage index. This prevents the rapid parameter drift that can make cached passage embeddings stale between training steps. The momentum encoder's weights are updated as an exponential moving average of the main encoder's weights, ensuring that the large queue of negative samples remains a reliable training signal throughout the optimization process.

In production, DPR relies on FAISS (Facebook AI Similarity Search) for billion-scale vector search. Passage embeddings are indexed using algorithms like Hierarchical Navigable Small World (HNSW) graphs or Inverted File Index (IVF) with Product Quantization (PQ) for compression. This enables sub-linear search times: instead of comparing a query against every passage, the index intelligently navigates to the most promising regions of the embedding space. The result is a system that can retrieve the top-K passages from millions of documents in milliseconds.

The foundational dual-encoder design where a query encoder and a passage encoder independently map text to dense vectors. This asymmetric search setup allows passage embeddings to be pre-computed and indexed offline, while queries are encoded on-the-fly.

• Query encoder: Maps user input to a d-dimensional vector
• Passage encoder: Maps document chunks to the same vector space
• Enables Maximum Inner Product Search (MIPS) for fast retrieval

DPR is trained using a contrastive loss function that pulls relevant query-passage pairs together while pushing irrelevant pairs apart in the embedding space. The training leverages in-batch negatives—other positive passages in the same mini-batch reused as negative examples—to dramatically increase training efficiency without explicit negative sampling.

• Positive pairs: (query, relevant passage)
• Hard negatives: Superficially similar but irrelevant passages
• Momentum encoder: Stabilizes representation learning with large negative queues

Once passages are encoded, they must be indexed for efficient retrieval. Approximate Nearest Neighbor (ANN) algorithms like HNSW and IVF trade marginal accuracy for massive speed gains. Libraries such as FAISS provide production-grade implementations.

• Product Quantization (PQ): Compresses vectors to reduce memory
• Inverted File Index (IVF): Clusters embeddings and searches only nearby partitions
• Typical index sizes: Billions of vectors with sub-10ms latency

A powerful cross-encoder can re-rank retrieved passages with full query-document attention, but is too slow for first-pass retrieval. Cross-encoder distillation transfers this scoring knowledge to a fast bi-encoder. The cross-encoder acts as a teacher, generating soft labels that train the student DPR model to mimic the more accurate relevance assessments.

• Teacher: Cross-encoder with full attention
• Student: Bi-encoder optimized for speed
• Result: Near cross-encoder accuracy at bi-encoder speed

Standard DPR produces a single vector per passage, which can lose fine-grained token-level information. Late interaction models like ColBERT store multiple vectors per passage—one per token—and perform a lightweight MaxSim operation at query time. This bridges the gap between bi-encoder efficiency and cross-encoder expressiveness.

• Multi-vector encoding: One embedding per token
• MaxSim: Computes maximum cosine similarity per query token
• Balances storage cost with retrieval precision

Dense retrieval excels at semantic matching but can miss exact keyword matches. Sparse-dense hybrid systems combine DPR with traditional BM25 scoring using fusion techniques like Reciprocal Rank Fusion (RRF). This ensures both conceptual relevance and lexical precision.

• Dense: Captures paraphrases and semantic intent
• Sparse: Handles rare terms, codes, and exact phrases
• Fusion: Weighted combination or reciprocal rank merging