Dense Passage Retrieval (DPR)

The core of DPR is a dual-encoder (or bi-encoder) architecture. It uses two separate transformer-based encoders: a question encoder and a passage encoder. These encoders independently process the input text, outputting fixed-dimensional dense vector embeddings (e.g., 768 dimensions). The relevance of a passage to a question is determined by the dot product or cosine similarity of their respective embeddings. This design enables highly efficient retrieval, as all passage embeddings can be pre-computed and indexed, allowing for fast approximate nearest neighbor (ANN) search at query time.

Unlike traditional systems using fixed algorithms like BM25, DPR is trained end-to-end on question-passage relevance pairs. The standard training objective is to maximize the similarity score between a question and its corresponding positive passage (the "gold" answer context) while minimizing its similarity to irrelevant (negative) passages. This is typically done using a contrastive loss function, such as negative log-likelihood. By learning from data, DPR's encoders are optimized to map semantically related questions and passages close together in the vector space, even if they share few exact keywords.

A key innovation in DPR's training is the use of in-batch negative sampling. During training, for each positive (question, passage) pair in a mini-batch, the passages associated with other questions in the same batch are treated as negatives. This provides a diverse and computationally efficient set of hard negatives without requiring explicit negative mining in the early stages. This technique helps the model learn fine-grained distinctions and improves retrieval precision by pushing apart embeddings of semantically dissimilar pairs.

For production deployment, DPR relies on dense vector indexing and Approximate Nearest Neighbor (ANN) search. After training, the passage encoder processes the entire corpus, generating a vector database (index) of all passage embeddings. At query time, the question encoder generates a query embedding, which is used to search this index. ANN libraries like FAISS, using algorithms such as HNSW or IVF, enable sub-linear search times over millions of vectors, making semantic search scalable. This contrasts with sparse retrieval's reliance on an inverted index for term matching.

DPR's primary advantage is enabling true semantic search. Because it uses dense embeddings from deep neural networks, it can retrieve passages based on conceptual meaning and paraphrasing, not just lexical overlap. For example, a query for "methods to reduce model size" can effectively retrieve a passage discussing "model compression techniques like pruning and quantization," even with no keyword matches. This addresses the vocabulary mismatch problem inherent in keyword-based systems, significantly improving recall for complex, natural language queries.

DPR is a foundational first-stage retriever in modern Retrieval-Augmented Generation (RAG) systems. Its role is to efficiently scan a large knowledge base (e.g., enterprise documents) and fetch a small set of top-K candidate passages most relevant to a user's query. These passages are then passed as context to a large language model (LLM) for answer generation. DPR's accuracy is critical for RAG performance, as errors in retrieval (missing the correct document) cannot be corrected by the LLM, leading to potential hallucinations. It is often used in hybrid retrieval setups combined with BM25 to balance semantic and lexical recall.

A dual encoder is the neural architecture underpinning DPR. It consists of two separate transformer-based encoders—one for the query and one for the passage—that independently map inputs into a shared dense vector space. This design enables:

• Independent encoding of queries and documents, allowing for pre-computation and indexing of all document vectors.
• Efficient similarity search via a simple dot product or cosine similarity between the query vector and millions of pre-computed passage vectors.
• Contrast with cross-encoders, which jointly process query-document pairs for higher accuracy but are too slow for first-stage retrieval.

Sentence-BERT is a seminal method for generating high-quality sentence embeddings, directly enabling models like DPR. It modifies the BERT architecture using siamese and triplet networks to produce embeddings where semantic similarity is reflected by vector proximity (e.g., cosine similarity).

Key innovations include:

• Pooling strategies (e.g., mean pooling of output embeddings) to derive a fixed-sized sentence representation.
• Training on natural language inference (NLI) and paraphrase datasets to learn semantic textual similarity.
• Providing a practical, efficient model for creating the embeddings used in dense retrieval pipelines.

ANN search is the algorithmic backbone that makes querying billion-scale vector indexes feasible. Since exact nearest neighbor search is computationally prohibitive in high dimensions, ANN algorithms trade a small amount of recall for massive speed gains.

Common algorithms used with DPR vector indexes include:

• HNSW (Hierarchical Navigable Small World): A graph-based method offering an excellent recall/speed trade-off.
• IVF (Inverted File Index): Partitions the vector space into clusters and searches only the most promising ones.
• These algorithms, implemented in libraries like Faiss and Milvus, allow sub-second retrieval from corpora containing millions of DPR embeddings.

A cross-encoder is a precision-oriented model used in a two-stage retrieve-and-rerank architecture, where DPR often serves as the first-stage retriever. Unlike the independent encoders in DPR, a cross-encoder jointly processes a query and a single candidate document.

This architecture provides:

• Higher accuracy: The model can perform deep, attention-based interaction between the query and document text.
• Computational cost: It is too slow to run against an entire corpus but is highly effective for reordering the top 100-200 candidates retrieved by a fast model like DPR.
• This combination leverages DPR's high recall and the cross-encoder's high precision for optimal end-to-end retrieval quality.

Learned retrieval is the paradigm where retrieval models are trained end-to-end on relevance signals, as opposed to using static, rule-based functions like BM25. DPR is a prime example of learned dense retrieval.

Its training involves:

• Contrastive learning: Using positive (relevant) and negative (irrelevant) passage pairs for a given query.
• Negative mining: Using in-batch negatives or hard negatives mined from a first-stage retriever to improve the model's discriminative power.
• This data-driven approach allows the retriever to learn domain-specific and semantic notions of relevance that are impossible to encode in lexical algorithms.

Hybrid retrieval combines the strengths of dense retrieval (like DPR) and sparse retrieval (like BM25) to improve overall recall and precision. DPR excels at semantic matching but can fail on exact keyword or rare entity matching, where sparse methods thrive.

A hybrid system typically:

• Runs DPR and BM25 in parallel.
• Fuses the two ranked result lists using techniques like Reciprocal Rank Fusion (RRF) or weighted score combination.
• This approach mitigates the individual weaknesses of each method, providing robust performance across diverse query types, from keyword-centric to conceptual.