Dense Passage Retrieval (DPR)

For production-scale retrieval, DPR relies on Approximate Nearest Neighbor (ANN) search libraries. The most common is FAISS (Facebook AI Similarity Search).

How it works with DPR:

1. The Passage Encoder processes all documents in the corpus, generating a dense vector for each.
2. These vectors are indexed using a FAISS data structure (e.g., IVF or HNSW).
3. At query time, the Query Encoder's output vector is used to perform a Maximum Inner Product Search (MIPS) over the FAISS index to find the top-k most similar passage vectors.

This decoupling of encoding and indexing is what makes DPR highly scalable to billion-passage corpora.

A critical efficiency trick in DPR training is in-batch negative sampling. Within a single training batch containing (q1, p1+), (q2, p2+), ... (qB, pB+) pairs, the positive passage for one question is treated as a negative example for all other questions in the batch.

Benefits:

• Computational Efficiency: No need to explicitly sample or encode separate negatives; they are generated from the existing batch.
• Diverse Negatives: Provides a natural set of varied, semi-hard negatives.

This technique, combined with large batch sizes, allows the model to learn a robust embedding space by contrasting each question against many other relevant-but-incorrect passages within a single forward/backward pass.

To improve retrieval precision, especially for difficult questions, DPR uses hard negative mining. These are passages that are semantically related to the question but do not contain the correct answer, forcing the model to learn finer distinctions.

Common mining strategies:

• BM25 Top Results: Passages retrieved by a keyword-based retriever (like BM25) that are highly lexically overlapping but factually incorrect.
• Previous Model Mistakes: Passages that were incorrectly ranked highly by an earlier version of the DPR model.

Including 1-2 hard negatives per training example is shown to significantly boost Recall@k metrics compared to using only random or in-batch negatives.

DPR is often deployed as the first-stage retriever in a two-stage Retrieval-Augmented Generation (RAG) pipeline. Its high recall is complemented by a more accurate cross-encoder reranker.

Typical workflow:

1. DPR (Bi-Encoder): Retrieves a large candidate set (e.g., top 100 passages) very quickly.
2. Cross-Encoder Reranker: Takes the query and each candidate passage, processes them together through a single transformer model, and produces a precise relevance score.
3. Final Ranking: The top 3-5 passages from the reranker are fed to the LLM for answer generation.

This hybrid approach combines the speed of DPR with the precision of cross-encoders, forming the backbone of modern open-domain QA systems.

Dense retrieval is the overarching neural search paradigm that DPR exemplifies. It shifts from traditional sparse retrieval methods (like BM25) to using learned dense vector embeddings to represent semantic meaning. The process involves:

• Encoding queries and documents into a shared, low-dimensional vector space.
• Using a similarity metric (e.g., dot product) to find the closest matches.
• This paradigm enables semantic search, where documents are retrieved based on conceptual relevance, not just keyword overlap.

DPR models are trained using contrastive learning. The objective is to maximize the similarity (dot product) between a question and its relevant positive passage while minimizing similarity with negative passages. Effective negative sampling is critical:

• In-batch negatives: Using other passages in the same training batch as negatives.
• Hard negatives: Retrieving passages that are semantically similar but not correct answers, forcing the model to learn finer distinctions.
• This training teaches the encoder to create a well-structured embedding space where relevant pairs are clustered together.

Maximum Inner Product Search (MIPS) is the precise mathematical retrieval problem DPR solves. After encoding, the relevance score between a query vector q and a passage vector p is computed as their dot product (q · p). The system must find the passages with the highest dot products from a corpus of millions. This is enabled by:

• Vector databases like Pinecone or Weaviate.
• Approximate Nearest Neighbor (ANN) libraries like Faiss or HNSW indexes, which trade minimal accuracy for massive speedups over brute-force search.

Retrieval-Augmented Generation (RAG) is the primary application architecture that utilizes DPR. In RAG, DPR acts as the retriever component:

1. A user query is encoded by the DPR question encoder.
2. DPR retrieves the top-K most relevant passages from a knowledge source (e.g., a vector database).
3. These passages are fed as context to a large language model (LLM) generator.
4. The LLM produces a final answer grounded in the retrieved evidence. This combination provides factual accuracy and reduces hallucinations.