Dense Passage Retrieval (DPR)

DPR employs two independent BERT-based encoders—one for questions and one for passages—that map text into dense, fixed-size vectors. Unlike cross-encoders that process question-passage pairs jointly, bi-encoders allow offline pre-computation of all passage embeddings, enabling sub-linear retrieval via approximate nearest neighbor search. The question encoder and passage encoder are trained simultaneously using a contrastive loss that maximizes the similarity between relevant pairs while minimizing it for in-batch negatives.

DPR is trained using a contrastive learning objective with in-batch negatives. For each relevant question-passage pair in a training batch, all other passages in the same batch serve as negative examples. This approach is computationally efficient because it reuses passage embeddings already computed for the batch. The loss function is a negative log-likelihood over the softmax of similarity scores, forcing the model to distinguish the true passage from a large pool of distractors without requiring explicit hard-negative mining.

DPR represents a fundamental shift from traditional sparse retrieval methods like BM25 and TF-IDF. Where sparse methods rely on exact lexical overlap and bag-of-words representations, DPR captures semantic similarity in a learned continuous space. Key advantages include:

• Synonym handling: 'automobile' and 'car' map to similar vectors
• Cross-lingual potential: multilingual encoders can align semantically equivalent passages across languages
• Paraphrase robustness: reworded queries retrieve the same relevant passages This semantic capability is critical for entity linking where surface forms vary widely.

In entity linking pipelines, DPR serves as the candidate generation module that retrieves a shortlist of potential knowledge base entries for a given entity mention. The mention and its surrounding context are encoded as the query, while entity descriptions from Wikidata or a proprietary knowledge base form the passage index. DPR typically achieves recall@100 > 90% on standard benchmarks, dramatically reducing the search space for downstream disambiguation models while maintaining high coverage of the correct entity.

Production DPR deployments require strategies for maintaining index freshness as knowledge bases evolve. Common approaches include:

• Full re-indexing: periodic batch recomputation of all passage embeddings when the underlying BERT model is fine-tuned
• Incremental indexing: computing embeddings only for newly added or modified passages and inserting them into the FAISS index
• Versioned indices: maintaining multiple index snapshots to support A/B testing of different encoder checkpoints Efficient index management is essential for entity linking systems that must reflect real-time knowledge base updates.

The dual-tower neural design at the heart of DPR. One encoder ingests the query or entity mention, the other encodes candidate passages independently.

• Enables offline pre-computation of passage embeddings
• Uses dot product or cosine similarity for fast scoring
• Contrast with cross-encoders, which process query-passage pairs jointly

This separation is what makes DPR scalable to millions of documents.

The initial retrieval phase in entity linking that produces a shortlist of possible knowledge base entries. DPR serves as a high-recall candidate generator by retrieving top-k passages via approximate nearest neighbor (ANN) search.

• Replaces sparse methods like TF-IDF or BM25
• Typically retrieves 20–100 candidates for downstream reranking
• Uses FAISS or ScaNN for efficient vector indexing

A training efficiency technique where positive passages for other queries in the same mini-batch are reused as negative examples. This avoids the cost of explicit hard negative mining.

• Dramatically increases the number of negatives per batch
• Works because random passages are likely irrelevant
• Standard practice in original DPR paper from Facebook AI

Enables effective training without a separate negative sampling pipeline.

The process of intentionally selecting negative passages that are superficially similar to the query but factually incorrect. Critical for training robust DPR models.

• Top BM25 results that don't contain the answer
• Passages with high lexical overlap but wrong entities
• Improves the model's ability to distinguish subtle differences

Without hard negatives, the retriever learns only coarse distinctions.

End-to-end deep learning approach where DPR often serves as the retrieval backbone. The full pipeline:

1. DPR retrieves candidate entity descriptions
2. A cross-encoder reranks candidates with fine-grained attention
3. A nil prediction head decides if the entity is out-of-knowledge-base

This architecture replaced traditional feature-engineering pipelines with learned dense representations.

Facebook AI Similarity Search is the predominant vector index used with DPR for production deployment. Key capabilities:

• IVF (Inverted File) indexes for approximate search
• PQ (Product Quantization) for memory-efficient storage
• GPU-accelerated brute-force search for exact results

A typical setup indexes millions of passage embeddings and retrieves top-k in < 50ms.