Passage Embedding

Regardless of the input passage length—whether a single sentence or a 512-token paragraph—the encoder compresses it into a fixed-size dense vector (e.g., 768 dimensions for BERT-base). This property is critical for efficient indexing and comparison:

• Every passage maps to a point in the same high-dimensional embedding space
• Enables constant-time similarity computation via dot product or cosine similarity
• Typical dimensionalities range from 384 (MiniLM) to 4096 (large proprietary models)
• Unlike sparse bag-of-words vectors, most dimensions carry non-zero values, hence "dense"

Passage embeddings are not simple word averages. A transformer encoder processes the entire sequence with full self-attention, producing context-aware token representations. These are then aggregated into a single vector:

• Mean pooling: Averages all output token vectors (common in Sentence-BERT)
• [CLS] token pooling: Uses the special classification token's representation (standard BERT)
• Max pooling: Takes the element-wise maximum across token dimensions
• The aggregation strategy significantly impacts retrieval quality and must align with training objectives

The defining property of a well-trained passage embedding is that semantically similar passages cluster together in the vector space. This is achieved through contrastive training:

• "How do I reset my password?" and "Password recovery instructions" map to nearby points
• "Apple releases new iPhone" and "Apple pie recipe" map to distant regions despite lexical overlap
• The cosine similarity between two passage vectors quantifies their semantic relatedness
• This property enables retrieval based on intent and meaning rather than keyword matching

Passage embeddings are designed to be stored in vector indexes that support efficient similarity search at massive scale. Key indexability characteristics:

• Vectors are stored in specialized data structures like HNSW graphs or IVF inverted files
• Product Quantization (PQ) compresses embeddings to reduce memory footprint by 10-30x
• FAISS and ScaNN libraries enable sub-millisecond retrieval over billions of passages
• The fixed dimensionality ensures predictable memory requirements: 1 billion 768-dim float32 vectors = ~3 TB

Passage embeddings are not interchangeable across encoders. Each model produces vectors in its own unique representation space with distinct geometric properties:

• Embeddings from all-MiniLM-L6-v2 cannot be directly compared to those from text-embedding-3-large
• The same passage encoded by different models yields vectors with no meaningful similarity
• Cross-encoder models produce passage representations optimized for pairwise scoring, not standalone indexing
• Production systems must maintain encoder consistency across indexing and querying pipelines

In Dense Passage Retrieval, queries and passages are processed by separate encoders with distinct parameterizations. This asymmetry is intentional:

• The query encoder is optimized for short, often incomplete natural language questions
• The passage encoder is optimized for longer, structured document chunks
• Both map into the same shared embedding space for direct similarity comparison
• This design enables the passage index to be pre-computed offline while queries are encoded in real-time at sub-millisecond latency

The foundational dual-encoder design that independently maps queries and passages into a shared embedding space. Unlike cross-encoders, bi-encoders allow passages to be pre-computed and indexed offline, enabling sub-linear retrieval over millions of documents. The query encoder and passage encoder may share weights or be independently parameterized for asymmetric search scenarios.

The objective function that teaches embedding models to produce meaningful representations. Contrastive loss pulls positive query-passage pairs together while pushing negative pairs apart in vector space. Key training strategies include:

• In-batch negatives: Reuse other positives in the mini-batch as negatives
• Hard negatives: Use superficially similar but irrelevant passages to sharpen discrimination
• Momentum encoders: Maintain a slowly updating copy of the encoder for consistent negative representations

ANN algorithms trade a small amount of accuracy for dramatic speedups when searching billion-scale vector indices. The most widely adopted method is Hierarchical Navigable Small World (HNSW), which builds a multi-layered graph structure for logarithmic-time traversal. Alternative approaches include Inverted File Index (IVF) with clustering and Product Quantization (PQ) for memory-efficient compressed storage.

A training technique where a computationally expensive cross-encoder—which processes query-passage pairs with full attention—teaches a fast bi-encoder to improve its retrieval accuracy. The cross-encoder provides soft relevance labels that capture nuanced semantic relationships. The distilled bi-encoder retains much of the teacher's precision while maintaining the speed required for production retrieval.

A retrieval paradigm exemplified by ColBERT that bridges the efficiency gap between bi-encoders and cross-encoders. Instead of pooling all tokens into a single vector, late interaction stores multi-vector encodings—one embedding per token—and computes relevance via a MaxSim operation at query time. This preserves fine-grained token-level matching while avoiding the quadratic cost of full cross-attention.

A production retrieval strategy that combines the complementary strengths of BM25 keyword matching and dense vector search. Sparse retrieval excels at exact term matching and rare entities, while dense retrieval captures semantic similarity and paraphrasing. Results are fused using Reciprocal Rank Fusion (RRF) or learned weighting schemes to maximize both precision and recall.