Bi-Encoder

The defining feature of a bi-encoder is its two separate, identical encoder networks (often BERT-based) that process the query and the document in complete isolation. There is no cross-attention between the query and document tokens during encoding. This allows the document embeddings to be pre-computed offline and stored in a vector database, while only the query embedding is computed at runtime. This architectural choice trades a small amount of accuracy for massive gains in retrieval speed.

Because the document encoder operates independently of the query, every document in a corpus can be transformed into a fixed-size dense vector once and indexed. This is the key to bi-encoder efficiency. When a new query arrives, it is encoded and compared against the pre-built index using a fast similarity metric like cosine similarity or dot product. This process reduces the search complexity from linear to sub-linear, making it suitable for billion-scale corpora.

Unlike sparse lexical methods like BM25, bi-encoders perform dense retrieval. They map semantically similar texts to nearby points in a high-dimensional vector space, even if they share no keywords. For example, a query for 'automobile repair' can retrieve a document about 'fixing a car'. This is achieved through contrastive training on pairs of related and unrelated texts, teaching the model to pull relevant pairs together and push non-relevant pairs apart.

Bi-encoders are typically trained using a contrastive loss function, such as InfoNCE or Multiple Negatives Ranking Loss. The model is presented with positive pairs (a query and its relevant document) and negative pairs (a query and non-relevant documents). The training objective minimizes the distance between positive pairs while maximizing the distance to negative pairs within a batch. This process shapes the embedding space to be highly effective for nearest-neighbor search.

In modern retrieval pipelines, the bi-encoder serves as the first-stage retriever. Its job is to efficiently narrow down a massive corpus of millions or billions of documents to a small set of top-k candidates (e.g., 100-1000). This candidate set is then passed to a more powerful but computationally expensive second-stage re-ranker, such as a cross-encoder, which performs joint query-document analysis to produce a highly precise final ordering.

The core design trade-off of a bi-encoder is between computational efficiency and representational power. By preventing the query and document from interacting in the encoder, the model loses the ability to capture fine-grained, token-level relationships. This can cause it to miss relevant documents where the meaning depends on specific word interactions. However, this sacrifice is what enables the pre-computation of document embeddings and the resulting millisecond-level search latency over massive datasets.

A deep learning architecture that processes a query and document jointly through a transformer network to produce a highly accurate relevance score. Unlike bi-encoders, cross-encoders apply full self-attention across both inputs simultaneously, enabling rich token-level interactions.

• Role: Second-stage re-ranking of top-k candidates from a bi-encoder
• Trade-off: 10-100x slower than bi-encoders but significantly more precise
• Example: BERT-based cross-encoder scoring 100 candidate passages after bi-encoder retrieval

A late interaction retrieval model that bridges the speed-accuracy gap between bi-encoders and cross-encoders. It encodes queries and documents into sets of token-level embeddings, then computes relevance via a fast, fine-grained similarity summation (MaxSim) without full joint processing.

• Key innovation: Stores per-token document embeddings for efficient nearest-neighbor search
• Advantage: Preserves token-level detail while enabling pre-computation like bi-encoders
• Use case: High-recall retrieval where exact phrase matching matters

A second-stage process in information retrieval where a more computationally intensive and precise model re-scores an initial set of candidate documents. The bi-encoder handles the fast, approximate first pass over millions of documents, while the re-ranker refines the top 50-200 results.

• Pipeline: Bi-encoder retrieval → Re-ranker scoring → Final ranked list
• Common re-rankers: Cross-encoders, ColBERT, or LLM-based scoring
• Latency budget: Typically 50-200ms for re-ranking vs. <10ms for bi-encoder retrieval

A foundational bi-encoder framework that uses separate BERT-based encoders for queries and passages, trained with contrastive learning on positive/negative passage pairs. DPR demonstrated that dense embeddings could outperform traditional sparse retrieval like BM25.

• Training: In-batch negatives with hard negative mining
• Architecture: Query encoder + Passage encoder, both producing fixed-size vectors
• Impact: Established the bi-encoder as the standard first-stage retriever for RAG systems

A retrieval evaluation metric that measures how quickly a system returns the first relevant result. MRR averages the reciprocal of the rank position of the first correct document across all queries.

• Formula: MRR = (1/|Q|) × Σ(1/rank_i)
• Range: 0 to 1, where 1 means the first result is always relevant
• Relevance to bi-encoders: Primary metric for evaluating first-stage retrieval quality before re-ranking
• Example: If the first relevant doc is at rank 3, reciprocal rank = 1/3 ≈ 0.33

A ranking quality metric that evaluates the full ordered list of results, giving higher weight to relevant documents appearing earlier. NDCG normalizes against an ideal ranking to produce a score between 0 and 1.

• Key feature: Handles graded relevance (not just binary relevant/not)
• Discount function: log₂(rank + 1) reduces contribution of lower-ranked items
• Use in bi-encoder pipelines: Evaluates end-to-end retrieval quality including re-ranking stages