Bi-Encoder

A re-ranking methodology where a transformer processes the concatenated query and candidate document simultaneously to produce a fine-grained relevance score. Unlike bi-encoders, cross-encoders perform full self-attention across the query-document pair, capturing subtle semantic interactions at the cost of computational latency.

• Precision: Significantly higher than bi-encoder scoring
• Latency: Linear with candidate set size; impractical for full corpus scan
• Typical use: Second-stage refinement of top-k candidates from a bi-encoder
• Architecture: Often a BERT-style model fine-tuned on relevance judgments

A foundational bi-encoder architecture that uses dense vector representations to encode queries and documents independently. DPR demonstrated that dense retrieval could outperform sparse BM25 on open-domain question answering by training on in-batch negatives with a contrastive loss.

• Training objective: Maximize similarity between query and relevant passage while minimizing similarity with all other passages in the batch
• Indexing: Documents are encoded offline and stored in a FAISS index
• Retrieval: Query is encoded online; top-k results found via ANN search
• Key insight: Dense vectors capture semantic similarity beyond lexical overlap

An algorithm that merges multiple ranked result lists into a single ranking by assigning a score based on the reciprocal of each document's rank position. RRF elegantly balances contributions from dense bi-encoder results and sparse BM25 results without requiring score calibration.

• Formula: score(d) = Σ 1 / (k + rank_i(d)) where k is a constant (typically 60)
• Advantage: No need for score normalization or training
• Robustness: Outperforms linear combination when score distributions differ wildly
• Use case: Fusing bi-encoder semantic results with lexical BM25 results in hybrid retrieval

A late interaction retrieval model that computes contextualized token-level embeddings for queries and documents independently, then performs a cheap yet powerful alignment using a MaxSim operation at scoring time. ColBERT bridges the efficiency gap between bi-encoders and the precision of cross-encoders.

• Indexing: Each document token is stored as a vector; documents become matrices
• Scoring: Each query token finds its most similar document token via MaxSim; scores are summed
• Storage cost: Higher than bi-encoders due to token-level indexing
• Advantage: Captures fine-grained term matching without full cross-attention at query time

A class of algorithms that trade a small amount of accuracy for significant speed improvements when finding the closest vectors to a query in high-dimensional embedding spaces. ANN is the runtime backbone that makes bi-encoder retrieval scalable to billions of documents.

• HNSW: Graph-based algorithm with logarithmic time complexity using multi-layered navigable structures
• IVF-PQ: Combines inverted file indexing with product quantization for memory-efficient search
• Trade-off: Recall vs. latency controlled by search parameters like ef_search or nprobe
• Libraries: FAISS, ScaNN, Annoy, NMSLIB

A model compression technique where a smaller, more efficient student model is trained to mimic the output scores or internal representations of a larger, more powerful teacher model. In retrieval, this often involves distilling a cross-encoder's fine-grained relevance scores into a bi-encoder for fast inference.

• Process: Teacher (cross-encoder) scores query-document pairs; student (bi-encoder) learns to replicate those scores
• Benefit: Bi-encoder gains cross-encoder-level precision at bi-encoder speed
• Training data: Can use teacher to label large unlabeled corpora
• Loss function: Often KL divergence between teacher and student score distributions