Late Interaction Model

A neural retrieval architecture where the query and document are encoded independently into fixed-dimensional vector embeddings using separate transformer models or a shared encoder. Relevance is computed via a simple, fast similarity measure (e.g., cosine similarity) between the two embeddings.

• Key Advantage: Enables pre-computation and indexing of document embeddings, leading to millisecond-level retrieval from large corpora via Approximate Nearest Neighbor (ANN) search.
• Key Limitation: The single-vector representation can struggle to capture fine-grained, multi-faceted semantic matches, a problem known as the representation bottleneck.

A neural reranking model that takes a query and document pair as a single concatenated input sequence. A transformer encoder then applies full self-attention across the entire combined sequence, enabling deep, token-level interaction to produce a direct relevance score.

• Key Advantage: Achieves state-of-the-art ranking accuracy by modeling the nuanced interplay between every query token and every document token.
• Key Limitation: Suffers from quadratic computational complexity (O(n²)) in sequence length, making it too slow for first-stage retrieval. It is typically used as a reranker on a small candidate set (e.g., top 100 documents) from a faster retriever.

The seminal late interaction model that introduced the architecture. ColBERT uses a shared BERT-based encoder to produce contextualized token-level embeddings for the query and document separately. Relevance is computed via a MaxSim operator: for each query token, find its maximum cosine similarity with any document token, then sum these maximums.

• Mechanism: This "late" interaction happens after encoding, avoiding the quadratic cost of cross-attention during encoding but allowing a richer match than a single vector.
• Practical Use: Enables efficient retrieval via a FAISS index over document token embeddings, followed by the late interaction scoring, balancing effectiveness and speed.

The production pipeline architecture where retrieval is broken into sequential, cascading stages of increasing accuracy and computational cost. The late interaction model typically serves in an intermediate stage.

• Stage 1 (Recall): A fast, high-recall retriever (e.g., BM25 or a bi-encoder) fetches a large candidate set (e.g., 1000 documents).
• Stage 2 (Reranking): A more accurate, computationally intensive model (e.g., a late interaction model or a cross-encoder) scores and reorders the top candidates (e.g., 100 documents).
• Stage 3 (Precision): An optional, very heavy model (e.g., a large cross-encoder) can be applied to the top 10-20 candidates for final precision.

A class of transformer model architectures designed to overcome the quadratic complexity bottleneck of standard self-attention. These are relevant as alternatives or components for scaling late interaction and cross-encoder models to longer sequences.

• Examples: Models like Longformer, BigBird, and Reformer use patterns like sliding window attention, global tokens, or locality-sensitive hashing to reduce complexity to linear or near-linear in sequence length.
• Application: Can be used to build cross-encoders capable of processing longer documents or to create more efficient variants of late interaction models that encode longer context windows.

The technique of training a smaller, faster student model to mimic the ranking behavior of a larger, more accurate teacher model. This is crucial for deploying effective late interaction or cross-encoder models in latency-sensitive environments.

• Process: A powerful but slow teacher model (e.g., a 440M parameter cross-encoder) generates soft labels (scores or rankings) for a dataset. A smaller student model (e.g., a 110M parameter late interaction model) is then trained on these labels.
• Outcome: The student model achieves comparable accuracy to the teacher at a fraction of the inference latency and cost, making advanced reranking feasible in production.