Bi-Encoder

The defining feature: a bi-encoder uses two separate, independently parameterized encoders—one for the query and one for the document. This allows documents to be encoded offline and stored in a vector index. At query time, only the query needs to be encoded, enabling sub-linear search times.

• Query Encoder: Processes the user's search query.
• Document Encoder: Processes all passages in the corpus, typically as a batch preprocessing step.

Unlike a cross-encoder, a bi-encoder defers all interaction between the query and the document until the final scoring step. The two texts are encoded into dense vectors completely independently. Their semantic relationship is then computed as a single, fast mathematical operation—typically cosine similarity or a dot product—on the resulting fixed-size embeddings.

A transformer model outputs a vector for every input token. To create a single, fixed-size passage embedding, a pooling strategy must be applied. The choice of strategy significantly impacts retrieval quality.

• Mean Pooling: Averages all token vectors; provides a stable, general-purpose representation.
• [CLS] Token: Uses the output vector of the special classification token; its effectiveness depends on how the model was pre-trained.
• Max Pooling: Takes the maximum value over time for each dimension.

Bi-encoders are architected specifically for MIPS. Because the query and document are independent vectors, the retrieval task is reduced to finding the document vectors that maximize the dot product with the query vector. This mathematical equivalence allows the use of highly optimized Approximate Nearest Neighbor (ANN) libraries like FAISS, which can search billion-scale vector indexes in milliseconds.

The query and document encoders do not need to share weights. This asymmetric design allows for optimization. A common pattern is to use a deeper, more powerful model for the shorter query and a slightly shallower, faster model for encoding the massive document corpus. This balances the need for a nuanced query understanding with the computational cost of indexing millions of passages.

Bi-encoders are trained using contrastive loss functions. The model learns by pulling the embeddings of a query and its relevant document closer together in the vector space while simultaneously pushing irrelevant documents apart. Key training techniques include:

• In-Batch Negatives: Reusing other positive documents in the mini-batch as negative examples for free.
• Hard Negatives: Using passages that are top-ranked by a sparse retriever (like BM25) but are not relevant, forcing the model to learn fine-grained distinctions.

The canonical bi-encoder implementation that demonstrated dense retrieval outperforming BM25 for open-domain QA. DPR uses two independent BERT encoders—one for queries, one for passages—trained with in-batch negatives and a contrastive loss function. The system encodes all Wikipedia passages offline into a FAISS index, then performs Maximum Inner Product Search (MIPS) at query time.

• Query encoder and passage encoder share no parameters
• Trained on question-passage pairs with hard negatives mined via BM25
• Achieves 79.4% top-20 accuracy on Natural Questions
• Indexes 21 million passages for sub-second retrieval

A practical bi-encoder adaptation that makes BERT viable for semantic similarity search by adding a mean pooling layer on top of token embeddings. Unlike vanilla BERT—which requires both sentences fed together and is too slow for retrieval—SBERT encodes sentences independently into fixed-size vectors. These vectors can be compared using cosine similarity and indexed in vector databases.

• Siamese network structure with tied weights for symmetric tasks
• Supports classification, regression, and triplet objective functions
• Reduces finding the most similar sentence pair from 65 hours to ~5 seconds
• Pre-trained models available via the sentence-transformers library

A bi-encoder variant that stores token-level embeddings rather than a single pooled vector. ColBERT encodes queries and documents into sets of vectors, then computes relevance via a MaxSim operation—summing the maximum cosine similarity for each query token against all document tokens. This preserves fine-grained lexical matching while retaining the indexing speed of a bi-encoder.

• Multi-vector encoding: each token gets its own embedding
• Late interaction defers cross-attention to a lightweight scoring step
• Achieves strong zero-shot retrieval without task-specific fine-tuning
• Used in production by companies requiring high-recall semantic search

A hosted bi-encoder service where the text-embedding-3-large model encodes text into vectors of up to 3,072 dimensions. The encoder processes input independently, allowing users to pre-compute document embeddings and store them in vector databases like Pinecone or Weaviate. Query embeddings are generated on-the-fly and compared via cosine similarity.

• Supports truncation to smaller dimensions (256, 512, 1024) with minimal accuracy loss
• Matryoshka representation learning enables flexible dimensionality
• Powers retrieval-augmented generation (RAG) across thousands of applications
• Stateless encoder design: no shared parameters between query and document passes

A multilingual bi-encoder supporting 8K token context windows—far exceeding the typical 512-token limit. Jina Embeddings v2 encodes documents up to ~8,000 tokens into a single dense vector using mean pooling with position-aware attention. This eliminates the need for aggressive chunking and enables long-document semantic search.

• Supports English, Chinese, German, and Spanish in a single model
• 8,192 token context window for encoding full articles or legal documents
• Outputs 768-dimensional embeddings compatible with any vector database
• Open-source under Apache 2.0 license

A bi-encoder designed for enterprise retrieval with native support for semantic chunking and input type specification. Embed v3 accepts a input_type parameter (search_document, search_query, classification, clustering) that adjusts encoding behavior. For search, the query and document encoders operate asymmetrically, optimizing the embedding space for retrieval rather than symmetric similarity.

• Asymmetric encoding tuned specifically for query-document relevance
• Compression-aware training maintains accuracy even with binary quantization
• Supports 100+ languages with strong cross-lingual retrieval
• Integrates with major vector databases via standardized API

The high-dimensional vector space where a bi-encoder maps queries and documents. Semantically similar text is positioned at proximate points, enabling retrieval via distance metrics. The quality of this space—its isotropy and density—directly determines retrieval accuracy. A well-trained bi-encoder produces an embedding space where cosine similarity correlates with human relevance judgments.

The core algorithmic task that makes bi-encoders practical. Once a query is encoded, MIPS finds the document vector with the highest dot product without scanning the entire index. This is what separates the bi-encoder's offline indexing advantage from a cross-encoder's exhaustive approach. Efficient MIPS is the reason bi-encoders scale to billion-document corpora.

The training objective that teaches a bi-encoder to structure its embedding space. It operates on paired examples, pulling a query and its relevant passage together while pushing irrelevant passages apart. Variants include:

• Triplet Loss: Uses an anchor, positive, and negative
• InfoNCE Loss: A multi-class formulation using in-batch negatives
• Multiple Negatives Ranking Loss: Optimizes for the correct pair among many negatives

A training strategy that transfers knowledge from a computationally expensive cross-encoder to a fast bi-encoder. The cross-encoder scores query-document pairs with full attention, producing highly accurate relevance labels. The bi-encoder is then trained to mimic these scores, achieving near cross-encoder accuracy with bi-encoder speed. This is the primary method for building state-of-the-art retrievers.

The class of algorithms that make vector search tractable at scale. Rather than exact nearest neighbor search—which is O(N) in complexity—ANN trades a small accuracy loss for orders-of-magnitude speedups. Key algorithms include HNSW (graph-based), IVF (clustering-based), and PQ (compression-based). Libraries like FAISS and ScaNN provide production-ready implementations.

A retrieval paradigm that sits between bi-encoders and cross-encoders. Instead of a single vector per passage, ColBERT stores one embedding per token. At query time, it computes a MaxSim operation between query tokens and document tokens. This preserves more fine-grained semantic matching than a bi-encoder while remaining far faster than a cross-encoder. It represents the state-of-the-art in accuracy-efficiency tradeoffs.