Bi-Encoder

The defining trait: a query encoder and a document encoder process inputs in complete isolation. There is no cross-attention between the two towers. This means the document tower can pre-compute embeddings for millions of documents offline, while the query tower generates a single vector at runtime. The final similarity is a simple, fast metric like cosine similarity or dot product between the two output vectors.

Because documents are encoded independently, their vectors can be generated in a batch vectorization process and stored in a vector index like HNSW or FAISS. This is the key to low-latency retrieval. When a query arrives, the system only needs to encode the query and perform an Approximate Nearest Neighbor (ANN) search, avoiding the prohibitive cost of re-encoding the entire corpus.

Bi-encoders are typically trained using a contrastive loss function, such as InfoNCE. The model learns to pull the embeddings of a query and its relevant document (positive pair) together in vector space, while pushing apart the embeddings of the query and irrelevant documents (negative pairs). In-batch negative sampling, as used in Dense Passage Retrieval (DPR), is a common technique where other documents in the same training batch serve as negatives.

The independence of the two towers is both a strength and a weakness. A bi-encoder cannot model the nuanced, token-level interactions between a query and a document that a cross-encoder can. This often results in slightly lower top-end relevance. The standard mitigation is a two-stage retrieval pipeline: a fast bi-encoder retrieves a broad set of candidates (e.g., top-100), and a slower cross-encoder re-ranks them for final precision.

In a symmetric bi-encoder, the query and document towers share the same weights, useful when queries and documents have a similar nature (e.g., sentence similarity). In an asymmetric bi-encoder, the towers have separate weights, which is standard for search where queries are short and documents are long. Asymmetric models allow each tower to specialize its representation for its specific input type.

The raw output of a transformer is a sequence of token vectors. To get a single, fixed-size embedding vector for a sentence or document, a pooling strategy is required. Common methods include taking the [CLS] token embedding, mean pooling (averaging all token vectors), or max pooling. Mean pooling with L2 normalization is a widely adopted default for Sentence Transformers, as it provides a robust representation of the entire input.

A neural network, typically a Sentence Transformer, that maps discrete data into a continuous, high-dimensional vector space where semantic similarity is represented by spatial proximity. The embedding model is the core component within both the query encoder and document encoder of a bi-encoder architecture.

• Common architectures: BERT, RoBERTa, T5-based encoders fine-tuned with contrastive loss
• Output: Fixed-length dense vectors (e.g., 768, 1024, or 4096 dimensions)
• Training data: Typically trained on billions of text pairs from natural language inference, paraphrase, and QA datasets
• Domain adaptation: General-purpose models (all-MiniLM-L6-v2) vs. domain-specific (e5, GTE, or Voyage models)

A retrieval strategy that fuses the semantic understanding of dense vector search (bi-encoder) with the precise keyword matching of sparse retrieval algorithms like BM25. This combination compensates for the bi-encoder's weakness on exact term matching, such as part numbers, code snippets, or rare entity names.

• Fusion methods: Reciprocal Rank Fusion (RRF) or linear score combination with normalization
• BM25 strength: Exact lexical matching for out-of-vocabulary terms the embedding model hasn't seen
• Bi-encoder strength: Capturing paraphrases, synonyms, and conceptual similarity
• Production standard: Most enterprise RAG systems use hybrid search as the default retrieval strategy