Hybrid Search

The keyword-based backbone of hybrid search, typically powered by BM25 (Best Match 25). This algorithm analyzes term frequency and inverse document frequency to rank documents by exact word matches.

• Excels at matching rare, specific terms like product codes, IDs, and technical jargon
• Handles out-of-domain queries without semantic drift
• Provides predictable, interpretable results based on word overlap
• Fails on synonyms and paraphrased queries where vocabulary differs

Uses transformer-based bi-encoders to map queries and documents into high-dimensional vector space, where proximity represents conceptual similarity.

• Captures synonyms, paraphrases, and cross-lingual meaning
• Retrieves relevant documents even with zero keyword overlap
• Relies on embedding models like text-embedding-3-large or bge-large-en
• May miss exact matches for rare identifiers or domain-specific codes

Combining sparse and dense relevance signals requires careful normalization. Common approaches include:

• Linear combination: Weighted sum of normalized scores (e.g., alpha * sparse_score + (1-alpha) * dense_score)
• Reciprocal Rank Fusion (RRF): Merges ranked lists without requiring score calibration, using the formula 1 / (k + rank) where k is a constant (typically 60)
• Learning-to-rank: A secondary model trained to predict final relevance from both signal types

Hybrid search often serves as the first-stage retriever, passing candidate documents to a more precise cross-encoder for re-ranking.

• First stage: Hybrid retrieval fetches 100-200 candidates with high recall
• Second stage: A cross-encoder like Cohere Rerank or bge-reranker-large jointly processes query-document pairs for fine-grained relevance scoring
• This two-stage architecture balances speed and accuracy, keeping latency under 100ms for production systems

Hybrid search demands databases that support both inverted indexes for sparse retrieval and approximate nearest neighbor (ANN) indexes for dense vectors.

• Elasticsearch with dense vector plugin combines BM25 and kNN search natively
• Weaviate and Pinecone offer native hybrid search APIs with configurable fusion weights
• Milvus supports multi-vector fields for storing both sparse and dense representations
• Indexing latency increases with dual-index maintenance; plan for 2-3x storage overhead

Advanced hybrid systems dynamically adjust the sparse-dense balance based on query type:

• Exact lookup queries (product SKUs, error codes): Weight heavily toward sparse/BM25
• Conceptual queries ("best practices for microservices"): Weight toward dense/semantic
• Mixed queries: Use a lightweight classifier to predict optimal alpha per query
• This adaptive approach prevents semantic drift on precise queries while maintaining recall on broad ones

The keyword-based backbone of hybrid search. BM25 is a probabilistic ranking function that scores documents based on term frequency and inverse document frequency, with a saturation curve that prevents common words from dominating. It excels at exact phrase matching, rare term identification, and queries where vocabulary overlap is critical. Unlike dense methods, BM25 requires no training and is fully deterministic, making it highly interpretable and debuggable.

Encodes queries and documents into high-dimensional vector embeddings using transformer models like BERT or Sentence-T5. Retrieval is performed via approximate nearest neighbor (ANN) search on cosine similarity. This approach captures conceptual meaning and paraphrases, handling synonymy and thematic relevance that lexical methods miss. Key trade-off: requires GPU-accelerated indexing and is opaque compared to BM25.

The most common algorithm for merging sparse and dense result sets into a single ranked list. RRF assigns a score of 1 / (k + rank) to each document in each result set, where k is a constant (typically 60). Documents appearing in both lists accumulate scores, naturally boosting consensus results. No tuning required — it's parameter-free and robust across diverse datasets.

A second-stage refinement applied after hybrid retrieval. A cross-encoder processes the query and each candidate document jointly through a transformer, producing a precise relevance score. This is computationally expensive but dramatically improves precision. Common models include Cohere Rerank, BGE-Reranker, and MonoT5. Typically applied to the top 100-200 candidates from the hybrid fusion step.

Pre-processing techniques that improve hybrid search recall by reformulating the user's original query. Methods include: synonym expansion using WordNet or domain thesauri, paraphrase generation via LLMs, and Hypothetical Document Embeddings (HyDE) where an LLM generates a synthetic answer whose embedding is used for retrieval. Critical for bridging the vocabulary gap between user intent and document language.

Specialized storage systems that index dense embeddings for fast ANN search. Leading options include Pinecone (managed), Weaviate (hybrid-native with BM25 built in), Milvus (distributed), and Qdrant (Rust-based). These databases support metadata filtering alongside vector search, enabling hybrid queries that combine semantic similarity with structured constraints like date ranges or categorical filters.