Hybrid Search

The dense retriever handles semantic search by encoding queries and documents into high-dimensional vector embeddings. It finds conceptually similar content even when keywords don't match. This component relies on:

• An embedding model (e.g., OpenAI's text-embedding-3-small, BGE, E5) to generate the vectors.
• An Approximate Nearest Neighbor (ANN) index (e.g., HNSW, IVF) within a vector database for fast similarity search.
• A similarity metric like cosine similarity or inner product to rank results.

The sparse retriever performs keyword-based search using traditional information retrieval algorithms. It excels at matching exact terms, phrases, and spelling variations. Key implementations include:

• BM25: The modern standard for probabilistic lexical ranking, which considers term frequency and inverse document frequency.
• TF-IDF: A foundational weighting scheme.
• These models create sparse, high-dimensional bag-of-words representations for efficient inverted index lookup.

This is the core logic that merges results from the dense and sparse retrievers into a single, improved ranked list. The algorithm must handle different score distributions. Common methods are:

• Reciprocal Rank Fusion (RRF): A simple, score-agnostic method that sums the reciprocal of each document's rank from each list. Highly effective and robust.
• Weighted Score Combination: Assigns a learned or heuristic weight (e.g., 0.7 dense, 0.3 sparse) to normalized scores from each retriever before summing.
• Learning-to-Rank (LTR): Uses a machine learning model to learn the optimal combination based on features from both result sets.

This component analyzes the incoming query to determine the optimal retrieval strategy. It decides how much to rely on semantic vs. lexical search, or whether to apply query expansion. It may:

• Classify query intent (e.g., factual lookup, exploratory, navigational).
• Detect if the query contains rare, specific keywords (favoring lexical) or is conceptual (favoring semantic).
• Automatically expand the query with synonyms or related terms before sending it to the sparse retriever.

A re-ranker is a powerful, computationally expensive model that refines the final list from the fusion stage. It performs a deeper relevance assessment on a small candidate set (e.g., top 100 results). Types include:

• Cross-Encoder: A transformer model that jointly processes a query-document pair to produce a highly accurate relevance score. Superior to bi-encoders for precision but too slow for first-stage retrieval.
• Listwise Re-ranker: Considers the entire candidate list contextually to optimize the final ordering.
• This stage maximizes precision at the cost of added latency.

Operates in tandem with retrieval to impose hard business logic constraints. This engine applies metadata filtering based on document attributes (e.g., date > 2023, department = engineering, language = EN).

• Can be applied pre-retrieval (filtering the corpus before search) or post-retrieval (filtering the results).
• Crucial for enterprise applications where results must comply with access control, freshness, or other domain rules.
• Often integrated via the vector database's filtered search capabilities.

Vector search is a retrieval technique that finds items by comparing their high-dimensional vector representations (embeddings). It measures semantic similarity using metrics like cosine similarity or Euclidean distance, enabling systems to find conceptually related content even without exact keyword matches. This is the core 'semantic' component of a hybrid search pipeline.

• Key Mechanism: Encodes text into dense vectors using an embedding model.
• Primary Use: Powering semantic search and dense retrieval.
• Infrastructure: Typically relies on a vector database or libraries like Faiss for efficient Approximate Nearest Neighbor (ANN) search.

Lexical search is a traditional information retrieval method that matches documents based on the exact words (lexemes) in a query. BM25 (Best Matching 25) is its state-of-the-art algorithm, a probabilistic function that scores documents based on term frequency and inverse document frequency. It excels at finding documents with precise keyword overlap.

• Key Mechanism: Represents documents and queries as high-dimensional, sparse vectors.
• Strengths: High precision for keyword matching, interpretable results, and fast execution.
• Limitation: Cannot handle synonyms or conceptual queries without explicit keyword overlap. This is the 'keyword' component hybrid search combines with vector search.

Reciprocal Rank Fusion (RRF) is a fundamental algorithm for merging results from multiple retrieval systems, such as vector and lexical search. It creates a unified ranking by summing the reciprocal of each document's rank from each individual result list. A document appearing high in multiple lists receives a boosted aggregate score.

• Formula: score = Σ (1 / (k + rank_i)) where k is a constant (often 60).
• Advantage: Stateless and simple, requiring no score normalization between disparate systems.
• Primary Use: The most common method for fusing ranked lists in a hybrid search architecture to improve overall recall.

Reranking is a two-stage retrieval process where a broad set of candidate documents (e.g., the top 100 from hybrid search) is re-scored by a more powerful, computationally expensive model. A cross-encoder is a transformer model that jointly processes a query-document pair to produce a highly accurate relevance score, dramatically improving final precision.

• Workflow: Hybrid Search (high recall) → Reranker (high precision).
• Model Type: Cross-encoder (computationally heavy, high accuracy) vs. Bi-encoder (efficient for initial retrieval).
• Outcome: Delivers the most relevant documents within the final Top-K results, such as those passed to a Retrieval-Augmented Generation (RAG) system.

Approximate Nearest Neighbor (ANN) search is a family of algorithms that enable fast similarity searches over massive vector datasets by trading a minimal amount of accuracy for massive gains in speed and memory efficiency. It is the enabling technology for practical vector search at scale.

• Key Algorithms: Hierarchical Navigable Small World (HNSW) graphs, Inverted File (IVF) indexes, and Product Quantization (PQ).
• Purpose: Solves the computational bottleneck of exact k-Nearest Neighbors (k-NN) search in high-dimensional spaces.
• Infrastructure: Implemented in libraries like Faiss, Annoy, and commercial vector databases to power the vector side of hybrid search.

Metadata filtering is a technique applied during or after search retrieval to constrain results based on structured document attributes. In a hybrid search pipeline, filters (e.g., date > 2023, author = 'Engineering') are often applied before or after semantic/keyword retrieval to ensure results meet hard business rules.

• Common Filters: Date ranges, categories, tags, access permissions, or source system.
• Integration: Can be applied as a pre-filter (limiting the search corpus) or a post-filter (refining final results).
• Value: Combines the conceptual power of hybrid search with the determinism of structured querying, crucial for enterprise agentic memory systems where context must be scoped.