Learned Fusion

Learned fusion models are trained on labeled query-document pairs where human annotators have judged relevance. The model learns to assign weights to features like BM25 scores, cosine similarity, and recency by minimizing a loss function, such as cross-entropy, against these ground-truth labels. This replaces manual trial-and-error weight tuning with a statistically optimal solution.

While heuristic methods like Reciprocal Rank Fusion (RRF) apply a fixed mathematical transform, learned fusion can capture complex interactions. A neural network or gradient-boosted tree can learn that a high BM25 score is only valuable when the dense vector similarity is low, or vice versa. This allows the model to apply conditional boosting based on query intent, a capability impossible with linear weighted sum fusion.

The power of learned fusion comes from the input features. The model ingests a vector of signals for each query-document pair:

• Sparse lexical features: BM25 score, TF-IDF overlap, exact match ratio.
• Dense semantic features: Cosine similarity, dot product, Euclidean distance.
• Query-level features: Query length, click-through rate history, entity count.
• Document-level features: PageRank, document length, freshness timestamp.

Learned fusion is typically implemented as a Learning to Rank (LTR) problem. Frameworks like LambdaMART or neural rankers are trained using a listwise loss function that directly optimizes for ranking metrics like Normalized Discounted Cumulative Gain (NDCG). The model is trained to order documents such that the cumulative gain is maximized, learning the fusion weights implicitly as part of the ranking function.

A key advantage is dynamic adaptation. The model can learn to behave like a pure lexical system for exact product code searches and like a pure semantic system for vague conceptual queries. This is achieved by including query intent classification scores as input features, allowing the fusion logic to shift its reliance between sparse and dense signals on a per-query basis without manual rule-setting.

The primary limitation is data dependency. A learned fusion model requires a substantial volume of in-domain relevance judgments to train. It may not generalize well to drastically different document corpora or query distributions without retraining. In contrast, unsupervised methods like RRF are zero-shot and work immediately, making learned fusion a high-precision upgrade for mature systems with established click-through and annotation data.

The critical preprocessing step that rescales heterogeneous relevance scores into a common range before fusion. Without normalization, a BM25 score of 25 can dominate a cosine similarity of 0.87, corrupting the final ranking.

• Min-Max Normalization: Rescales to [0,1] using observed min/max
• Z-Score Normalization: Centers around mean with unit variance
• Sum-to-Unity: Divides each score by the sum of all scores in the list

A precision-focused second stage where a Transformer model processes the query and candidate document jointly through full self-attention. Unlike a Bi-Encoder, the Cross-Encoder captures fine-grained token-level interactions at the cost of latency.

• Produces a single high-fidelity relevance score per query-document pair
• Computationally prohibitive for full corpus; applied only to top-K candidates
• Often used as the final fusion step in a multi-stage retrieval pipeline

A supervised machine learning framework that trains a model to optimally combine multiple relevance signals—sparse scores, dense scores, recency, and popularity—into a single ranking function. LTR learns non-linear weighting patterns from labeled query-document pairs.

• Pointwise: Predicts absolute relevance per document
• Pairwise: Learns relative ordering between document pairs
• Listwise: Directly optimizes list-level metrics like NDCG
• LambdaMART is the classic gradient-boosted tree implementation

A retrieval paradigm bridging the efficiency of Bi-Encoders and the precision of Cross-Encoders. ColBERT stores token-level embeddings for documents and computes relevance via the sum of maximum cosine similarities (MaxSim) between query and document token representations.

• Enables fine-grained lexical-semantic matching without full joint attention
• Document embeddings can be pre-computed and indexed for fast retrieval
• Ideal for scenarios requiring nuanced matching with strict latency budgets

A preprocessing step that analyzes a user's search query to dynamically adjust fusion weights. A navigational query for a product code should boost lexical matching, while an informational query benefits from semantic boosting.

• Navigational: User seeks a specific document or page
• Informational: User seeks knowledge on a topic
• Transactional: User intends to perform an action
• Enables adaptive fusion strategies without manual tuning