Weighted Sum Fusion

The core mechanism is a straightforward linear combination: Final Score = w1 * Score_A + w2 * Score_B. This requires all input scores to be on a comparable scale, typically achieved through min-max normalization to a [0, 1] range. The simplicity of this formula makes the fusion process highly interpretable and computationally cheap to execute at query time.

The weights assigned to each retrieval stream are critical hyperparameters. A common starting point is a 0.5/0.5 split, but optimal performance requires systematic tuning. This is often performed via grid search or Bayesian optimization against a relevance metric like NDCG. The ideal weight distribution is highly dependent on the query type and the specific characteristics of the underlying indexes.

Static weights can be suboptimal for diverse query loads. A more advanced implementation uses query intent classification to dynamically adjust weights at runtime.

• Exact Code Search: Boost lexical weight (e.g., w1=0.9) for precise identifier matching.
• Conceptual Query: Boost semantic weight (e.g., w2=0.8) to bridge the lexical-semantic gap. This dynamic approach prevents the dilution of precision for exact-match queries while maximizing recall for abstract searches.

Unlike RRF, which operates on rank positions and is hyperparameter-free, Weighted Sum Fusion operates on normalized scores and requires explicit weight tuning.

• Weighted Sum: Sensitive to the magnitude of relevance scores; allows fine-grained control over subsystem influence.
• RRF: Agnostic to score magnitude; purely democratic based on rank consistency. Weighted Sum is preferred when the confidence scores from retrievers are well-calibrated and meaningful.

A critical failure point is fusing raw, unnormalized scores. A BM25 score of 15 cannot be directly summed with a cosine similarity of 0.85. Effective Weighted Sum Fusion mandates a score normalization step. Common techniques include:

• Min-Max Normalization: Rescales to [0, 1].
• Z-Score Normalization: Standardizes based on mean and standard deviation. Without this, the subsystem with the larger raw score magnitude will dominate the final ranking, rendering the weights meaningless.

A known weakness is score saturation. If a dense vector search returns many results with a cosine similarity near 1.0, the normalized scores become compressed into a tiny range. This effectively nullifies the influence of the dense retriever, as all its top candidates receive nearly identical scores. This contrasts with RRF, which is immune to this problem as it only considers ordinal rank position.

A critical preprocessing step for Weighted Sum Fusion that rescales raw relevance scores from heterogeneous retrieval subsystems into a common, comparable range. Without normalization, a system producing scores in the range [0, 1000] would dominate one producing scores in [0, 1]. Common techniques include:

• Min-Max Normalization: Rescales values to [0, 1] using the observed minimum and maximum.
• Z-Score Normalization: Centers scores around zero based on the mean and standard deviation. The choice of normalization strategy directly impacts the stability of the final fused ranking.

A score aggregation method that extends CombSUM (simple score summation) by multiplying the summed normalized score by the number of retrieval systems that returned the document. The formula is CombMNZ(d) = Σ score_i(d) * |{systems returning d}|. This mechanism heavily boosts documents found by multiple independent sources, operating on the principle that an item retrieved by both sparse and dense pipelines is more likely to be relevant than one found by a single pipeline with a deceptively high score.

A machine learning approach that trains a model to optimally combine sparse and dense retrieval signals, moving beyond the fixed linear weights of Weighted Sum Fusion. A Learning to Rank (LTR) model, such as LambdaMART, can ingest features like BM25 score, cosine similarity, and recency to learn non-linear weighting patterns from labeled relevance data. This method captures complex interactions between signals that a static weighted sum cannot, but requires a curated dataset of query-document relevance judgments for training.

The overarching retrieval architecture that combines the precision of sparse lexical matching (e.g., BM25) with the semantic understanding of dense vector search. Weighted Sum Fusion is one specific strategy for merging the result sets within this architecture. The core challenge hybrid search addresses is the lexical-semantic gap: a keyword query for 'CPU' will miss a document about 'central processor' in a pure sparse system, while a dense system might drift conceptually. Hybrid fusion bridges this gap by requiring consensus or combining evidence from both representations.

An architectural pattern where retrieval is performed independently on separate sparse and dense indexes, and the final result lists are merged and re-ranked only after each subsystem has completed its search. Weighted Sum Fusion is a classic late fusion scoring algorithm. This contrasts with early fusion, where query representations are combined before a single retrieval step. Late fusion allows each subsystem to be optimized independently but shifts the complexity to the merging logic, where score calibration and weight tuning become critical.