Score Normalization

A linear scaling technique that transforms scores to a fixed range, typically [0, 1], by subtracting the minimum observed score and dividing by the range.

• Formula: (score - min) / (max - min)
• Best for: Bounded fusion where you know the theoretical or empirical score range
• Weakness: Highly sensitive to outliers; a single extreme score compresses the rest of the distribution
• Example: Normalizing BM25 scores from [0, 45] to [0, 1] for combination with cosine similarity scores

Transforms scores to have a mean of zero and a standard deviation of one by subtracting the population mean and dividing by the standard deviation.

• Formula: (score - μ) / σ
• Best for: Gaussian-like score distributions where relative positioning matters more than absolute magnitude
• Key property: Preserves the shape of the original distribution while centering and scaling
• Example: Standardizing cross-encoder logit scores before combining with dense retrieval scores in a weighted sum

Applies the softmax function with a temperature parameter to convert a vector of raw scores into a probability distribution where all values sum to 1.

• Formula: exp(score_i / T) / Σ exp(score_j / T)
• Best for: Converting uncalibrated logits into interpretable probabilities for weighted fusion
• Temperature control: Lower T sharpens the distribution; higher T flattens it
• Example: Normalizing re-ranker scores across top-100 candidates so they can be interpreted as confidence estimates

Discards raw scores entirely and assigns new scores based solely on ordinal position within each ranked list, making it distribution-free and outlier-robust.

• Formula: 1 / (rank + k) where k is a constant (often 60 for RRF)
• Best for: Reciprocal Rank Fusion when score magnitudes are uncalibrated or incomparable
• Key advantage: Completely invariant to score scale, range, and distribution shape
• Example: Merging results from BM25, dense retrieval, and a neural re-ranker using RRF without any score calibration

Maps raw scores to a target distribution by computing empirical cumulative distribution functions and applying an inverse CDF transform, forcing scores to follow a known statistical shape.

• Process: Compute ECDF of raw scores → apply inverse Gaussian CDF → scores are now normally distributed
• Best for: Fusing scores from systems with radically different output distributions
• Requirement: Needs a representative sample of scores to estimate the empirical CDF
• Example: Transforming gamma-distributed BM25 scores and bimodal cosine similarity scores into a common Gaussian distribution before weighted averaging

Fits a logistic regression model on top of raw classifier scores to produce well-calibrated probability estimates, correcting systematic over- or under-confidence.

• Formula: P(y=1|score) = 1 / (1 + exp(A × score + B))
• Best for: Converting SVM outputs or uncalibrated neural network scores into reliable probabilities
• Training: Requires a held-out calibration set with binary relevance labels
• Example: Calibrating cross-encoder relevance scores so a score of 0.8 consistently means an 80% chance of relevance across different query distributions

A linear transformation that rescales raw scores into a fixed range, typically [0, 1]. The formula (X - X_min) / (X_max - X_min) preserves the original distribution's shape but is highly sensitive to outliers. In re-ranking pipelines, a single extremely high BM25 score can compress the rest of the distribution, making this method risky unless combined with winsorization or clipping to handle extreme values.

Transforms scores to have a mean of 0 and a standard deviation of 1 using (X - μ) / σ. Unlike min-max scaling, Z-scores do not bound the output to a specific range, which can be problematic for fusion algorithms expecting non-negative inputs. However, it is highly effective for normalizing cosine similarity outputs from different embedding models where the score distributions are approximately Gaussian.

Linear fusion techniques that aggregate normalized scores from multiple rankers. CombSUM simply sums the normalized scores for each document. CombMNZ multiplies the CombSUM score by the number of non-zero scores (the 'hit count'), heavily rewarding documents retrieved by multiple systems. Both methods require prior score normalization to prevent a single high-variance ranker from dominating the final ordering.

A calibration method that divides raw logits by a learned temperature parameter T before applying softmax. When T > 1, the distribution softens, reducing overconfidence. In retrieval, this is used to calibrate the confidence of cross-encoder relevance scores before fusion with sparse retrieval signals. Unlike heuristic normalization, temperature scaling optimizes a probabilistic metric like Expected Calibration Error (ECE) on a held-out validation set.

A critical consideration when normalizing scores derived from user interaction data. Click-Through Rate (CTR) as a raw score is confounded by position bias—users click top results regardless of relevance. Normalization must account for this by applying inverse propensity weighting (IPW) or training an Examination Model to estimate the true relevance probability before scores are fused with algorithmic ranking signals.