NDCG (Normalized Discounted Cumulative Gain)

NDCG applies a logarithmic discount to relevance gains based on rank position. A highly relevant document at position 1 contributes its full gain, while the same document at position 10 contributes only a fraction. The discount function is typically 1 / log2(rank + 1), meaning the drop-off is steepest at the top ranks. This reflects user behavior: searchers rarely scroll beyond the first few results, so top-heavy precision is weighted more heavily than recall at depth.

Unlike binary metrics (relevant/not relevant), NDCG supports multi-level relevance scales. Documents can be labeled with scores like:

• 0: Irrelevant
• 1: Marginally relevant
• 2: Relevant
• 3: Highly relevant
• 4: Perfectly relevant

This granularity allows NDCG to distinguish between a system that places a 'perfect' document at rank 1 versus one that places a merely 'relevant' document there. The cumulative gain is the sum of these graded scores across the ranked list before discounting.

The 'N' in NDCG stands for normalization. After computing the Discounted Cumulative Gain (DCG) of the system's ranked list, NDCG divides it by the Ideal DCG (IDCG) — the DCG of the perfect ranking where all documents are sorted in descending order of relevance. This normalization produces a score between 0.0 and 1.0, making NDCG comparable across queries with different numbers of relevant documents. A score of 1.0 means the system produced the optimal ordering.

NDCG is typically reported at a specific cutoff depth, written as NDCG@k (e.g., NDCG@10). This evaluates only the top k results, aligning with the reality that users rarely examine results beyond the first page. Common cutoffs include:

• NDCG@1: Measures if the top result is relevant
• NDCG@5: Evaluates the first half of a typical SERP
• NDCG@10: Standard full-page evaluation

Different k values reveal different system behaviors: a system might excel at NDCG@1 but degrade at NDCG@20.

Standard NDCG uses linear gain (relevance score = gain). However, many implementations use an exponential gain function: 2^relevance - 1. This dramatically amplifies the difference between relevance levels. For example:

• Relevance 3 → gain of 7
• Relevance 4 → gain of 15

This variant, sometimes called NDCG with exponential gain, strongly penalizes systems that fail to place highly relevant documents at the very top, making it the preferred formulation in modern search evaluation benchmarks like MS MARCO.

NDCG is a listwise metric — it evaluates the entire ordering as a single unit rather than individual query-document pairs. This contrasts with:

• Pointwise metrics like precision@k, which treat each position independently
• Pairwise metrics like Mean Reciprocal Rank (MRR), which only consider the first relevant document

NDCG's listwise nature makes it the preferred optimization target for Learning to Rank (LTR) algorithms like LambdaMART, which use NDCG's lambda gradients to directly optimize ranking order during training.

A family of training objectives that optimize the entire ordering of a document list rather than individual pairs or points. These losses directly maximize list-level metrics like NDCG.

• ListMLE: Minimizes the negative log-likelihood of the ground-truth permutation.
• ListNet: Uses a top-one probability distribution over permutations.
• ApproxNDCG: A smooth, differentiable surrogate for NDCG that enables direct gradient-based optimization.
• Benefit: Avoids the positional bias and local optima of pairwise methods.

A systematic error where user click feedback used to train re-rankers is confounded by the propensity of users to click on top-ranked items regardless of relevance. This corrupts NDCG calculations based on implicit feedback.

• Impact: Inflates the perceived relevance of highly-positioned documents.
• Mitigation: Inverse Propensity Scoring (IPS) weights clicks by the inverse of their examination probability.
• Relevance: Uncorrected position bias leads to NDCG scores that measure user behavior rather than true relevance, undermining metric validity.

The process of adjusting raw model logits so that output scores reflect true empirical relevance probabilities. Well-calibrated scores are essential for NDCG to accurately reflect ranking quality.

• Platt Scaling: Fits a logistic regression on top of raw scores.
• Temperature Scaling: Divides logits by a learned temperature parameter.
• Why It Matters: Overconfident but incorrect predictions at top ranks disproportionately degrade NDCG; calibration ensures the metric penalizes genuine failures rather than scaling artifacts.