Normalized Discounted Cumulative Gain (NDCG)

NDCG applies a logarithmic discount to relevance gains based on rank position. The gain for a document at position p is divided by log₂(p+1). This models user behavior where searchers are exponentially less likely to examine documents deeper in the list. A highly relevant document at rank 1 contributes full value, while the same document at rank 10 contributes roughly 30% of its original gain. This discounting mechanism makes NDCG more realistic than precision-based metrics that treat all positions equally.

Unlike binary relevance metrics, NDCG supports multi-level graded relevance scores. Relevance judgments can be assigned on an ordinal scale—such as 0 (irrelevant), 1 (tangentially relevant), 2 (relevant), 3 (highly relevant), and 4 (perfectly relevant). This granularity allows evaluators to distinguish between marginally useful documents and authoritative sources. The cumulative gain at each rank position sums these graded scores, enabling nuanced evaluation that rewards search systems for surfacing the most authoritative content at the top.

The 'N' in NDCG comes from dividing the achieved Discounted Cumulative Gain (DCG) by the Ideal DCG (IDCG)—the DCG of a perfect ranking where documents are sorted in descending order of relevance. This normalization produces a score between 0 and 1, making NDCG comparable across queries with different numbers of relevant documents. A score of 1.0 indicates the system returned the optimal ordering. Without normalization, DCG scores would be query-length dependent and impossible to average across a test set.

NDCG is commonly reported at specific rank cutoffs, denoted as NDCG@k where k represents the truncation depth. Common variants include:

• NDCG@5: Evaluates the top 5 results, emphasizing above-the-fold performance
• NDCG@10: Standard cutoff for web search evaluation, representing the first page of results
• NDCG@20: Used when recall-oriented evaluation is needed Choosing the appropriate k depends on the use case—question-answering systems may prioritize NDCG@1, while exploratory search interfaces benefit from higher cutoffs.

To emphasize highly relevant documents, NDCG often uses an exponential gain function: gain = 2^rel - 1, where rel is the graded relevance score. This formulation creates a non-linear reward structure where a document with relevance 4 contributes gain of 15, while relevance 2 contributes only 3. The exponential scaling ensures that search systems are strongly incentivized to place the most authoritative documents at the top, rather than being satisfied with moderately relevant results in prominent positions.

NDCG differs from other ranking metrics in key ways:

• MRR (Mean Reciprocal Rank): Only considers the position of the first relevant document, ignoring subsequent results and graded relevance
• MAP (Mean Average Precision): Uses binary relevance and weights precision equally across positions without logarithmic discounting
• NDCG: Combines graded relevance, position discounting, and ideal normalization, making it the preferred metric when relevance judgments exist on a multi-level scale and ranking order matters throughout the entire list

NDCG requires multi-level relevance scores, not just binary relevant/not-relevant. Graded relevance assigns ordinal values (e.g., 0-4) to query-document pairs:

• 0: Irrelevant
• 1: Marginally relevant
• 2: Relevant
• 3: Highly relevant
• 4: Perfectly authoritative This granularity allows NDCG to reward systems that place highly relevant documents above merely relevant ones, providing a more nuanced evaluation than precision-based metrics.

The foundational component of NDCG before normalization. DCG sums the relevance scores of documents in a ranked list, applying a logarithmic discount to penalize relevant documents appearing lower:

• Position 1: rel₁
• Position p: relₚ / log₂(p+1) The discount function models user behavior—searchers are exponentially less likely to examine documents deeper in the ranking. DCG captures both relevance magnitude and position quality.

The theoretical maximum DCG achievable by sorting all documents in descending order of true relevance. IDCG serves as the normalization factor:

• NDCG = DCG / IDCG
• Produces a score in [0, 1] range
• 1.0 represents a perfect ranking By dividing by IDCG, NDCG becomes query-independent, enabling fair comparison across queries with different numbers of relevant documents. Without this normalization, queries with many relevant documents would dominate aggregate scores.

A complementary metric focused on the first relevant document rather than the entire ranking. MRR averages the reciprocal of the rank position of the earliest relevant result:

• MRR = (1/|Q|) × Σ 1/rankᵢ
• Perfect for navigational queries where users need exactly one answer
• Insensitive to additional relevant documents below the first Use MRR alongside NDCG when evaluating systems where top-position precision is the primary success criterion, such as featured snippets or direct answers.

A gradient boosted tree algorithm that directly optimizes NDCG using LambdaRank gradients. Key characteristics:

• Listwise approach: Considers the entire ranked list during training
• Lambda gradients: Encodes the cost of swapping document pairs weighted by NDCG delta
• Ensemble of trees: Builds additive regression trees with gradient boosting LambdaMART dominated learning-to-rank competitions for years and remains a production baseline. It explicitly targets the metric you care about rather than a proxy loss function.

Click data used for training rankers suffers from position bias—users click top results regardless of relevance. NDCG evaluation with biased judgments produces misleading scores. Correction techniques:

• Propensity weighting: Inverse propensity scoring using randomization experiments
• Click models: Estimating true relevance by modeling examination probability
• Result randomization: Periodically shuffle results to collect unbiased feedback Without bias correction, NDCG measured on implicit feedback will overestimate systems that simply match user click patterns rather than true relevance.