Normalized Discounted Cumulative Gain (NDCG)

NDCG applies a logarithmic discount to the relevance gain of each document based on its rank position. A highly relevant document at position 1 contributes full value, while the same document at position 10 contributes substantially less. This mirrors real user behavior—searchers rarely scroll past the first page.

• Discount formula: gain / log₂(rank + 1)
• Position 1: discount factor of 1.0 (no penalty)
• Position 5: discount factor of ~0.39
• Position 10: discount factor of ~0.29
• Models the positional attention decay observed in eye-tracking studies

The 'N' in NDCG stands for normalization against an ideal ranking. The system's actual DCG score is divided by the Ideal DCG (IDCG)—the score achieved by a perfect ranking where all documents are sorted in descending order of relevance. This produces a value between 0.0 and 1.0, making scores comparable across queries with different numbers of relevant documents.

• NDCG = DCG / IDCG
• A score of 1.0 indicates a perfect ranking
• Normalization enables cross-query averaging
• Prevents queries with many relevant documents from dominating aggregate metrics

Unlike binary metrics (relevant/not relevant), NDCG supports multi-level relevance grades. Documents can be labeled on a scale—such as 0 (irrelevant), 1 (tangentially relevant), 2 (relevant), 3 (highly relevant), or 4 (perfectly relevant). This granularity rewards systems that prioritize the most authoritative or comprehensive results.

• Binary relevance: 0 or 1 (simplified NDCG)
• Graded relevance: 0–4 scale common in benchmark datasets
• Higher grades receive exponentially more gain weight
• Used extensively in MS MARCO and TREC evaluation campaigns
• Captures the nuance that some relevant documents are better than others

NDCG is almost always computed at a specific cutoff depth K, written as NDCG@K. Common values are NDCG@5, NDCG@10, and NDCG@20. This focuses evaluation on the top results that users actually see, rather than the entire ranked list. NDCG@10 is the most widely reported metric in academic information retrieval literature.

• NDCG@10: Evaluates the first page of results
• NDCG@5: Stricter metric for mobile or voice search interfaces
• NDCG@100: Used for recall-oriented evaluation
• Choosing K should align with your application's user interface constraints

The 'C' in DCG stands for cumulative—the metric sums the discounted gain across all positions up to rank K. This means a system is rewarded for every relevant document it retrieves, not just the first one. Unlike Mean Reciprocal Rank (MRR), which only cares about the first relevant result, NDCG captures the full quality of the ranked list.

• DCG@K = Σ(gainᵢ / log₂(i + 1)) for i = 1 to K
• Rewards systems that surface multiple relevant items
• Contrasts with MRR, which only evaluates the first hit
• Better suited for exploratory search where users examine multiple results

To strongly reward highly relevant documents, NDCG often applies exponential gain scaling: gain = 2^relevance - 1. A document with relevance grade 4 receives a gain of 15, while grade 1 receives only 1. This creates a strong incentive for ranking systems to place the most authoritative documents at the very top.

• Linear gain: gain = relevance (less common)
• Exponential gain: gain = 2^relevance - 1 (standard)
• Grade 0 → gain 0, Grade 1 → gain 1, Grade 3 → gain 7, Grade 4 → gain 15
• Places heavy penalty on mis-ranking highly relevant documents
• Aligns with business metrics where top positions drive most engagement

A rank-aware metric that focuses exclusively on the position of the first relevant document. It calculates the average of the reciprocal of the rank at which the first relevant item appears across a set of queries.

• Formula: MRR = (1/|Q|) * Σ (1/rank_i)
• Key Difference: Unlike NDCG, MRR ignores all relevant documents after the first one.
• Use Case: Best suited for navigational queries or question-answering systems where the user needs exactly one correct answer, such as a factoid lookup.

A set-based metric that measures the fraction of the top K retrieved documents that are relevant, without considering their rank order within that set.

• Calculation: (Number of relevant documents in top K) / K
• Key Difference: P@K treats all positions in the top K equally, unlike NDCG which applies a logarithmic discount to lower positions.
• Use Case: Critical for evaluating the quality of the first page of search results where user attention is roughly uniform across the visible set.

A coverage metric that measures the fraction of all relevant documents in the corpus that are successfully retrieved within the top K results.

• Calculation: (Relevant docs in top K) / (Total relevant docs in corpus)
• Key Difference: NDCG evaluates ranking quality, while Recall@K evaluates retrieval completeness.
• Use Case: Essential for high-recall domains like e-discovery and medical literature review, where missing a relevant document is a critical failure.

The un-normalized precursor to NDCG that sums the graded relevance scores of documents in a result list, applying a logarithmic discount factor to penalize relevant documents appearing lower in the ranking.

• Formula: DCG_p = Σ (2^{rel_i} - 1) / log₂(i + 1)
• Key Difference: DCG is an absolute score that cannot be compared across queries with different numbers of relevant documents. NDCG solves this by dividing by the Ideal DCG (IDCG).
• Gain Function: The exponential gain (2^{rel} - 1) heavily rewards highly relevant documents over marginally relevant ones.

A supervised machine learning framework where NDCG often serves as the primary optimization objective. LTR models learn to combine hundreds of relevance signals into a scoring function.

• LambdaMART: A gradient-boosted tree algorithm that directly optimizes NDCG by using gradients derived from the LambdaRank framework.
• Training Data: Requires query-document pairs with graded relevance judgments.
• Connection to NDCG: NDCG is the most common evaluation metric for LTR models because it captures both graded relevance and positional importance.

The theoretical maximum DCG achievable for a query, calculated by sorting all relevant documents in descending order of relevance grade and computing the DCG of this perfect ranking.

• Purpose: Serves as the normalization factor in NDCG, constraining the final score to the range [0, 1].
• Calculation: IDCG_p = Σ (2^{rel_i} - 1) / log₂(i + 1) for the ideal sorted order.
• Why It Matters: Without IDCG normalization, comparing DCG scores across queries with different numbers of highly relevant documents would be meaningless.