Normalized Discounted Cumulative Gain (NDCG)

NDCG fundamentally operates on the principle that highly relevant items must appear at the top of a ranked list. Unlike flat metrics like precision, NDCG applies a logarithmic discount to relevance gains based on their position. An item at rank 1 contributes its full relevance score, while an item at rank 100 contributes a heavily discounted fraction. This directly models user behavior, where attention and click probability decay rapidly as users scan down a list of search results or product recommendations.

NDCG supports nuanced, non-binary relevance judgments. Instead of a simple relevant/irrelevant label, items are assigned a graded relevance score (e.g., 0 for 'irrelevant', 1 for 'tangential', 3 for 'relevant', 5 for 'perfect match'). This allows the metric to penalize a system that ranks a 'tangential' result above a 'perfect' one, even if both are technically 'relevant' in a binary sense. The gain is typically computed as 2^rel - 1, giving exponentially higher credit to higher relevance grades.

The 'Normalized' in NDCG is its most critical feature. The raw Discounted Cumulative Gain (DCG) is divided by the Ideal DCG (IDCG)—the DCG of the perfect ranking for that specific query. This normalization bounds the score between 0.0 and 1.0, making it query-agnostic. A score of 1.0 indicates a perfect ranking. This allows you to average NDCG across a test set of diverse queries where the number of relevant items varies wildly, a scenario where raw DCG would be meaningless.

The formal definition is NDCG@k = DCG@k / IDCG@k. The DCG is calculated as the sum over all positions i up to k of (2^{rel_i} - 1) / log_2(i + 1). The numerator is the gain from the item's relevance grade, and the denominator is the positional discount. The IDCG is computed by sorting all items in the corpus by their true relevance grade in descending order and calculating the DCG of that ideal list. This normalization ensures the score reflects the system's ability to sort items correctly.

NDCG is almost always evaluated at a specific cutoff k, denoted as NDCG@k (e.g., NDCG@10, NDCG@50). This focuses the evaluation on the top of the ranked list, which is the only part most users interact with. The choice of k is domain-specific: a search engine might optimize for NDCG@5, while a recommendation carousel might look at NDCG@20. This metric ignores the ordering of items beyond the cutoff, aligning the evaluation with the practical limits of a user interface.

A practical challenge with NDCG is unjudged documents. In large-scale evaluation, it's impossible to label every item. The standard approach is to treat unjudged items as irrelevant (gain=0). This is a conservative assumption that penalizes systems for surfacing unvetted content. More advanced techniques like bpref or inferred relevance models exist, but the zero-gain assumption remains the industry default for its simplicity and safety, ensuring that systems are not rewarded for returning items of unknown quality.

An evaluation metric that measures the proportion of all relevant items that appear within the top K positions of a ranked list. Unlike NDCG, Recall@K is position-insensitive within the cutoff—a relevant item at rank 1 counts the same as one at rank K. This makes it ideal for assessing retrieval stages where exhaustive recall is prioritized over precise ordering. For example, if a user has 10 relevant items in a catalog and 7 appear in the top 100 results, Recall@100 = 0.7. It is often paired with Precision@K to balance completeness against signal-to-noise ratio.

A metric that evaluates systems where there is exactly one relevant item and the user needs it at the top. MRR is the average of the reciprocal of the rank at which the first relevant item appears across multiple queries. A relevant item at rank 1 yields a score of 1.0; at rank 5, 0.2. This metric is dominant in question-answering systems and FAQ retrieval where the user expects a single correct answer, not a ranked list of possibilities.

A binary relevance metric that computes the average of precision values calculated at each position where a relevant document is retrieved. MAP provides a single-figure measure of quality across recall levels, heavily penalizing systems that bury relevant items deep in the ranking. Unlike NDCG, MAP treats all relevant items as equally relevant—it cannot distinguish between highly relevant and marginally relevant results. It remains standard in information retrieval benchmarks where graded relevance judgments are unavailable.

The unnormalized precursor to NDCG. DCG accumulates the relevance score of each item, logarithmically discounted by its position. The formula is: DCG@K = Σ(i=1 to K) (2^rel_i - 1) / log₂(i + 1). The exponential gain function (2^rel - 1) ensures that highly relevant items contribute disproportionately more than marginally relevant ones. DCG is useful for comparing systems on the same query but cannot be averaged across queries with different numbers of relevant items—hence the need for normalization.

The maximum possible DCG achievable for a given query, computed by sorting all items in descending order of true relevance and calculating DCG on this perfect ranking. IDCG serves as the normalization factor in NDCG: NDCG = DCG / IDCG. This normalization bounds the score between 0 and 1 and enables meaningful averaging across queries with vastly different numbers of relevant items. Without IDCG normalization, a query with 50 relevant items would dominate the aggregate metric.

Training objectives that frame ranking as a binary classification on item pairs. Instead of predicting absolute relevance scores, the model learns to predict which of two items should rank higher. Bayesian Personalized Ranking (BPR) is the canonical example, maximizing the probability that an observed item is preferred over an unobserved one. Pairwise losses directly optimize the relative ordering that NDCG measures, often yielding better ranking performance than pointwise regression approaches that treat relevance prediction as an independent task.