Normalized Discounted Cumulative Gain (NDCG)

NDCG applies a logarithmic discount function to penalize relevant documents that appear lower in the ranked list. The discount for position p is typically 1 / log₂(p + 1), meaning a highly relevant document at rank 1 contributes fully, while the same document at rank 10 contributes only about 29% of its original value. This mirrors real user behavior—searchers rarely scroll past the first page of results. The cumulative gain is the sum of these discounted relevance scores across all positions, ensuring the metric rewards systems that front-load the most valuable content.

The 'Normalized' in NDCG divides the system's DCG score by the Ideal DCG (IDCG)—the DCG achieved by a perfect ranking where all documents are sorted in descending order of relevance. This normalization serves two critical purposes:

• Cross-query comparability: Scores fall between 0.0 and 1.0 regardless of result set size
• Fair evaluation: Queries with many highly relevant documents don't artificially inflate scores An NDCG of 1.0 indicates the system produced the theoretically optimal ordering for that query.

Unlike binary metrics such as Precision@K, NDCG supports graded relevance labels that capture varying degrees of usefulness. Common grading schemes include:

• 0: Not relevant
• 1: Marginally relevant (tangential topic)
• 2: Relevant (answers the query partially)
• 3: Highly relevant (definitive answer)
• 4: Perfectly relevant (navigational intent, exact match) This granularity allows NDCG to distinguish between a system that returns ten 'marginally relevant' documents versus one that returns three 'highly relevant' ones—a distinction binary metrics completely miss.

NDCG is commonly computed at specific cutoff points—NDCG@5, NDCG@10, NDCG@20—to evaluate ranking quality within the viewport users actually see. NDCG@10, for instance, only considers the top 10 results, making it the de facto standard for web search evaluation. This cutoff approach acknowledges that users rarely paginate deeply, so ranking quality beyond position 20 has diminishing practical importance. Different applications use different cutoffs: e-commerce product search might optimize for NDCG@5, while academic literature search might use NDCG@20 or higher.

The 'Gain' component can use either linear gain (gain = relevance_label) or exponential gain (gain = 2^relevance_label - 1). Exponential gain dramatically amplifies the value of highly relevant documents. Under exponential gain, a document with relevance 4 contributes 15 points, while relevance 1 contributes only 1 point—a 15:1 ratio versus 4:1 under linear gain. This is critical for navigational queries where only the single most relevant result matters. The choice of gain function should align with the application's tolerance for partially relevant results.

NDCG sits within a family of ranking evaluation metrics, each with distinct properties:

• Mean Reciprocal Rank (MRR): Only cares about the position of the first relevant document; ignores all others
• Mean Average Precision (MAP): Uses binary relevance and averages precision at each relevant document's position
• NDCG: Combines graded relevance with position discounting and normalization NDCG is generally preferred over MAP when relevance judgments are graded rather than binary, making it the standard for modern search systems where nuanced relevance distinctions matter.

A metric that evaluates retrieval systems by measuring the rank position of the first relevant document. MRR calculates the average of the reciprocal ranks across a set of queries.

• Formula: MRR = (1/|Q|) * Σ (1/rank_i)
• Focuses exclusively on the first relevant result
• Ignores all subsequent relevant documents in the list
• Best suited for navigational queries where the user needs exactly one correct answer
• Example: If the first relevant doc is at position 3, the reciprocal rank is 1/3 ≈ 0.33

The foundational precursor to DCG that sums the relevance scores of all documents in a ranked list without any positional discounting. CG treats all positions equally.

• Calculated as the simple sum of graded relevance labels (e.g., 0-3 scale)
• No positional weighting means a relevant doc at rank 10 counts the same as rank 1
• Useful as a baseline before applying discount and normalization
• Limitation: Fails to capture the critical insight that users prefer top-ranked results

The direct predecessor to NDCG that applies a logarithmic discount to relevance scores based on position. Documents appearing later in the ranked list contribute progressively less to the total score.

• Formula: DCG = Σ (rel_i / log₂(i+1))
• The logarithmic discount models user behavior where attention decays with scroll depth
• A highly relevant document at rank 1 contributes its full score
• The same document at rank 10 contributes roughly one-third of its value
• DCG is not normalized, making it impossible to compare across queries with different numbers of relevant documents

The theoretical maximum DCG achievable by sorting all documents in perfect relevance order. IDCG serves as the normalization factor that transforms DCG into NDCG.

• Calculated by ranking all known relevant documents from highest to lowest relevance grade
• Represents the upper bound of ranking quality for a given query
• NDCG = DCG / IDCG, producing a score between 0 and 1
• A score of 1.0 indicates the system achieved the perfect ranking
• Critical for cross-query comparison because it accounts for varying numbers of relevant documents per query

A cutoff-based metric that measures the proportion of relevant documents within the top K results. Unlike NDCG, Precision@K uses binary relevance judgments and ignores ranking order within the top K.

• Formula: P@K = (number of relevant docs in top K) / K
• Binary relevance means documents are either relevant or not, with no graded distinctions
• All positions within the top K are treated equally
• Commonly used alongside Recall@K for a complete picture
• Limitation: Cannot distinguish between a perfect ranking and one where relevant docs are at the bottom of the top K

A metric that combines precision and recall by averaging precision values at each position where a relevant document is found. AP rewards systems that place relevant documents higher in the ranking.

• Calculated as the mean of precision scores at each relevant document's rank
• Penalizes systems that bury relevant documents deep in the results
• Unlike NDCG, AP typically uses binary relevance rather than graded judgments
• Mean Average Precision (MAP) averages AP across all queries
• Particularly popular in information retrieval research and TREC evaluations