Normalized Discounted Cumulative Gain (NDCG)

NDCG applies a logarithmic discount function to reduce the contribution of relevant documents as their rank position increases. A highly relevant document at position 1 contributes its full gain, while the same document at position 10 contributes roughly 1/log₂(11) ≈ 30% of its value. This mirrors real user behavior—searchers rarely browse beyond the first page of results. The discount is calculated as 1 / log₂(rank + 1), creating a steep decay curve that heavily rewards top-ranked precision.

Unlike binary metrics that treat relevance as a simple yes/no, NDCG supports multi-level relevance scales. Documents can be scored on a graded scale such as:

• 0: Not relevant
• 1: Marginally relevant
• 2: Relevant
• 3: Highly relevant
• 4: Perfectly relevant This granularity allows the metric to distinguish between a system that returns 'good enough' results versus one that surfaces the definitive, authoritative answer for a query.

The 'Normalized' component divides the system's DCG by the Ideal DCG (IDCG)—the score achieved by a perfect ranking where all documents are sorted in descending order of relevance. This normalization produces a value between 0.0 and 1.0, making NDCG comparable across queries with different numbers of relevant documents. A score of 1.0 indicates the system returned results in the optimal order. Without normalization, comparing DCG across queries of varying difficulty would be meaningless.

NDCG is typically reported at specific rank cutoffs, written as NDCG@k where k represents the number of top results evaluated. Common values include:

• NDCG@5: Measures the quality of results visible without scrolling on mobile
• NDCG@10: Evaluates the first page of results
• NDCG@100: Assesses deep ranking quality for recall-oriented tasks This cutoff specificity allows engineers to optimize for different use cases—a voice assistant might prioritize NDCG@1, while a legal discovery system cares about NDCG@100.

The metric builds on Cumulative Gain (CG), which is simply the sum of relevance scores across all retrieved documents. Discounted Cumulative Gain (DCG) extends this by applying the position discount, calculated as: DCG = Σ (2^relevance - 1) / log₂(rank + 1). The exponential gain function 2^relevance - 1 ensures that highly relevant documents contribute disproportionately more value—a document with relevance 4 contributes 15 times the gain of a document with relevance 1, emphasizing the premium on surfacing the best content first.

In large-scale evaluation, human annotators cannot judge every document for every query. NDCG implementations must handle unjudged documents carefully. Common strategies include:

• Treating unjudged as irrelevant (0): Conservative approach that may unfairly penalize systems
• Ignoring unjudged documents: Removes them from the calculation entirely
• Inferred relevance: Using click-through data or model predictions as proxy judgments The choice of strategy significantly impacts reported scores and must be documented in any rigorous evaluation framework.

A metric that measures the effectiveness of a ranking system by identifying the position of the first relevant result. It is calculated as the average of the reciprocal ranks across a set of queries.

• Formula: MRR = 1/|Q| * Σ (1/rank_i)
• Focus: Perfect for use cases where the user only needs one relevant document, such as a direct answer or a 'known-item' search.
• Limitation: It ignores all relevant documents appearing after the first one, making it unsuitable for evaluating recall-oriented tasks.

A binary relevance metric that calculates the average precision at each position where a relevant document is found, then averages this across all queries. It provides a single-figure measure of quality across recall levels.

• Precision Focus: Heavily penalizes systems that retrieve irrelevant documents high in the ranking.
• Binary Assumption: Assumes relevance is strictly binary (relevant or not), unlike NDCG which handles graded relevance.
• Usage: Commonly used in academic benchmarks where judgments are binary.

The un-normalized precursor to NDCG. It sums the graded relevance scores of documents in a result list, applying a logarithmic discount to penalize relevant documents appearing lower in the ranking.

• Discount Logic: The value of a relevant document at position 10 is much lower than at position 1.
• Raw Score: DCG is an absolute score that cannot be easily compared across queries with varying numbers of relevant documents, which is why normalization (NDCG) is essential.
• Formula: DCG_p = Σ (rel_i / log2(i+1))

The theoretical maximum DCG achievable for a given query. It is calculated by sorting all known relevant documents in descending order of their graded relevance scores, producing the perfect ranking.

• Normalization Factor: NDCG is derived by dividing the actual DCG by the IDCG.
• Upper Bound: An NDCG of 1.0 indicates the system achieved the perfect ranking order.
• Dependency: The quality of IDCG is entirely dependent on the completeness of your relevance judgments.

The foundational data input for NDCG that assigns a multi-level score to a document-query pair, moving beyond simple binary relevance.

• Scale Example: 0 (Irrelevant), 1 (Marginally Relevant), 2 (Relevant), 3 (Highly Relevant), 4 (Perfectly Authoritative).
• Gain Function: NDCG often uses an exponential gain (e.g., 2^rel - 1) to make highly relevant documents disproportionately more valuable.
• Human Annotation: These judgments are typically created by human raters following strict guidelines.

A simple metric measuring the proportion of relevant documents within the top-K results. It is purely precision-focused and does not consider the order of documents within that set.

• Position Agnostic: A relevant document at rank 1 counts the same as one at rank K.
• Contrast with NDCG: Unlike NDCG, Precision@K does not reward systems for placing the most relevant documents first.
• Use Case: Useful for front-page analysis where the user scans all top results equally.