Normalized Discounted Cumulative Gain (NDCG)

NDCG operates on multi-level relevance scores rather than binary relevant/not-relevant labels. In legal retrieval, a Supreme Court precedent directly on point might score 3, a related circuit court decision 2, a tangentially relevant statute 1, and an irrelevant document 0. This granularity captures the nuanced utility of legal documents where degrees of relevance matter for building arguments. The gain for each document is typically computed as 2^relevance - 1, giving exponentially more weight to highly relevant items.

The 'Discounted' in NDCG applies a logarithmic discount factor to each document's gain based on its rank position. The discount for position p is 1 / log₂(p + 1), meaning:

• Position 1: full gain (discount = 1.0)
• Position 5: gain reduced by ~39%
• Position 10: gain reduced by ~54% This models real legal research behavior—attorneys rarely examine documents beyond the first page of results, making top-heavy ranking precision critical.

The 'Normalized' component divides the achieved DCG by the Ideal DCG (IDCG)—the DCG score of a perfect ranking where all documents are sorted by descending relevance. This produces a score between 0.0 and 1.0, where 1.0 represents perfect ordering. Normalization enables:

• Cross-query comparability: queries with many relevant documents don't inflate scores
• Benchmark consistency: different test collections can be compared fairly
• Legal retrieval systems can be evaluated uniformly whether a query returns 3 or 30 relevant cases

NDCG is typically reported at specific rank cutoffs, written as NDCG@k where k is the number of top results considered. Common values in legal retrieval:

• NDCG@5: Measures first-page quality for tight integration into legal workflows
• NDCG@10: Standard information retrieval benchmark
• NDCG@20: Evaluates recall-oriented legal research where exhaustive search matters A system might achieve NDCG@5 of 0.89 but NDCG@20 of 0.72, revealing that it excels at top-heavy precision but struggles with exhaustive recall.

NDCG gracefully handles incomplete relevance judgments—a common scenario in large legal corpora where human annotation is expensive. Unjudged documents are typically treated as non-relevant (gain = 0). For tied relevance scores within the ideal ranking, any permutation of equally relevant documents produces the same IDCG. This property makes NDCG robust to annotation sparsity, as missing judgments simply lower the absolute score without breaking the metric's comparative validity across systems.

NDCG differs from related ranking metrics in important ways:

• vs. Mean Average Precision (MAP): MAP assumes binary relevance; NDCG captures graded relevance essential for legal documents where 'somewhat relevant' differs from 'directly on point'
• vs. Mean Reciprocal Rank (MRR): MRR only cares about the first relevant document; NDCG rewards systems that surface multiple highly relevant documents
• vs. Recall@k: Recall ignores ranking order entirely; NDCG heavily penalizes relevant documents appearing late in results For legal multi-document reasoning tasks, NDCG is preferred because it aligns with the attorney's need to find all strong precedents, not just one.

A binary relevance metric that computes the mean of average precision scores across multiple queries. Unlike NDCG, MAP treats relevance as a binary judgment (relevant or not relevant) rather than a graded scale.

• Calculation: Averages the precision at each position where a relevant document is retrieved
• Use Case: Strong baseline for legal search when relevance is a yes/no decision
• Limitation: Cannot distinguish between highly relevant and marginally relevant documents, making it less suitable than NDCG for nuanced legal ranking

The un-normalized precursor to NDCG that accumulates graded relevance scores with a logarithmic position discount. DCG provides the raw numerator that NDCG normalizes against the ideal ranking.

• Formula: DCG_p = Σᵢ₌₁ᵖ (2^{rel_i} - 1) / log₂(i + 1)
• Position Discount: The log₂(i+1) divisor penalizes relevant documents appearing lower in results
• Relationship: NDCG = DCG / IDCG, where IDCG is the ideal DCG for a perfect ranking
• Insight: DCG alone cannot be compared across queries with different numbers of relevant documents, which is why normalization is essential

A two-stage retrieval architecture where a computationally expensive cross-encoder jointly processes the query and each candidate document to produce a fine-grained relevance score. This reranking step directly optimizes the NDCG of the final result list.

• Mechanism: Unlike bi-encoders that encode query and document independently, cross-encoders attend across all token pairs
• NDCG Impact: Applying a cross-encoder reranker to top-k candidates from a fast first-pass retrieval typically improves NDCG by 5-15%
• Trade-off: High latency per document pair limits application to a small candidate set, making it complementary to efficient embedding-based retrieval

A training data strategy that identifies documents which are superficially similar to a query but not relevant, and uses them as negative examples during contrastive learning. This directly improves the model's ability to discriminate, which is measured by NDCG.

• Method: Retrieve top candidates with a baseline model, then label false positives as hard negatives
• Legal Application: In case law retrieval, a dissenting opinion on the same topic is a classic hard negative
• NDCG Connection: Models trained with hard negatives show significantly higher NDCG because they learn to push away confusing near-matches that would otherwise pollute top ranks

A technique that augments the original search query with related terms, synonyms, or generated text to improve recall. When evaluated with NDCG, effective query expansion increases the metric by surfacing relevant documents that would otherwise be missed due to vocabulary mismatch.

• Legal Example: Expanding 'slip and fall' to include 'premises liability' and 'trip hazard'
• Methods: Pseudo-relevance feedback, thesaurus lookup, or synthetic query generation via LLMs
• NDCG Effect: Primarily improves recall at lower ranks, increasing cumulative gain; the position discount in NDCG ensures that newly retrieved relevant documents are rewarded proportionally to their rank