Dense Retrieval Metrics

Recall at K measures the proportion of all relevant documents for a query that are successfully retrieved within the top K results. It directly assesses a dense retriever's ability to find the complete set of necessary context.

• Formula: (Relevant docs in top K) / (Total relevant docs in corpus)
• Use Case: Critical for ensuring high-coverage retrieval in RAG, where missing a key document can lead to incomplete or incorrect answers.
• Example: If there are 5 relevant documents total and the top 10 results contain 4 of them, R@10 = 0.8 (80%).

Mean Reciprocal Rank evaluates the rank of the first relevant document across multiple queries. It is the average of the reciprocal of the rank at which the first relevant item appears.

• Formula: MRR = (1 / |Q|) * Σ (1 / rank_i) for each query i.
• Interpretation: Higher scores indicate the first relevant result appears earlier in the list. A perfect MRR of 1.0 means the first result is always relevant.
• Application: Particularly important for user-facing systems where the quality of the very first result drives user satisfaction and can reduce latency by limiting the context window passed to the LLM.

NDCG evaluates a ranked list by accounting for the graded relevance of each item (e.g., highly relevant, somewhat relevant, not relevant) and penalizing relevant items that appear lower in the list.

• Core Principle: It uses a logarithmic discount factor, meaning the utility of a relevant document diminishes the further down the list it appears.
• Graded Relevance: Unlike binary metrics (relevant/not), NDCG works with scores like 0, 1, 2, 3, making it suitable for evaluating retrievers where some passages are more critical than others.
• Normalization: Scores are normalized against an ideal ranking (IDCG), producing a value between 0 and 1.

Hit Rate is a simple, binary metric measuring the percentage of queries for which at least one relevant document is found within the top K retrieved results.

• Calculation: (Number of queries with a hit) / (Total number of queries).
• Utility: Provides a high-level success rate for the retriever. A low Hit Rate indicates the retrieval system is fundamentally failing to find useful context for many queries, which will cascade into poor RAG performance.
• Threshold Sensitivity: The choice of K (e.g., Hit Rate @ 5 vs. @ 10) significantly impacts the score and should align with the number of passages typically passed to the LLM.

Dense retrieval evaluation often uses semantic similarity metrics rather than lexical overlap metrics like BLEU or ROUGE.

• Key Metric: BERTScore computes similarity between a retrieved passage and a query (or ground truth) using contextual embeddings from models like BERT. It correlates better with human judgment of relevance.
• Advantage: Captures paraphrasing and conceptual similarity that keyword matching (e.g., TF-IDF) would miss.
• Contrast: Traditional Precision@K and Recall@K still apply but require a binary relevance judgment, which can be derived from a semantic similarity threshold (e.g., a BERTScore > 0.8 is considered relevant).

This measures the improvement in retrieval quality gained by applying a secondary, more computationally expensive cross-encoder model to rerank the initial candidate set from the dense retriever.

• Typical Flow: A fast bi-encoder retrieves 100 candidates (high recall), then a precise cross-encoder reranks them to produce the final top 10 (high precision).
• Measurement: The lift in metrics like NDCG@10 or MAP after reranking versus the initial dense retrieval results.
• Engineering Trade-off: Evaluates whether the latency and compute cost of the reranker are justified by the quality gain for the downstream generation task.

Recall at K measures the proportion of all relevant documents for a query that are successfully retrieved within the top K results. It is the primary metric for assessing the comprehensiveness of a dense retriever.

• Formula: (Number of relevant docs in top K) / (Total number of relevant docs in corpus).
• Use Case: Critical for ensuring the generator has access to necessary information. A low R@K indicates the retriever is missing key context, leading to incomplete or incorrect answers.
• Typical K Values: R@5, R@10, or R@100, depending on how many documents are passed to the reranker or generator.

Mean Reciprocal Rank evaluates the rank of the first relevant document. It is the average of the reciprocal of the rank at which the first relevant document appears across multiple queries. It emphasizes the system's ability to place a useful result high in the list.

• Formula: For a set of queries Q, MRR = (1/|Q|) * Σ (1 / rank_i), where rank_i is the position of the first relevant document for the i-th query.
• Interpretation: A perfect MRR of 1.0 means the first retrieved document is relevant for every query. It is sensitive to the position of the first hit, making it crucial for user-facing systems where the top result is paramount.

NDCG evaluates a ranked list using graded relevance (e.g., scores of 0, 1, 2, 3), where higher-ranked relevant items contribute more to the score. It is the standard metric for evaluating ranking quality when relevance is not binary.

• Core Concept: It combines Cumulative Gain (sum of relevance scores), Discounting (reducing weight for lower ranks), and Normalization (comparing to an ideal ranking).
• Application in Dense Retrieval: Used to evaluate retrievers when documents have varying degrees of usefulness. A retriever that places a highly relevant (score=3) document at rank 1 scores much higher than one that places it at rank 10.
• NDCG@K (e.g., NDCG@10) is commonly reported.

Hit Rate is a simple, binary metric: for a given cutoff K, it measures the percentage of queries for which at least one relevant document is found in the top K results.

• Calculation: (Number of queries with a relevant doc in top K) / (Total number of queries).
• Utility: Provides a high-level success rate. A Hit Rate@5 of 0.95 means for 95% of queries, the system found something useful in the top 5. It is often the first metric checked to see if the retriever is functioning at a basic level.
• Difference from Recall: Recall measures coverage of all relevant documents per query, while Hit Rate only cares if any relevant document is found.

Standardized benchmarks are essential for comparing dense retrievers. Two primary suites are:

• BEIR (Benchmarking-IR): A heterogeneous benchmark containing 18 datasets across 9 tasks (e.g., fact-checking, question answering, citation prediction). It evaluates zero-shot retrieval performance, showing how well a model generalizes to new domains without task-specific fine-tuning.
• MTEB (Massive Text Embedding Benchmark): Evaluates text embeddings across 8 tasks (including retrieval) on 58 datasets. Its Retrieval category includes benchmarks like SciFact and NFCorpus.

Practice: Dense retrievers like Sentence-BERT, OpenAI embeddings, and Cohere models are routinely evaluated on these benchmarks using metrics like NDCG@10 and Recall@100 to establish leaderboards and performance baselines.

Retrieval Precision measures the fraction of retrieved documents that are relevant to the query. For dense retrieval, this is calculated from the top-K results returned by a vector similarity search.

• Formula: (Relevant docs in top-K) / K.
• Use Case: Critical for user-facing search where screen space is limited and irrelevant results degrade trust.
• Trade-off: High precision often comes at the cost of lower recall, as the system becomes more conservative.

Retrieval Recall measures the fraction of all relevant documents in the corpus that are successfully retrieved. In dense retrieval, this assesses the embedding model's ability to surface all pertinent passages.

• Formula: (Relevant docs retrieved) / (Total relevant docs in corpus).
• Use Case: Essential for research, legal discovery, or any task where missing information is costly.
• Challenge: Maximizing recall for large corpora is computationally intensive, as it requires scoring against many candidates.

Mean Average Precision (MAP) is a single-figure metric summarizing ranking quality across multiple queries. It calculates the mean of the Average Precision scores for each query.

• Average Precision (AP): Summarizes precision at each rank where a relevant document is found.
• Interpretation: A higher MAP indicates the system consistently ranks relevant documents higher across diverse queries.
• Application: The standard benchmark for academic retrieval datasets like MS MARCO and BEIR, providing a holistic view of a dense retriever's effectiveness.

Normalized Discounted Cumulative Gain (NDCG) evaluates ranked lists with graded relevance (e.g., highly relevant, somewhat relevant). It discounts the contribution of relevant documents based on their rank position.

• Graded Relevance: Unlike binary precision/recall, NDCG handles multi-level relevance judgments.
• Discounting: Gain from a document is divided by the log of its rank, penalizing relevant items that appear lower in the list.
• Normalization: Scores are divided by the Ideal DCG, providing a value between 0 and 1. This is crucial for evaluating rerankers that operate on the output of a dense retriever.

Hit Rate is a binary, query-level metric. It measures the proportion of queries for which at least one relevant document is found within the top K retrieved results.

• Formula: (Queries with ≥1 relevant doc in top-K) / (Total queries).
• Utility: Reflects user satisfaction for simple factoid queries where finding any correct answer is the goal.
• Context: Often reported as Hit Rate @ K (e.g., Hit Rate @ 5). A primary metric for evaluating the recall-oriented capability of a dense retrieval model in a RAG pipeline.

Reranking Effectiveness quantifies the improvement in retrieval quality achieved by applying a secondary, computationally intensive model to an initial candidate set from a fast dense retriever.

• Typical Architecture: A bi-encoder (dense retriever) fetches top 100 candidates; a cross-encoder (reranker) precisely scores and reorders them.
• Measured By: The lift in metrics like NDCG@10 or MAP after reranking.
• Engineering Trade-off: Rerankers are more accurate but slower, making this metric key for balancing latency and quality in production RAG systems.