Recall@K

Recall@K measures the proportion of relevant items retrieved in the top K results relative to the total number of relevant items that exist in the corpus.

• Formula: Recall@K = (Number of Relevant Items in Top K) / (Total Relevant Items in Corpus)
• Range: Always between 0.0 and 1.0, where 1.0 indicates perfect completeness.
• Contrast with Precision@K: While Precision@K measures the purity of the top K results, Recall@K measures their coverage.
• Example: If 10 relevant documents exist for a query and a system returns 7 of them in the top 20 results, Recall@20 = 0.7.

The choice of K fundamentally changes what the metric evaluates and must align with the user experience being designed.

• Small K (e.g., K=5): Evaluates the quality of the first screen or snippet. Critical for answer engines where only the top few chunks are fed to a generator.
• Large K (e.g., K=100): Evaluates the system's ability to find all relevant items, often used during evaluation-driven development to tune ANN index parameters.
• Trade-off: A larger K makes it easier to achieve high recall but increases latency and the computational cost of downstream re-ranking.

Approximate Nearest Neighbor (ANN) algorithms like HNSW or DiskANN introduce a direct recall-performance trade-off.

• ANN Recall: In vector search, "recall" often refers to the proportion of true nearest neighbors found by the approximate algorithm compared to an exact brute-force search.
• Tuning: Increasing the ef_search parameter in HNSW improves ANN recall but increases query latency.
• Impact: Low ANN recall directly caps the maximum achievable Recall@K for the overall retrieval pipeline, as relevant vectors are lost before any re-ranking occurs.

Recall@K has critical limitations that must be understood to avoid misleading evaluation results.

• Total Relevant Count Problem: The denominator requires knowing the absolute number of relevant items in the corpus, which is often unknown in large-scale production systems.
• Binary Relevance Assumption: The metric treats all relevant items as equally important, ignoring graded relevance where one document might be far more authoritative than another.
• Ignores Ranking Order: A system that places all relevant items at positions 90-100 receives the same Recall@100 score as one that places them at positions 1-10, masking poor ranking quality.

Recall@K is the primary metric for evaluating the retrieval stage of a pipeline, before re-ranking and generation.

• High-Recall Retrieval: Modern Retrieval-Augmented Generation (RAG) systems often combine dense vector search and sparse BM25 using Reciprocal Rank Fusion (RRF) to maximize recall before a final cross-encoder re-ranker.
• Cascading Architecture: The retrieval stage optimizes for high recall (e.g., K=100), while the re-ranking stage optimizes for high precision at a smaller K (e.g., K=5).
• Evaluation: Recall@K is measured at the retrieval output, while metrics like Faithfulness or Answer Correctness evaluate the final generated output.

Consider a user query: "How do I configure a circuit breaker for a vector database connection?"

• Ground Truth: 8 relevant documents exist in the index covering circuit breaker patterns, connection pooling, and backpressure.
• System Output (Top 10): Returns 6 of the 8 relevant documents, plus 4 irrelevant ones.
• Result: Recall@10 = 6 / 8 = 0.75
• Interpretation: The system found 75% of the available relevant information. The missing 2 documents might be due to poor chunking or an embedding model that failed to capture the semantic relationship between "circuit breaker" and "fault tolerance."

The complementary metric to Recall@K, measuring the proportion of retrieved items that are relevant within the top K results. While Recall@K quantifies completeness, Precision@K quantifies signal-to-noise ratio. A system with high recall but low precision retrieves all relevant documents but buries them in irrelevant results, frustrating users and wasting context window tokens. The F1-Score is the harmonic mean of Precision and Recall, providing a single balanced metric.

An evaluation metric that focuses on the rank position of the first relevant item. MRR is calculated by averaging the reciprocal of the rank at which the first relevant document appears across a set of queries. A score of 1.0 means the first result was always relevant. This metric is critical for answer engine architectures where the top result is used directly for synthesis, making the position of the first correct answer more important than the total number of correct answers.

A ranking-aware metric that accounts for graded relevance, not just binary relevance. NDCG assigns higher scores to relevant documents appearing at the top of the list and discounts documents further down. This is essential when relevance is not binary—for example, a document that perfectly answers a query is more valuable than one that is tangentially related. NDCG normalizes the score against an ideal ranking to produce a value between 0 and 1.

A binary metric measuring whether at least one relevant document appears in the top K results. Unlike Recall@K, which counts all relevant documents, Hit Rate only checks for the presence of any relevant item. A Hit Rate of 1.0 at K=10 means every query had at least one relevant result in the top 10. This is a useful pass/fail signal for answer generation systems where a single good document is sufficient for synthesis.

The fundamental engineering tension between search speed and result accuracy in approximate nearest neighbor algorithms. Tuning ANN parameters like HNSW ef_search or IVF nprobe directly impacts Recall@K. A higher ef_search traverses more graph nodes, improving recall at the cost of increased latency. Production systems must balance this trade-off within their latency budget, often accepting 99% ANN recall to achieve sub-10ms query times.

An algorithm for combining multiple ranked result lists into a single unified ranking. RRF scores each document by summing 1/(k + rank) across all retrieval sources, where k is a constant (typically 60). This elegantly handles the fusion of dense vector results, sparse BM25 results, and metadata-filtered results without requiring score normalization. The fused ranking directly impacts Recall@K by aggregating diverse retrieval signals.