Recall

Recall is formally defined as the fraction of relevant documents that are successfully retrieved.

Formula: Recall = (Number of Relevant Documents Retrieved) / (Total Number of Relevant Documents in the Collection)

• A score of 1.0 (or 100%) indicates the system retrieved every relevant document.
• A score of 0.0 means it retrieved none of the relevant documents.
• It is a set-based metric, initially agnostic to the ranking order of results.

Recall and Precision have an intrinsic, often inverse, relationship.

• High Recall, Low Precision: A system retrieves nearly all relevant documents but also returns many irrelevant ones (e.g., returning an entire database for a query).
• High Precision, Low Recall: A system returns mostly relevant documents but misses many others (e.g., returning only the single most perfect match).

Optimizing a system requires balancing this trade-off based on the use case. Search in a legal discovery context prioritizes high recall, while a chatbot answer might prioritize high precision.

In practical systems, users only examine the top results. Recall@k adapts the core metric to evaluate ranking quality.

Formula: Recall@k = (Number of Relevant Documents in Top-k Results) / (Total Number of Relevant Documents)

• It measures recall within a constrained result set, such as the first 5, 10, or 100 results.
• Critical for RAG: Determines if the correct context for an LLM is present within the retrieved passages fed into the context window. If Recall@k is low, the generator lacks necessary information, leading to hallucinations or incomplete answers.

Calculating recall requires definitive knowledge of all relevant documents for a query—the ground truth or relevance judgments.

• In academic benchmarks (e.g., TREC, BEIR), this is established by human assessors pooling results from many systems.
• In enterprise settings, establishing ground truth is challenging and often requires human-in-the-loop evaluation or proxy metrics.
• Without accurate ground truth, reported recall scores are estimates. This makes Inter-Annotator Agreement and rigorous labeling protocols essential for reliable evaluation.

Different retrieval architectures directly influence achievable recall.

• Sparse Retrievers (e.g., BM25): Excel at lexical recall for keyword matches but struggle with semantic variation.
• Dense Retrievers (e.g., Embedding Models): Excel at semantic recall for conceptual matches but can miss exact keyword terms.
• Hybrid Retrieval: Combines sparse and dense methods (e.g., weighted scores) to maximize overall recall by covering both lexical and semantic relevance. This is a standard approach in production RAG to mitigate the weaknesses of any single method.

Within Retrieval-Augmented Generation, recall underpins several specialized evaluation metrics.

• Contextual Recall: Measures the proportion of statements from a ground truth answer that are attributable to the retrieved context. Directly tests the retriever's ability to provide necessary information.
• Faithfulness/Groundedness: While a generator metric, low faithfulness often stems from low recall—the LLM invents facts because the correct evidence wasn't retrieved.
• Answer Relevance: A poor answer can result from low recall (missing key info) or low precision (context is noisy).

These metrics, part of frameworks like RAGAS, decompose overall system performance into retrievable and generatable components.

In e-Discovery and regulatory compliance audits, the primary objective is to ensure no relevant document is missed. Systems are optimized for high recall, often at the expense of lower initial precision, because the cost of missing a single relevant document (e.g., in litigation) is catastrophic. Subsequent human review or Cross-Encoder reranking is used to filter the high-recall result set. This is a classic example of a recall-oriented search workflow.

When clinicians or researchers search for all known studies on a rare drug side effect or genetic mutation, missing a key paper could impact patient care or invalidate research. Systematic reviews mandate exhaustive retrieval. Search systems in this domain use query expansion with MeSH terms and synonyms, and often employ hybrid retrieval combining BM25 with dense vector search to maximize the chance of retrieving all relevant documents, regardless of the specific terminology used.

The retrieval stage of a Retrieval-Augmented Generation system is fundamentally a recall-critical task. The goal is to fetch all context chunks that contain information necessary for the LLM to generate a complete and faithful answer. If relevant context is missed (low recall), the LLM is prone to hallucination. Engineers often implement a multi-stage retrieval strategy: a high-recall first stage (e.g., using a broad vector search) fetches many candidates, which are then reordered by a high-precision reranker before being passed to the LLM.

Patent examiners must identify all existing prior art to assess an invention's novelty. Missing a single relevant patent can lead to an invalid grant. This requires near-perfect recall across global patent databases. Systems use sophisticated semantic search across titles, claims, and descriptions, combined with citation graph analysis to uncover connected documents. The metric Recall@100 or Recall@200 is often used to evaluate these systems, acknowledging the need to review many results.

Security operations centers scan data feeds, forums, and logs for indicators of compromise. The priority is to detect all potential threats (high recall), generating many false positives that are later triaged, rather than risk missing a true positive. Anomaly detection models and pattern matching over unstructured text are tuned for sensitivity. Low recall in this context equates to a security breach.

Achieving high recall involves explicit engineering trade-offs:

• vs. Precision: High-recall result sets are typically larger and noisier. This increases the burden on downstream processes or users to identify truly relevant items.
• vs. Latency: Techniques that boost recall, such as searching multiple vector indexes with different parameters, performing extensive query reformulation, or using larger context windows, increase computational cost and retrieval latency.
• The Balance: The optimal point on the precision-recall curve is determined by the application's cost function—whether missing an item (low recall) or reviewing false alarms (low precision) is more expensive.

Precision is the fraction of retrieved documents that are relevant. It answers: "Of all the documents I retrieved, how many were correct?"

• Trade-off with Recall: Optimizing for high recall often lowers precision (retrieving more items increases the chance of including irrelevant ones).
• Balanced Metric: The F1 Score is the harmonic mean of precision and recall, providing a single metric that balances both concerns.
• Context: In a RAG system, high precision ensures the language model receives mostly relevant context, reducing noise and potential for hallucination.

These are ranking-aware versions of precision and recall, evaluated only on the top k retrieved results.

• Precision@k: Measures the proportion of relevant documents within the first k results. Critical for user-facing systems where only the top results are viewed.
• Recall@k: Measures the proportion of all relevant documents found within the top k results. A Recall@k of 1.0 means all relevant documents are in the top-k, which is often the practical goal.
• Application: Setting k to match the context window size of a language model (e.g., k=10) directly evaluates retrieval quality for RAG.

Mean Average Precision (MAP) evaluates ranking quality by calculating the mean of Average Precision scores across multiple queries.

• Average Precision (AP): For a single query, it is the average of precision values computed at each point a new relevant document is retrieved. It rewards systems that rank relevant documents higher.
• Interpretation: A high MAP indicates the system consistently retrieves many relevant documents and ranks them near the top of the list. It is a standard metric in academic benchmarks like TREC.

NDCG is a ranking metric for graded relevance, where documents can be partially relevant (e.g., rated on a scale of 0-3).

• Gain: Each document contributes a "gain" based on its relevance grade.
• Discounted: The gain is reduced (discounted) logarithmically based on its rank position, penalizing relevant documents that appear lower in the list.
• Normalized: The score is divided by the ideal DCG (the best possible ranking), resulting in a value between 0 and 1.
• Use Case: Essential for evaluating systems where relevance is not binary, such as product or document search.

Contextual Recall is a RAG-specific metric from frameworks like RAGAS that measures retrieval completeness from the answer's perspective.

• Calculation: It checks what proportion of statements in a ground truth answer can be attributed to (are supported by) the retrieved context.
• Difference from Standard Recall: Standard recall measures coverage of relevant documents. Contextual Recall measures coverage of relevant information needed to formulate a correct answer.
• Goal: A high Contextual Recall score indicates the retriever provided the necessary evidence for the LLM to generate a factually complete answer.