Recall@K

The complementary metric to Recall@K that measures the proportion of retrieved items that are relevant within the top-K results.

• Formula: (Number of relevant items in top-K) / K
• Focus: Evaluates the signal-to-noise ratio of the retrieval system
• Trade-off: High recall often comes at the cost of lower precision
• Use case: Critical when users only examine the first few results and irrelevant items erode trust

An evaluation metric that averages the reciprocal of the rank at which the first relevant document is retrieved across a set of queries.

• Formula: MRR = (1/|Q|) * Σ (1 / rank_i)
• Focus: Rewards systems that place the first relevant result as high as possible
• Range: 0 to 1, where 1 means the first result is always relevant
• Use case: Navigational queries and question-answering where users need exactly one correct answer

A rank-aware evaluation metric that accounts for the position of relevant documents and their graded relevance levels, not just binary relevance.

• Key insight: Relevant documents at higher ranks contribute more to the score
• Discount function: Logarithmically reduces the contribution of lower-ranked items
• Normalization: Divided by the ideal DCG to produce a score between 0 and 1
• Use case: Scenarios with multi-level relevance judgments, such as "perfect," "good," and "fair" matches

The fundamental retrieval operation that returns the K documents with the highest similarity scores to a query from a vector index or search engine.

• Parameter K: A tunable hyperparameter balancing coverage and latency
• Pipeline role: Top-K retrieval feeds candidates into downstream re-ranking or generation stages
• Recall@K dependency: The metric directly evaluates the quality of this operation
• Typical values: K=100 or K=1000 for first-stage retrieval, K=10 for final presentation

A comprehensive retrieval metric that computes the average precision at each relevant document's position and then averages across all queries.

• Calculation: AP = (1 / total relevant) * Σ (Precision@k * rel(k))
• Characteristic: Rewards systems that rank all relevant documents highly, not just the first one
• Sensitivity: Penalizes both missed relevant documents and poorly ranked ones
• Use case: Ad-hoc retrieval benchmarks where users value comprehensive recall across multiple relevant items

The harmonic mean of Precision and Recall, providing a single balanced metric when both false positives and false negatives carry significant cost.

• Formula: F1 = 2 * (Precision * Recall) / (Precision + Recall)
• Harmonic mean property: Penalizes extreme imbalances more than the arithmetic mean
• F1@K variant: Computed at a specific cutoff K to evaluate retrieval quality at that depth
• Use case: When you need a single number to compare systems and both missing relevant items and returning irrelevant ones are equally problematic