F1@K

Measures the fraction of the top-K predicted keyphrases that are actually correct (present in the gold standard). It answers: Of the K keyphrases we suggested, how many were right?

• Formula: (Number of correct predictions in top-K) / K
• Focus: Minimizing false positives and irrelevant suggestions
• Use case: Critical when users only scan the first few results and irrelevant terms erode trust
• Example: If a system predicts 10 keyphrases and 7 match the author's labels, Precision@10 = 0.7

Measures the fraction of the total gold-standard keyphrases that are successfully captured within the top-K predictions. It answers: Of all the correct keyphrases, how many did we find?

• Formula: (Number of correct predictions in top-K) / (Total number of gold-standard keyphrases)
• Focus: Minimizing false negatives and missed concepts
• Limitation: Recall can be artificially inflated by increasing K, which is why it must be paired with Precision
• Example: If a document has 8 author-assigned keyphrases and the top-10 predictions include 4 of them, Recall@10 = 0.5

Evaluates ranking quality by averaging the reciprocal of the rank position where the first correct keyphrase appears. Unlike F1@K, MRR focuses purely on the highest-ranked hit.

• Formula: MRR = (1/|Q|) × Σ (1 / rank_i), where rank_i is the position of the first relevant keyphrase
• Strength: Heavily rewards systems that place correct answers at the very top
• Contrast with F1@K: MRR cares about order; F1@K treats the top-K set as unordered
• Example: If the first correct keyphrase appears at rank 1, reciprocal rank = 1.0; if at rank 4, reciprocal rank = 0.25

A re-ranking algorithm that balances relevance against redundancy when selecting the final keyphrase set. It prevents the system from returning near-duplicate phrases.

• Mechanism: Iteratively selects phrases that maximize λ × Relevance(doc, phrase) − (1−λ) × max Similarity(phrase, already_selected)
• λ parameter: Controls the diversity-relevance trade-off; higher λ favors relevance
• Synergy with F1@K: MMR is applied post-candidate scoring to diversify the top-K set before F1 evaluation
• Common pitfall: Without MMR, systems often return "machine learning," "deep learning," and "neural networks" as three separate keyphrases when one would suffice

A robust data fusion method that combines ranked lists from multiple keyphrase extraction systems into a single consensus ranking. It is the ensemble counterpart to single-model F1@K evaluation.

• Formula: RRF_score(d) = Σ (1 / (k + rank_i(d))), where k is a constant (typically 60)
• Advantage: Does not require score normalization across different algorithms, unlike weighted sum fusion
• Application: Used in ensemble scoring pipelines where outputs from TextRank, KeyBERT, and a supervised model are merged
• Relationship to F1@K: RRF produces the final ranked list on which F1@K is then computed

The process of assigning a numerical weight to each candidate phrase based on features such as phraseness (linguistic well-formedness) and informativeness (topical relevance). This step directly determines the ranking evaluated by F1@K.

• Common features: TF-IDF, position in document, semantic similarity to document embedding, capitalization patterns
• Phraseness: Filters out grammatically incomplete n-grams like "learning algorithm the"
• Informativeness: Ensures generic phrases like "this paper" are downweighted
• Pipeline role: Candidate scoring is the final step before top-K selection; poor scoring guarantees poor F1@K regardless of candidate generation quality