Unsupervised Keyphrase Extraction

Methods like YAKE and RAKE compute scores using lexical and positional features without external corpora.

• Term Frequency: How often a word appears.
• Term Relatedness: Co-occurrence with other frequent words.
• Casing & Position: Bias toward capitalized terms or early-sentence placement.
• TF-ICF: Weights domain specificity by inverse corpus frequency. These lightweight algorithms are ideal for rapid prototyping and resource-constrained environments.

Algorithms like TextRank and SingleRank construct a co-occurrence graph where nodes are candidate words or phrases.

• Edges connect terms appearing within a sliding window.
• PageRank or weighted variants compute node centrality.
• Top-ranked nodes form the final keyphrases. This approach captures structural importance without semantic understanding, making it domain-independent and language-agnostic.

Methods like KeyBERT and EmbedRank leverage pre-trained transformer models to measure semantic alignment.

• Document Embedding: A dense vector representing the full text.
• Candidate Embedding: Vectors for each n-gram or phrase.
• Cosine Similarity: Ranks candidates by proximity to the document centroid.
• Maximal Marginal Relevance (MMR): Re-ranks to maximize relevance while minimizing redundancy. This bridges statistical and semantic approaches without fine-tuning.

Before scoring, unsupervised systems must generate a pool of candidate phrases.

• Part-of-Speech (POS) Tagging: Filters for noun phrases matching patterns like (JJ)*(NN)+.
• Stopword Delimitation: Splits text on stopwords to form contiguous chunks.
• N-gram Extraction: Generates all sequences up to a maximum length (e.g., trigrams).
• Phraseness Filtering: Removes candidates that are not linguistically well-formed. The quality of this step directly impacts final extraction precision.

Combining multiple unsupervised methods often yields more robust results than any single algorithm.

• Reciprocal Rank Fusion (RRF): Merges ranked lists by summing 1/(k + rank).
• Weighted Voting: Assigns confidence weights to each extractor's output.
• Borda Count: Aggregates positional scores across ranked lists. Ensemble techniques mitigate individual method biases, such as graph-based methods favoring frequent terms while embedding methods favor semantic centrality.

Unsupervised extraction is evaluated against human-annotated gold standards using rank-aware metrics.

• F1@K: Harmonic mean of precision and recall for the top-K predictions.
• Mean Reciprocal Rank (MRR): Averages 1/rank of the first correct keyphrase.
• Precision@K: Fraction of top-K predictions that are correct.
• Recall@K: Fraction of gold keyphrases captured in top-K. Benchmarks like KP20k and Inspec provide standardized evaluation for scientific and web documents.