Keyphrase Generation

The foundational architecture for keyphrase generation is the encoder-decoder framework. The encoder (e.g., a Transformer) reads the source document and compresses it into a dense context vector. The decoder then autoregressively generates a sequence of keyphrases, one token at a time, often separated by a delimiter.

• One2Seq: A seminal model that generates multiple keyphrases in a single sequence.
• Copy Mechanism: Allows the decoder to copy rare or out-of-vocabulary words directly from the source text, crucial for generating present keyphrases accurately.
• Target: NLP Engineers building custom generation pipelines.

A key distinction in generation is the model's ability to produce two types of keyphrases. Present keyphrases appear verbatim in the source text, while absent keyphrases are semantically relevant but do not exist as contiguous spans.

• Present Generation: Relies heavily on a copy mechanism to point to tokens in the source.
• Absent Generation: Requires the model to synthesize new phrases from its learned vocabulary, demanding a deeper semantic understanding of the text.
• Evaluation: Models are often benchmarked on F1@K for both present and absent subsets separately.

Traditional Seq2Seq models impose an artificial order on the target keyphrase set, which is inherently unordered. The One2Set paradigm addresses this by removing the order bias.

• Set Loss: Uses a loss function like the K-step target assignment loss that is permutation-invariant, aligning predicted phrases with the most similar ground-truth phrases.
• Control Codes: Models like CatSeq use control codes to indicate the presence or absence of a keyphrase, guiding the decoder's generation mode.
• Benefit: Prevents the model from learning spurious sequential correlations between unrelated keyphrases.

To capture the structural relationship between a document and its keyphrases, hierarchical models process text at multiple granularities. A hierarchical encoder first builds sentence-level representations, then aggregates them into a document-level vector.

• Hierarchical Attention: The decoder can attend to both word-level and sentence-level context, improving the relevance of generated phrases.
• Title Guidance: Some architectures encode the document title separately and use it as an additional signal to the decoder, as titles often contain the most salient concepts.
• Use Case: Highly effective for long-form scientific documents where keyphrases summarize distinct sections.

A common failure mode is generating repetitive or near-duplicate keyphrases. Reinforcement Learning (RL) can be used to directly optimize for diversity and coverage.

• Self-Critical Sequence Training (SCST): Uses the model's own output as a baseline to reduce variance during RL training.
• Reward Shaping: Combines F1 score (relevance) with a repetition penalty or semantic diversity metric to encourage a varied set of keyphrases.
• Result: Produces a more comprehensive set of keyphrases that covers different facets of the document.

Keyphrase generation is evaluated against a gold-standard set of author-assigned or expert-curated keyphrases. The primary metrics are precision, recall, and F1-score computed on the top-K predictions.

• F1@5 / F1@10: The harmonic mean of precision and recall for the top 5 or 10 generated keyphrases. This is the standard benchmark.
• Mean Reciprocal Rank (MRR): Measures the rank of the first correct keyphrase, useful when the order of generation matters.
• Stemming/Levenshtein: Exact match is too strict; evaluation often uses Porter Stemming or a Levenshtein distance threshold to count near-matches as correct.