Phraseness

Measures the internal grammatical integrity of a candidate sequence. High phraseness requires that words form a valid syntactic constituent, such as a noun phrase or verb phrase, rather than a random collocation.

• Uses Part-of-Speech (POS) tagging to validate sequences like JJ + NN (e.g., "machine learning")
• Rejects sequences ending in stop words or prepositions (e.g., "learning about")
• Often implemented via regular expression chunking over POS-tagged text

Phraseness scoring enforces strict rules about stop word delimiters. Candidate phrases are typically extracted from text segments bounded by punctuation or function words.

• Sequences must not begin or end with a stop word
• Internal stop words are permitted only in specific patterns (e.g., "United of America")
• This heuristic dramatically reduces the candidate pool while preserving multi-word expressions

Evaluates whether words appear together more frequently than chance would predict, using statistical association measures. This distinguishes true multi-word expressions from accidental co-occurrences.

• Pointwise Mutual Information (PMI): Measures the probability of co-occurrence versus independent occurrence
• Chi-squared and log-likelihood ratio tests identify statistically significant bigrams and trigrams
• High collocation strength indicates the sequence functions as a single lexical unit

Assesses whether a candidate n-gram follows the morphological and syntactic rules of the language. This filters out sequences that are mathematically frequent but linguistically invalid.

• Validates subject-verb agreement and adjective-noun ordering
• Uses language models to score the probability of the sequence being well-formed
• Distinguishes "artificial intelligence" (high grammaticality) from "intelligence artificial" (low grammaticality in English)

Phraseness is one of two orthogonal scoring dimensions in keyphrase extraction, alongside informativeness. A candidate must score high on both to be selected.

• Phraseness: Is it a valid phrase? (linguistic quality)
• Informativeness: Does it capture the document's topic? (semantic relevance)
• A sequence like "the quick brown" may have high phraseness but low informativeness, while "quantum chromodynamics" scores high on both in a physics paper

The C-Value method explicitly incorporates phraseness by evaluating nested multi-word terms. Longer phrases receive higher scores when they appear independently of shorter sub-phrases.

• C-Value = log₂(|phrase|) × (frequency - average frequency of containing longer phrases)
• NC-Value extends this with context words that signal termhood
• These algorithms penalize sequences that only appear as parts of longer terms, improving phraseness precision

The complementary scoring component to Phraseness. While Phraseness measures linguistic well-formedness, Informativeness quantifies how well a candidate phrase captures the core topical content of a document. It evaluates domain specificity, often using techniques like TF-IDF or semantic similarity against the document centroid. A high-quality keyphrase must score well on both dimensions.

The process of assigning a numerical weight to each candidate phrase. Phraseness is a critical feature within this scoring function. Other features include:

• Term Frequency: How often the phrase appears.
• Position: Occurrence in the title or abstract.
• Length: Penalizing overly long or short n-grams. The final score is often a weighted linear combination of these features.

The initial step that produces the set of potential n-grams to be scored. This stage typically uses Part-of-Speech (POS) tagging and noun phrase chunking with regex patterns like (JJ)*(NN)+. The quality of this generation directly limits the effectiveness of Phraseness scoring, as a linguistically perfect phrase cannot be extracted if it was never generated as a candidate.

The task of accurately identifying the start and end tokens of a multi-word keyphrase. Phraseness is inherently linked to boundary detection, as a sequence with high Phraseness exhibits strong internal cohesion and clear separation from surrounding tokens. Sequence labeling models like BiLSTM-CRF explicitly learn these boundaries, implicitly encoding Phraseness in their transition scores.

A lightweight, unsupervised keyphrase extraction algorithm that relies heavily on statistical text features. YAKE computes a score for each candidate n-gram based on five features, including Term Frequency, Term Relatedness to Context, and crucially, a Phraseness-like feature that penalizes candidates containing stopwords at the beginning or end, implicitly favoring well-formed phrases without explicit linguistic parsing.

A technique that combines the ranked outputs of multiple keyphrase extraction algorithms. Since different algorithms model Phraseness and Informativeness with varying biases, ensemble methods like Reciprocal Rank Fusion (RRF) can produce a more robust final ranking. An algorithm strong in linguistic Phraseness can compensate for one focused purely on statistical salience.