TextRank

TextRank constructs a co-occurrence graph where vertices represent words or phrases, and edges are weighted by their co-occurrence frequency within a sliding window. The algorithm then applies a recursive PageRank-style computation to score each vertex based on the importance of its neighbors. The core formula iterates until convergence:

S(Vi) = (1 - d) + d * Σ (w_ji / Σ w_jk) * S(Vj)

• d: Damping factor (typically 0.85), controlling the probability of jumping to a random vertex
• w_ji: Weight of the edge between vertex j and vertex i
• Convergence: Scores stabilize when the difference between iterations falls below a threshold

TextRank requires no labeled training data and operates independently of domain-specific corpora. It relies solely on the intrinsic structure of the input document:

• No external knowledge bases: Unlike Wikification approaches, TextRank does not require Wikipedia or ontologies
• Language agnostic: Works on any language with tokenization and POS tagging support
• Single-document processing: Extracts keyphrases from one document at a time without needing a background corpus
• Bootstrapping: Uses the document's own lexical cohesion patterns to determine salience

Before graph construction, TextRank applies syntactic filters to select candidate phrases. The original implementation uses part-of-speech (POS) tag patterns to extract noun phrases:

• Pattern: Sequences of zero or more adjectives followed by one or more nouns (JJ)*(NN)+
• Stopword handling: Stopwords are excluded from candidate phrases but preserved for co-occurrence context
• Multi-word phrases: Post-processing merges adjacent single-word candidates that appear consecutively in the text
• Filtering rationale: Restricts the graph to content-bearing terms, reducing noise and computational complexity

After the PageRank scores converge, TextRank applies post-processing steps to produce the final ranked keyphrase list:

• Phrase merging: Adjacent single-word candidates with high individual scores are combined into multi-word phrases
• Score aggregation: The score of a merged phrase is the sum of its constituent word scores
• Top-K selection: The highest-scoring phrases are returned, typically the top 10-20 candidates
• Redundancy control: Unlike MMR-based approaches, TextRank does not explicitly penalize redundancy, which can lead to semantically overlapping keyphrases

Beyond keyphrase extraction, TextRank extends naturally to extractive summarization by treating sentences as vertices:

• Sentence graph: Vertices are sentences; edges are weighted by inter-sentence similarity (e.g., TF-IDF cosine or lexical overlap)
• Similarity threshold: Edges are only created when similarity exceeds a minimum threshold, preventing a fully connected graph
• Ranking: PageRank identifies the most central sentences that collectively represent the document's core content
• Multi-document variant: Can be applied across document clusters by building a unified sentence graph

TextRank offers a compelling balance of simplicity and effectiveness, but has known trade-offs:

Strengths:

• No training required: Fully unsupervised, deployable on any text immediately
• Interpretable: Graph structure and scores are transparent and auditable
• Lightweight: Minimal computational overhead compared to embedding-based methods like KeyBERT

Limitations:

• Cannot extract absent keyphrases: Only identifies phrases present verbatim in the text
• Frequency bias: Favors frequently occurring terms, potentially missing rare but salient concepts
• No semantic understanding: Relies on surface co-occurrence rather than deep contextual meaning

The foundational family of algorithms that TextRank belongs to. These methods represent text units (words or phrases) as vertices in a network and use structural centrality measures to determine salience.

• Mechanism: Builds a co-occurrence graph and iteratively computes vertex scores until convergence.
• Key Variants: TextRank, SingleRank, TopicRank, and PositionRank.
• Advantage: Unsupervised and domain-independent, requiring no labeled training data.

The specific network representation that powers TextRank. Nodes represent candidate words or phrases, while edges are weighted by their frequency of co-occurrence within a sliding window.

• Window Size: Typically set between 2 and 10 words to capture local syntactic relations.
• Edge Weighting: Can be binary or weighted by the number of times two terms appear together.
• Role: This graph structure directly determines which terms reinforce each other during the PageRank scoring iterations.

The process of assigning a numerical weight to each candidate phrase after graph construction. TextRank uses PageRank convergence scores as the primary relevance signal.

• Post-Processing: Scores can be combined with positional features or frequency boosts.
• Top-K Selection: The highest-scoring candidates are selected as the final keyphrases.
• Contrast: Unlike statistical methods like TF-IDF, the score reflects global structural importance rather than local frequency.

The standard evaluation metric for measuring keyphrase extraction performance against a gold-standard set. It computes the harmonic mean of precision and recall for the top-K predicted keyphrases.

• Precision: The fraction of predicted keyphrases that match the ground truth.
• Recall: The fraction of ground-truth keyphrases that are successfully predicted.
• Usage: F1@5 and F1@10 are common benchmarks on datasets like KP20k and Inspec.

A re-ranking algorithm often applied to TextRank's output to improve diversity. It balances a phrase's relevance to the document against its redundancy with already selected keyphrases.

• Lambda Parameter: Controls the trade-off between relevance and novelty.
• Benefit: Prevents the top-K list from being dominated by semantically identical phrases.
• Integration: Applied as a post-processing step after the initial graph-based ranking is complete.

The critical pre-processing step before TextRank can be applied. This stage produces a set of potential n-grams using Part-of-Speech (POS) tagging and noun phrase chunking.

• Filters: Sequences matching specific POS patterns (e.g., adjective-noun) are retained.
• Stopword Delimiting: Multi-word phrases are often identified by splitting text on stopwords.
• Impact: The quality of candidate generation directly limits the maximum achievable extraction accuracy.