Salient Terms

The saliency of a term w for a topic k is defined by a tunable relevance score: r(w,k|λ) = λ * log P(w|k) + (1-λ) * log (P(w|k)/P(w)). This formula interpolates between pure topic probability and the lift (ratio of topic frequency to corpus frequency).

• λ = 1.0: Ranks terms solely by their frequency within the topic (P(w|k)).
• λ = 0.0: Ranks terms exclusively by their lift, surfacing words that are highly specific to the topic but may be rare overall.
• Optimal λ (often ~0.6): Balances frequency and exclusivity to surface terms that are both common in the topic and distinctive from the background corpus.

A term's lift is the ratio P(w|k) / P(w). A high lift indicates that a word appears in the topic far more often than it does in the general corpus, making it a strong diagnostic signal.

• High Lift, Low Frequency: A rare word that is highly concentrated in one topic (e.g., 'allele' in a genetics topic) has high lift but may be invisible in a pure frequency ranking.
• Low Lift, High Frequency: A common word like 'data' may appear frequently in a topic but has low lift because it is ubiquitous across the entire corpus.
• Saliency metrics penalize low-lift terms, preventing generic corpus-wide frequent words from dominating topic descriptions.

Saliency and topic coherence serve different purposes in model evaluation. Saliency is a per-term ranking metric for human interpretation, while coherence is a per-topic aggregate score measuring semantic interpretability.

• Saliency: Ranks individual words within a single topic to aid a human in quickly grasping the topic's theme.
• Coherence (e.g., C_V): Computes a single score for a topic by measuring the co-occurrence of its top-N words in a reference corpus, used to evaluate overall model quality.
• Complementary Use: A topic can have high coherence (its top words are semantically related) while the saliency ranking helps identify which of those words is the single best label for the topic.

The marginal probability P(w)—the frequency of a term across the entire corpus—is the critical baseline against which topic-specific frequency is compared. Without this normalization, topic descriptions are dominated by stop words and domain-general jargon.

• Stop Word Suppression: Words like 'the', 'is', and 'of' have high P(w) and thus very low lift, automatically suppressing them in saliency rankings.
• Domain-General Terms: In a technical corpus, words like 'system' or 'process' may have high overall frequency. Saliency metrics penalize these, ensuring they don't appear as top terms for every topic.
• Corpus Design Matters: The composition of the background corpus directly impacts lift calculations. A well-curated corpus ensures that true domain-specific signals are amplified.

The ultimate goal of identifying salient terms is to facilitate topic labeling—assigning a concise, human-readable phrase to a discovered latent theme. Salient terms serve as the primary input for both manual and automatic labeling processes.

• Manual Labeling: A human analyst inspects the top 10-15 salient terms (at λ ≈ 0.6) and synthesizes a descriptive label.
• Automatic Labeling: Algorithms can select the highest-ranked salient term or generate a short phrase using the top salient terms as candidates.
• Multi-Word Expressions: Advanced techniques extract salient bigrams and trigrams (e.g., 'neural network', 'climate change') to provide more precise labels than single unigrams.

An evaluation metric that measures the semantic interpretability of a topic by quantifying the degree of co-occurrence between its top-ranked words in reference corpora.

• Provides a quantitative proxy for human judgment of topic quality
• High coherence scores indicate that top words frequently appear together in documents
• Often calculated using Pointwise Mutual Information (PMI) or normalized variants
• Essential for selecting the optimal number of topics (K) in a model

A robust topic coherence measure that combines normalized pointwise mutual information with cosine similarity over word context vectors.

• Correlates highly with human interpretability judgments
• Uses a sliding window to create context vectors for top-N words
• Computes cosine similarity between word context vectors
• Aggregates pairwise similarities into a final coherence score
• Currently the most widely adopted coherence metric in topic modeling research

An information-theoretic measure of association between two words, used as a foundational component in calculating topic coherence scores.

• Defined as the log ratio of the joint probability to the product of marginal probabilities
• PMI(w1, w2) = log(P(w1, w2) / (P(w1) * P(w2)))
• Positive values indicate words co-occur more often than chance
• Zero indicates statistical independence
• Normalized PMI (NPMI) constrains values to [-1, 1] for better interpretability
• Forms the mathematical backbone of most coherence metrics

An evaluation method where human annotators identify an injected outlier word within a topic's top terms, measuring the interpretability of the latent space.

• A topic's top 5 words are presented alongside 1 random intruder word
• The intruder has low probability in the topic but high probability in another topic
• Topic Intrusion Score = percentage of intruders correctly identified
• High scores indicate topics are semantically distinct and interpretable
• Complements coherence metrics with direct human validation

A metric assessing the uniqueness of topics by calculating the percentage of unique words across the top-N terms of all discovered topics in a model.

• High diversity indicates topics are distinct and non-redundant
• Low diversity suggests topic duplication or overly overlapping themes
• Often reported alongside coherence to balance interpretability with coverage
• A model with high coherence but low diversity may be overfitting to a few narrow themes
• Used to diagnose when the number of topics (K) is set too high