pyLDAvis

Projects topic centroids onto a 2D plane using multidimensional scaling (MDS) based on the Jensen-Shannon divergence between topic-term distributions. Each bubble represents a topic, with its area proportional to the topic's prevalence in the corpus. Overlapping bubbles indicate semantically similar topics, while distant bubbles suggest distinct themes. This allows practitioners to quickly assess model quality and identify redundant topics that may require merging or adjusting the Number of Topics (K) hyperparameter.

Introduces a tunable relevance metric that ranks terms within a selected topic. The formula balances a term's probability within the topic against its lift over the corpus-wide background frequency: relevance(term | topic) = λ * p(term | topic) + (1 - λ) * p(term | topic) / p(term). Adjusting the λ slider from 0 to 1 lets users shift the ranking from distinct, topic-specific terms to frequent, general terms. This directly addresses the limitation of raw top-N word lists, which often include generic stopwords or corpus-wide frequent terms.

Displays the top 30 most salient terms for the selected topic as a horizontal bar chart. The bar overlay shows both the corpus-wide frequency and the topic-specific frequency of each term. This dual encoding helps analysts understand why a term is ranked highly—whether because it is genuinely distinctive to the topic or simply a high-frequency word in the overall corpus. This visualization directly supports qualitative Topic Coherence assessment.

Uses Principal Coordinate Analysis (PCoA) , also known as classical multidimensional scaling, to compute the intertopic distance map. The algorithm takes the matrix of pairwise Jensen-Shannon divergences between all topic-term distributions and finds a low-dimensional embedding that preserves these distances. This is computationally efficient for the typical number of topics (K < 100) and provides a deterministic projection, unlike t-SNE, which can produce different layouts on different runs.

Designed to work directly with Gensim LDA models. A single function call—pyLDAvis.gensim.prepare(lda_model, corpus, dictionary)—extracts the topic-term distributions, document-topic distributions, and vocabulary mappings needed for the visualization. The output is a self-contained HTML/JavaScript object that can be rendered in a Jupyter notebook or saved as a standalone web page, requiring no server-side dependencies for sharing interactive results with stakeholders.

Hovering over any topic bubble in the intertopic distance map dynamically updates the term bar chart to show that topic's most relevant terms. This direct manipulation interaction creates a tight feedback loop for exploratory analysis. Users can rapidly scan multiple topics, compare their top terms, and identify semantically overlapping clusters. The selected topic is highlighted, and its bubble is outlined, making it easy to track focus during deep-dive sessions into model interpretability.

The primary visualization component of pyLDAvis that uses multidimensional scaling (MDS) to project topic centroids onto a two-dimensional plane. Each bubble represents a topic, with its area proportional to the topic's prevalence in the corpus. The spatial distance between bubbles indicates the semantic similarity between topics—overlapping bubbles suggest shared vocabulary, while isolated bubbles indicate distinct thematic content. This allows practitioners to quickly assess model quality and identify redundant topics.

A tunable ranking function that balances a term's frequency within a topic against its lift over the corpus-wide background frequency. The relevance metric is defined as relevance(term w | topic k) = λ * p(w | k) + (1 - λ) * p(w | k) / p(w). Adjusting the λ parameter (0 to 1) allows users to filter between purely topic-specific terms (λ near 0) and terms that are frequent in the topic but also common globally (λ near 1). This interactivity is critical for human interpretation of latent themes.

An evaluation metric measuring the semantic interpretability of a topic by quantifying the degree of co-occurrence between its top-ranked words in a reference corpus. High coherence scores correlate strongly with human judgments of topic quality. Common implementations include:

• C_V Coherence: Combines normalized pointwise mutual information with cosine similarity over word context vectors.
• UMass Coherence: Calculates pairwise word co-occurrence counts within the training corpus itself. pyLDAvis visualizations are often used to qualitatively validate topics that score highly on these quantitative metrics.

A metric assessing the uniqueness of topics by calculating the percentage of unique words across the top-N terms of all discovered topics. A model with high diversity avoids redundant topics that share the same high-probability words. In pyLDAvis, low diversity manifests as heavily overlapping bubbles on the intertopic distance map. The formula is typically: |unique(top-N words across all topics)| / (N * K), where K is the number of topics. This metric complements coherence to ensure a model is both interpretable and non-redundant.

The posterior probability vector θ_d representing the mixture of latent topics that constitute a specific document. In pyLDAvis, selecting a topic highlights the documents with the highest proportion of that topic. This bridges the gap between the global topic view and individual document analysis. Each document is a Dirichlet-distributed random variable over the K-dimensional topic simplex, with the alpha hyperparameter controlling the sparsity of this distribution—lower alpha values enforce documents to contain fewer topics.