Structural Topic Model (STM)

In STM, topic prevalence refers to the proportion of a document devoted to a topic. Unlike standard LDA, STM allows this proportion to vary systematically with document-level metadata.

• Mechanism: A document's topic proportions are drawn from a logistic-normal distribution whose mean is a linear function of the covariates.
• Example: In a corpus of political blogs, the prevalence of a 'foreign policy' topic might be modeled as a function of the author's political affiliation (party) and the publication date.
• Inference: This allows researchers to test hypotheses like 'Does the prevalence of Topic X increase over time?' using standard regression coefficients.

STM uniquely allows the word distribution within a topic to be influenced by covariates. This captures how the language used to discuss a theme changes based on context.

• Mechanism: The topic-word distribution is formed by a deviation from a baseline word frequency, where the deviation interacts with covariates.
• Example: A topic about 'energy' might use words like 'fracking' and 'offshore' when the covariate party is 'Republican', but 'solar' and 'renewable' when party is 'Democrat'.
• Distinction: This is a key differentiator from models like LDA or CTM, which assume a fixed vocabulary for each topic across all documents.

STM uses a spectral initialization method based on the method of moments to provide a deterministic and fast starting point for variational inference.

• Stability: This approach resolves the reproducibility issues common in topic models that rely on random initialization, ensuring consistent results across multiple runs.
• Process: It leverages the empirical word co-occurrence matrix to find a robust initial topic-word distribution before iterative optimization begins.
• Benefit: Spectral initialization significantly reduces convergence time and avoids local optima, leading to higher quality and more interpretable topics.

STM provides robust estimation of document-topic proportions (theta) by conditioning on the full document structure and metadata.

• Uncertainty Quantification: The variational EM algorithm approximates the posterior distribution, providing measures of uncertainty for topic proportions.
• Marginal Effects: The estimateEffect function allows users to simulate counterfactuals, such as 'How does topic prevalence change when moving a binary covariate from 0 to 1, holding all else constant?'
• Visualization: Results are easily plotted with confidence intervals to show the estimated relationship between a covariate and topic prevalence.

STM introduces exclusivity as a diagnostic metric to complement semantic coherence, helping to select the optimal number of topics (K).

• Exclusivity: Measures the degree to which a topic's top words are unique to that topic and do not appear as top words in others. High exclusivity prevents topic redundancy.
• Semantic Coherence: Quantifies how frequently a topic's top words co-occur in the corpus, a proxy for human interpretability.
• Trade-off: STM models often visualize the trade-off between these two metrics to guide model selection, seeking a K that maximizes both interpretability and distinctiveness.

Like the Correlated Topic Model (CTM), STM explicitly models the correlation between topics using a logistic-normal prior on the document-topic proportions.

• Covariance Matrix: The model estimates a topic covariance matrix, revealing which themes tend to co-occur in documents.
• Network Visualization: This matrix can be visualized as a topic correlation network, where edges represent positive correlations.
• Practical Use: Understanding topic correlation helps in identifying semantic 'neighborhoods' and prevents the misinterpretation of topics as completely independent entities.