Document-Topic Distribution

The distribution is a point on a K-1 dimensional simplex, meaning every element is a probability between 0 and 1, and the entire vector sums exactly to 1. This constraint is enforced by the Dirichlet prior during inference.

• Mathematically: θ_d = (θ_{d1}, ..., θ_{dK}), where Σ θ_{dk} = 1
• This allows for mixed membership, where a document like a news article about a tech IPO can be 60% 'Finance' and 40% 'Technology'.

The concentration parameter α (alpha) of the Dirichlet prior directly controls the sparsity of the distribution. A low α value (e.g., 0.1) enforces sparse distributions, forcing documents to be dominated by very few topics. A high α value (e.g., 10.0) creates dense, uniform distributions where documents are a shallow mixture of many topics.

• Typical LDA implementations use a symmetric α < 1 to reflect the reality that most documents focus on a handful of themes.
• Asymmetric α priors can be learned via variational inference to better fit the data.

Unlike hard clustering (k-means), the distribution provides a soft assignment of documents to topics. This vector serves as a low-dimensional semantic representation of the document.

• A document's topic vector can be used as a feature input for downstream classifiers, such as logistic regression for sentiment analysis.
• Cosine similarity between two document-topic distributions is a robust measure of semantic similarity, often outperforming raw TF-IDF comparisons.

The distribution is not directly observed; it is the posterior probability inferred from the text. In collapsed Gibbs sampling, the distribution is estimated by counting the topic assignments (z) of each word in the document.

• θ_{dk} = (N_{dk} + α_k) / (N_d + Σ α_i), where N_{dk} is the count of words in document d assigned to topic k.
• This count-based estimation makes the distribution a direct reflection of the document's lexical content.

The distribution identifies the dominant themes of a document, but the topics themselves are unlabeled latent variables. Topic labeling is the process of assigning a human-readable name (e.g., 'Macroeconomic Policy') to Topic 4 by inspecting its top words.

• A document with a high probability for Topic 4 is then labeled accordingly.
• Tools like pyLDAvis visualize these distributions to help analysts validate the semantic coherence of the assignments.

In a Dynamic Topic Model (DTM), the document-topic distribution is not static. The priors evolve over time slices, allowing the thematic composition of a document to reflect the changing discourse.

• A document published in 2020 might have a high probability for a 'Pandemic Response' topic that did not exist in the 2018 model.
• This temporal evolution is modeled using a state-space model on the natural parameters of the distribution.