Beta Hyperparameter

The beta hyperparameter is the concentration parameter for the symmetric Dirichlet prior on the topic-word distributions. A low beta value (e.g., 0.01) enforces sparsity, meaning each topic is characterized by a small set of highly probable, specific words. A high beta value (e.g., 1.0) results in dense distributions, where topics share many common, high-frequency terms, reducing distinctiveness.

Beta acts as a pseudo-count added to every word before observing data. In the Dirichlet distribution, a beta < 1 concentrates probability mass onto a few dimensions, favoring peaked distributions. When beta = 1, the prior is uniform. When beta > 1, the prior encourages smoother, more uniform distributions. During Gibbs sampling, the conditional probability of assigning a word to a topic is proportional to the count of that word in the topic plus beta, normalized appropriately.

Setting beta too low can produce overly specific, fragmented topics that capture noise rather than coherent themes. Setting beta too high creates generic, overlapping topics dominated by stop words and corpus-wide frequent terms. Optimal beta values typically range from 0.01 to 0.1 for most text corpora, balancing specificity with generalization. The choice interacts with the alpha hyperparameter and the number of topics (K).

While standard LDA uses a symmetric beta applied uniformly across all topics, advanced implementations allow asymmetric priors. An asymmetric beta vector can encode prior knowledge by assigning higher pseudo-counts to known domain-specific terms, guiding the model toward semantically meaningful themes. This technique is central to Seeded LDA, where expert knowledge is injected directly into the prior to anchor topics around predefined concepts.

Beta tuning directly affects topic coherence metrics like C_V coherence and PMI-based scores. Empirical studies show that lower beta values generally improve coherence by reducing the intrusion of generic terms into topic representations. However, excessively low beta can degrade coherence by fragmenting semantically related terms across multiple topics. Grid search over beta values, evaluated against topic coherence and topic diversity, is standard practice for model selection.

In the Gensim library, the beta hyperparameter is controlled via the eta parameter in LdaModel. The default is eta='auto', which learns an asymmetric prior. Setting eta=0.01 applies a low symmetric prior for sparse topics. Gensim also supports passing a full matrix to specify per-topic asymmetric priors. The update_every and passes parameters control how frequently the beta-derived distributions are updated during online variational inference.