Dirichlet Distribution

The PDF is defined over the (K-1)-simplex, where a vector α = (α₁, ..., αK) of concentration parameters controls the shape.

• Support: All vectors p where pᵢ ≥ 0 and Σ pᵢ = 1.
• Formula: f(p|α) = (1/B(α)) ∏ pᵢ^(αᵢ - 1), where B(α) is the multivariate Beta function.
• Interpretation: The density is high where p aligns with the normalized α vector.

The sum of the parameter vector, α₀ = Σ αᵢ, is the precision or concentration of the distribution.

• α₀ large: The distribution is sharply peaked around the mean, indicating high confidence.
• α₀ small: The distribution is flat and dispersed, indicating high uncertainty.
• In evidential deep learning, a neural network directly outputs α to parameterize a Dirichlet over class probabilities.

The expected value and dispersion are analytically tractable, making the Dirichlet computationally efficient for Bayesian updates.

• Mean: E[pᵢ] = αᵢ / α₀. The predicted probability for class i is the normalized evidence.
• Variance: Var[pᵢ] = (αᵢ(α₀ - αᵢ)) / (α₀²(α₀ + 1)).
• Covariance: Cov[pᵢ, pⱼ] = -(αᵢ αⱼ) / (α₀²(α₀ + 1)) for i ≠ j, enforcing the negative correlation inherent to the simplex.

The Dirichlet is the conjugate prior to the multinomial distribution, enabling closed-form Bayesian updates.

• Prior: Dir(α_prior).
• Likelihood: Multinomial with counts c.
• Posterior: Dir(α_prior + c).
• This additive property allows a model to accumulate evidence from observations without iterative optimization, forming the mathematical basis for evidential deep learning.

The differential entropy of a Dirichlet distribution quantifies the total predictive uncertainty.

• Formula: H[Dir(α)] = log B(α) + (α₀ - K)ψ(α₀) - Σ (αᵢ - 1)ψ(αᵢ), where ψ is the digamma function.
• Maximum Entropy: Occurs when all αᵢ = 1 (uniform distribution over the simplex), representing total ignorance.
• Minimum Entropy: Approaches negative infinity as α₀ → ∞ for a single peaked class, representing absolute certainty.

If p ~ Dir(α₁, ..., αK), then any merged partition of p also follows a Dirichlet distribution.

• Example: Merging classes 1 and 2 yields (p₁+p₂, p₃, ..., pK) ~ Dir(α₁+α₂, α₃, ..., αK).
• This marginalization property is critical for hierarchical classification, allowing uncertainty to be coherently propagated up a class taxonomy without re-sampling.

A neural network architecture that places a Dirichlet distribution directly over class probabilities. Instead of outputting a single softmax point estimate, the model predicts the parameters of a Dirichlet, allowing a single deterministic forward pass to output both a prediction and its associated epistemic uncertainty. This avoids the computational cost of Monte Carlo sampling or ensembles.

A prior distribution is conjugate to a likelihood function if the resulting posterior distribution belongs to the same probability distribution family. The Dirichlet distribution is the conjugate prior for the categorical and multinomial distributions. This mathematical property enables closed-form Bayesian updates: observing new count data simply updates the Dirichlet's concentration parameters, making it computationally tractable.

The probability simplex (or standard (K-1)-simplex) is the geometric space in which the Dirichlet distribution lives. It is the set of all vectors of length K whose components are non-negative and sum to exactly 1. For a 3-class problem, this is a triangle; for 2 classes, it is a line segment. Every point on the simplex represents a valid categorical probability distribution.

The vector of positive real numbers (α₁, α₂, ..., α_K) that governs the shape of a Dirichlet distribution. The sum α₀ = Σαᵢ is the precision or total evidence. Higher α₀ concentrates density tightly around the mean, indicating low uncertainty. Uniformly low αᵢ values produce a distribution spread across the simplex, indicating high epistemic uncertainty. In evidential deep learning, these are the direct outputs of the final neural network layer after a softplus activation.

The Dirichlet distribution enables explicit decomposition of predictive uncertainty. Epistemic uncertainty (model ignorance) is high when total evidence α₀ is low—the distribution is flat across the simplex. Aleatoric uncertainty (data noise) is captured by the entropy of the expected categorical probabilities. A peaked Dirichlet with high α₀ but a mean near-uniform distribution indicates irreducible data noise, not model ignorance.

A class of neural networks where weights are treated as probability distributions rather than point estimates. While BNNs place priors over weights, evidential methods place a Dirichlet prior directly over the output likelihood. This distinction makes evidential approaches computationally cheaper: they require no weight-space sampling and produce uncertainty estimates in a single forward pass, avoiding the heavy cost of variational inference or Hamiltonian Monte Carlo.