Bayesian Nonparametrics

The foundational Bayesian nonparametric prior for clustering. A Dirichlet Process (DP) defines a distribution over distributions, allowing an infinite number of potential mixture components. In practice, only a finite number of clusters are populated based on the observed data. The concentration parameter α controls the prior expectation of cluster count—a larger α favors more clusters. This is the standard model for discovering novel disease subtypes without imposing artificial boundaries on patient heterogeneity.

A generative metaphor for understanding how the Dirichlet Process assigns patients to clusters. Imagine a restaurant with an infinite number of tables:

• The first patient sits at the first table.
• Each subsequent patient sits at an existing table with probability proportional to the number already seated there, or starts a new table with probability proportional to α. This rich-get-richer property naturally favors a small number of large clusters while allowing outlier patients to form their own groups, mirroring clinical reality.

The workhorse algorithm for fitting Bayesian nonparametric mixture models. Gibbs sampling iteratively samples each patient's cluster assignment conditional on all other assignments and the model parameters. Key computational steps:

• Collapsed Gibbs sampling integrates out the mixture component parameters analytically, sampling only the cluster indicators.
• Neal's Algorithm 8 is the standard approach for non-conjugate models, using auxiliary parameters to approximate the infinite mixture.
• Convergence is assessed via trace plots of the number of occupied clusters and the log-likelihood.

Extends the DP to model grouped data where clusters are shared across groups but with different proportions. In patient stratification, this enables:

• Multi-center clinical trials: Each hospital site is a group, sharing the same disease subtypes but with varying prevalence.
• Cross-population genomics: Identifying latent genetic subpopulations that appear in different proportions across ethnic cohorts. The Hierarchical Dirichlet Process (HDP) places a base DP as a prior over group-specific DPs, ensuring that the same atoms (clusters) are reused across all groups.

A generalization of the Dirichlet Process with an additional discount parameter d (0 ≤ d < 1). The Pitman-Yor Process produces a power-law distribution over cluster sizes, generating more small, rare clusters than the DP. This is particularly relevant for:

• Rare disease variant discovery: Capturing small patient subgroups that a DP might absorb into larger clusters.
• Immune repertoire analysis: Modeling the heavy-tailed distribution of T-cell receptor clonotypes. When d = 0, the Pitman-Yor process reduces to the standard Dirichlet Process.

MCMC methods become computationally prohibitive for large-scale biobank data. Variational inference recasts posterior inference as an optimization problem:

• Truncated stick-breaking approximates the infinite DP with a finite number of components K, where K is set large enough to capture all occupied clusters.
• Evidence Lower Bound (ELBO) is maximized to find the optimal variational distribution.
• Stochastic variational inference enables training on mini-batches of patient data, scaling Bayesian nonparametrics to millions of records while maintaining uncertainty quantification.

The canonical Bayesian nonparametric prior for clustering. A Dirichlet Process (DP) defines a distribution over distributions, allowing a mixture model to have an unbounded number of components. The Chinese Restaurant Process (CRP) provides an intuitive construction: each new patient is assigned to an existing cluster with probability proportional to its size, or to a new cluster with probability proportional to a concentration parameter α. This directly enables the number of patient subgroups to grow with the data.

The finite parametric counterpart to the Dirichlet Process Mixture. A Gaussian Mixture Model assumes data is generated from a fixed number of K Gaussian distributions, each with its own mean and covariance. Unlike Bayesian nonparametrics, K must be specified a priori, typically selected via the Bayesian Information Criterion (BIC) or Akaike Information Criterion (AIC). GMMs provide soft cluster assignments, where each patient receives a posterior probability of belonging to each component.

Extends the Dirichlet Process to grouped data, enabling the discovery of shared patient subtypes across multiple hospitals or studies. An HDP posits a global base distribution drawn from a DP, with each group's data modeled by a separate DP sharing that global base. This ensures that clusters discovered in one cohort correspond to the same molecular subtypes in another, critical for multi-center clinical trials where patient populations must be aligned.

An explicit representation of the Dirichlet Process that reveals its infinite-dimensional nature. The stick-breaking process iteratively breaks off a random fraction of a unit-length stick to define mixture weights:

• Draw β_k ~ Beta(1, α) for k = 1, 2, ...
• Set weight π_k = β_k ∏_{i=1}^{k-1} (1 - β_i) This construction is essential for variational inference approximations to DP mixtures, where the infinite sum is truncated at a large but finite number for computational tractability.

The Bayesian nonparametric prior for latent feature models, the feature-based analog of the CRP. Instead of assigning each patient to a single cluster, the Indian Buffet Process allows each patient to possess a potentially infinite set of binary latent features. In patient stratification, this models comorbidities or overlapping disease mechanisms where a patient can simultaneously belong to multiple molecular subtypes, providing a richer representation than hard clustering.

The computational engine for fitting Bayesian nonparametric models. Markov Chain Monte Carlo (MCMC) methods, particularly Gibbs sampling with auxiliary variable schemes like Algorithm 8 by Neal, are the gold standard for posterior inference in Dirichlet Process mixtures. Each iteration reassigns patients to clusters based on the CRP predictive distribution, integrating over the infinite-dimensional parameter space. Split-merge proposals improve mixing by allowing clusters to fission or fuse in a single step.