Denoising Diffusion Probabilistic Model (DDPM)

The forward process is a fixed Markov chain that progressively corrupts a real data sample $x_0$ into pure Gaussian noise $x_T$ over $T$ timesteps. At each step, a small amount of Gaussian noise is added according to a predefined variance schedule $\beta_t$.

• No learnable parameters: The process is a fixed, non-parametric destruction mechanism.
• Reparameterization trick: $x_t$ can be sampled directly from $x_0$ in closed form without iterating through all intermediate steps, enabling efficient training.
• Variance schedules: Typically use a linear, cosine, or sigmoid schedule to control the rate of information destruction.

The reverse process is a learned Markov chain that iteratively removes noise, starting from pure noise $x_T$ and reconstructing a coherent data sample $x_0$. A neural network, typically a U-Net, predicts the noise added at each step.

• Noise prediction: The model learns $\epsilon_\theta(x_t, t)$ to predict the noise component, rather than directly predicting $x_{t-1}$.
• Iterative refinement: Generation requires $T$ sequential denoising steps, trading speed for sample quality.
• Stochastic sampling: Each reverse step samples from a Gaussian distribution, introducing controlled randomness that enhances output diversity.

DDPMs employ a U-Net—a convolutional encoder-decoder with skip connections—as the core denoising network. This architecture preserves fine-grained spatial details essential for high-resolution medical image synthesis.

• Skip connections: Directly concatenate encoder feature maps to decoder layers, preventing information loss across the bottleneck.
• Time embedding: A sinusoidal or learned embedding of the timestep $t$ is injected into each residual block via FiLM (Feature-wise Linear Modulation) layers, conditioning the network on the noise level.
• Self-attention layers: Inserted at specific resolutions to capture long-range dependencies, critical for generating anatomically consistent structures in synthetic patient scans.

DDPMs are trained using a simplified variational bound that reduces to a mean squared error (MSE) between the predicted noise and the actual noise added during the forward process.

• Simplified loss: $L_{simple} = \mathbb{E}{t, x_0, \epsilon} \left[ | \epsilon - \epsilon\theta(x_t, t) |^2 \right]$, where $\epsilon$ is the true Gaussian noise.
• Stable convergence: Unlike GANs, this loss avoids adversarial min-max optimization, eliminating mode collapse and training instability.
• No discriminator required: The model learns the data distribution directly through denoising score matching, providing a smoother optimization landscape.

Classifier-free guidance steers generation toward specific conditions (e.g., a disease phenotype) without requiring a separate pre-trained classifier. During training, the conditioning label is randomly dropped, teaching the model to function both conditionally and unconditionally.

• Guidance scale: At inference, the model interpolates between conditional and unconditional noise predictions: $\tilde{\epsilon}\theta = (1 + w) \epsilon\theta(x_t, t, c) - w \epsilon_\theta(x_t, t)$.
• Fidelity-diversity trade-off: Higher guidance weights ($w > 0$) improve sample quality and condition adherence but reduce diversity.
• Medical relevance: Enables controlled generation of synthetic patient data with specific biomarker profiles or disease states for targeted clinical research.

DDPMs offer inherent privacy advantages for synthetic patient data generation. The iterative denoising process learns the data manifold rather than memorizing individual training samples, reducing re-identification risk.

• Differential privacy integration: Noise can be injected into gradients during training to provide formal $(\epsilon, \delta)$-DP guarantees on the generated distribution.
• Membership inference resistance: The stochastic generation process makes it harder for adversaries to determine if a specific patient record was in the training set compared to GAN-generated data.
• Clinical plausibility: Generated samples remain constrained to the learned manifold, producing medically coherent records that respect physiological correlations.

The Markov chain that progressively destroys the data structure by adding Gaussian noise over T timesteps. Starting from a real patient record x₀, each step samples xₜ = √(1-βₜ)xₜ₋₁ + √βₜε, where βₜ follows a predefined variance schedule. By timestep T, the data distribution converges to an isotropic Gaussian N(0,I), completely obscuring all clinical features. This process requires no learning—it is a fixed, analytical corruption mechanism that defines the target distribution the model must learn to reverse.

The learned parameterized Markov chain that iteratively removes noise to reconstruct a coherent sample. A neural network ε_θ(xₜ, t) predicts the noise component at each timestep, enabling the transition from xₜ to xₜ₋₁. For synthetic patient generation, this network must learn to recover clinically plausible combinations of lab values, diagnoses, and demographics. The reverse process is trained by minimizing the variational bound on negative log-likelihood, which simplifies to a mean-squared error between predicted and actual noise: L_simple = ||ε - ε_θ(xₜ, t)||².

The variance schedule {βₜ}ₜ₌₁ᵀ controls how quickly information is destroyed during forward diffusion. Common parameterizations include:

• Linear schedule: βₜ scales linearly from β₁ to β_T
• Cosine schedule: βₜ follows a cosine function for smoother information decay
• Learned schedule: βₜ optimized jointly with the denoising network For tabular clinical data, the schedule must be tuned to preserve discrete feature structures (ICD-10 codes, categorical labs) long enough for the reverse process to learn meaningful transitions, while ensuring sufficient noise at T for proper generative sampling.

DDPMs are equivalent to score-based models trained with multiple noise scales. The denoising objective implicitly estimates the Stein score function ∇ₓ log p(xₜ)—the gradient of the log-density pointing toward higher probability regions. This connection, formalized through stochastic differential equations (SDEs), unifies DDPMs with Noise Conditional Score Networks (NCSN). For synthetic EHR generation, this perspective enables continuous-time sampling via SDE solvers like Euler-Maruyama or ancestral sampling, offering flexibility in trading sample quality against generation speed.

A technique for conditioning DDPMs on auxiliary variables (e.g., disease state, age group) without training a separate classifier. During training, the conditioning signal c is randomly dropped with probability p_uncond, forcing the model to learn both conditional ε_θ(xₜ, t, c) and unconditional ε_θ(xₜ, t) denoising. At sampling time, the noise prediction is extrapolated: ε̂ = ε_uncond + w(ε_cond - ε_uncond), where w > 1 amplifies conditioning strength. This enables controlled generation of synthetic patient cohorts with specific clinical characteristics while maintaining high sample fidelity.

Applying DDPMs to structured clinical data requires addressing non-continuous features:

• Mixed data types: Continuous labs, binary diagnoses, categorical ICD codes
• One-hot encoding with Gaussian diffusion corrupts mutual exclusivity
• Multinomial diffusion uses categorical transition matrices instead of Gaussian noise
• TabDDPM combines Gaussian diffusion for numerical features with multinomial diffusion for categorical features
• Missing data mechanisms must be modeled explicitly, as clinical datasets contain structured missingness (not-at-random patterns tied to care workflows)