Forward Process

The forward process is defined as a Markov chain, meaning the state at timestep t depends only on the state at the previous timestep t-1. This property simplifies the mathematical formulation, allowing the noisy sample at any timestep to be expressed in closed form relative to the original data. The transition from (x_{t-1}) to (x_t) is governed by a Gaussian distribution: (q(x_t | x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t} x_{t-1}, \beta_t I)).

The progression of noise addition is controlled by a predetermined noise schedule, defined by variances (\beta_1, \beta_2, ..., \beta_T) where (0 < \beta_t < 1). This schedule is not learned; it is a hyperparameter. Common schedules include linear, cosine, and sigmoid functions. The schedule dictates the signal-to-noise ratio (SNR) decay, determining how quickly the original data structure is destroyed. A well-designed schedule is critical for stable model training and efficient reverse process sampling.

A key mathematical convenience is that due to the properties of Gaussian noise, we can sample the noisy data (x_t) at any arbitrary timestep t directly from the original data (x_0), without iterating through the chain. This is expressed as:

(q(x_t | x_0) = \mathcal{N}(x_t; \sqrt{\bar{\alpha}_t} x_0, (1-\bar{\alpha}_t)I))

where (\alpha_t = 1 - \beta_t) and (\bar{\alpha}t = \prod{s=1}^{t} \alpha_s). This property is essential for efficient training, as it allows for random, non-sequential sampling of timesteps during optimization.

The forward process can be parameterized in two primary ways:

• Variance-Preserving (VP): The total variance of the noisy sample (x_t) is constrained (e.g., to 1). As (t) increases, the mean shrinks toward zero while the variance remains constant. This is used in DDPM.
• Variance-Exploding (VE): The variance of the noise increases without bound over time, while the mean of the signal goes to zero. This formulation connects more directly to score-based generative modeling. The choice affects the scaling of the model's predictions and the formulation of the training loss.

As the number of timesteps (T) approaches infinity (or in practice, for a sufficiently large (T)), the final distribution (q(x_T | x_0)) converges to a standard isotropic Gaussian distribution (\mathcal{N}(0, I)), regardless of the starting sample (x_0). This property guarantees that the reverse process has a simple, known prior distribution to start from: pure noise. The model learns to transform this noise back into a complex data sample through the learned reverse process.

Crucially, the forward process contains no learnable parameters. It is a fixed, analytical procedure for data destruction. All learning occurs in the reverse process, where a neural network (e.g., a U-Net) is trained to approximate the conditional distributions (p_\theta(x_{t-1} | x_t)) needed to denoise the data. This separation simplifies the training objective, which typically becomes a series of weighted mean-squared error losses between the true added noise and the network's noise prediction.

The learned generative trajectory that iteratively denoises a sample of pure Gaussian noise, reversing the fixed forward process to synthesize new data from the target distribution. This is the core of image generation in models like DDPM and Stable Diffusion.

• Mechanism: A neural network (e.g., a U-Net) is trained to predict and remove the noise added at each step of the forward process.
• Connection: It is the parameterized inverse of the forward process, transforming noise back into structured data.

A predefined function that controls the variance (beta) of the Gaussian noise added at each timestep t during the forward diffusion process. It dictates the rate and progression from clean data to pure noise.

• Key Types: Linear, cosine, and sigmoid schedules are common, affecting training stability and final sample quality.
• Role: Determines the signal-to-noise ratio over time, defining the difficulty of the denoising task the model must learn to reverse.

A foundational class of generative model that formalizes the forward and reverse processes. The model is trained to reverse a fixed forward Markov chain that gradually adds Gaussian noise.

• Training Objective: The model, typically a U-Net, is trained as a noise prediction network, learning to estimate the noise component ε added to a noisy input x_t.
• Significance: DDPM established the practical, high-quality image generation paradigm that underpins modern diffusion models.

An alternative training framework for generative models where a neural network (score network) learns to estimate the gradient of the log data density, known as the score function.

• Score Function: Defined as ∇_x log p(x), it points in the direction of higher data density.
• Equivalence: Under certain parameterizations, training a noise prediction network in a DDPM is equivalent to learning a (scaled) score function, linking diffusion models to score-based generative models.

A continuous-time framework that generalizes the discrete forward and reverse processes of diffusion models. The data corruption is described by a differential equation with a stochastic (Wiener) noise term.

• Forward SDE: Models the gradual addition of noise as a continuous process.
• Reverse SDE: Provides a continuous-time equation for generation, which can be solved with numerical solvers. This perspective unifies many diffusion model variants.

A specific parameterization of the forward diffusion process where the total variance of the noisy sample x_t is constrained to remain constant (typically 1) across all timesteps.

• Mathematical Constraint: Ensures q(x_t | x_0) has unit variance as t → T, converging to a standard Gaussian N(0, I).
• Prevalence: This is the most common formulation, used in seminal works like DDPM and many subsequent image generation models.