Flow Matching

Unlike traditional diffusion models that rely on simulating a forward noising process, Flow Matching directly regresses a neural network to a target vector field. This target field defines a probability path from a simple prior (e.g., Gaussian) to the complex data distribution. The training objective is a simple mean-squared error loss between the network's predicted vector field and the target, eliminating the need for iterative sampling during training and often leading to faster convergence.

Flow Matching provides a practical method for training Continuous Normalizing Flows (CNFs). CNFs define a transformation of data via an ordinary differential equation (ODE) governed by a time-dependent vector field. By learning this vector field, the model can deterministically map samples between the prior and data distributions. This results in invertible transformations and allows for exact likelihood computation, providing advantages over stochastic diffusion processes.

The framework is agnostic to the choice of probability path. While a straight linear path is common, the theory supports any continuously differentiable path connecting the distributions.

• Optimal Transport Paths: Paths can be designed based on principles from optimal transport, such as the rectified flow, which encourages straight trajectories for faster sampling.
• Conditional Paths: The path can be conditioned on auxiliary information (e.g., class labels, text embeddings), enabling controlled generation.
• This flexibility allows practitioners to design paths that optimize for specific properties like sampling speed or likelihood.

Flow Matching provides a unifying perspective that encompasses diffusion models. Key connections include:

• Diffusion as a Flow: The forward/reverse processes of a Variance-Preserving diffusion model can be reinterpreted as a specific instance of a probability flow ODE.
• Score Matching Equivalence: Under certain conditions (e.g., Gaussian conditional paths), the Flow Matching objective is equivalent to score matching, the foundation of score-based generative models.
• This reveals diffusion models as a special case of learning a vector field that generates a marginal-preserving transformation.

Once trained, generating a sample is performed by solving an initial value problem (IVP). Starting from a prior sample z_0, the final data sample x_1 is obtained by integrating the learned vector field: x_1 = z_0 + ∫_0^1 v_θ(x_t, t) dt

• ODE Solvers: This integration can be performed with fast, adaptive numerical ODE solvers (e.g., Dormand-Prince, Euler).
• Few-Step Generation: Unlike ancestral sampling in DDPMs, high-quality samples can often be generated in 10-20 function evaluations.
• Determinism: For a fixed prior sample and solver, the generation process is deterministic, aiding reproducibility.

A major research direction is trajectory straightening. The learned vector field often defines curved paths; straightening them enables single-step or few-step generation.

• Rectified Flow: An iterative procedure that refines a flow model by straightening its trajectories, converging to a straight line connecting distributions.
• Consistency Models: Can be derived as a distillation of a probability flow ODE, learning to map any point on a trajectory directly to its origin, enabling one-step generation.
• This principle bridges Flow Matching with ultra-fast latent consistency model techniques.

A class of generative models that define data generation as the solution to an ordinary differential equation (ODE). A neural network parameterizes a time-dependent vector field that transforms samples from a simple prior distribution (like Gaussian noise) into samples from the complex data distribution. Flow Matching provides a simulation-free method to train these CNFs by regressing to a target probability path.

A foundational training objective for learning the score function—the gradient of the log data density. Models like Denoising Score Matching train a neural network to estimate this gradient from perturbed data samples. Flow Matching is closely related; while score matching learns gradients of static distributions, flow matching learns a time-dependent vector field that defines a dynamic path between distributions.

In the Stochastic Differential Equation (SDE) view of diffusion models, there exists a deterministic ordinary differential equation whose trajectories share the same marginal probability densities. This ODE defines a continuous-time generative process. Flow Matching directly trains a model to match the vector field that defines this ODE, providing an alternative to learning the score function via diffusion SDEs.

A mathematical framework that describes the forward (noising) and reverse (denoising) processes in diffusion models as continuous-time stochastic processes. The forward SDE adds noise, while the reverse SDE requires the score function to denoise. Flow Matching offers a deterministic (ODE-based) alternative to this stochastic framework for training generative models.

A field of mathematics concerned with finding the most efficient way to morph one probability distribution into another. Flow Matching is inspired by optimal transport theory, where the target vector field often corresponds to a displacement that minimizes a transport cost (like the Wasserstein distance). The probability path in Flow Matching can be designed to follow optimal transport trajectories.

A class of generative models trained to be self-consistent along the trajectories of a probability flow ODE. They can map any point on an ODE trajectory directly back to its origin in a single step. While both leverage ODEs, Consistency Models enforce a specific consistency property, whereas Flow Matching is a general framework for regressing to any vector field defining a probability path.