Variational Autoencoder (VAE)

Unlike a standard autoencoder that maps an input to a single fixed point in latent space, a VAE learns the parameters of a probability distribution—typically a multivariate Gaussian—defined by a mean vector and a variance vector. This forces the latent space to be continuous and smooth, meaning that small perturbations in the latent code produce semantically similar outputs. The encoder does not output a latent vector directly; it outputs the parameters (μ and σ) that define the conditional distribution q(z|x).

The core innovation that makes VAEs trainable via backpropagation. Sampling a latent vector z directly from the distribution defined by μ and σ is a stochastic operation that blocks gradient flow. The reparameterization trick expresses z as a deterministic function: z = μ + σ ⊙ ε, where ε is random noise sampled from a standard normal distribution N(0,1). This moves the stochasticity into the input noise, allowing gradients to flow through μ and σ during optimization.

VAEs are trained by maximizing the Evidence Lower Bound (ELBO), which decomposes into two competing terms:

• Reconstruction Loss: Measures how accurately the decoder can reconstruct the original input from the sampled latent vector z. Typically mean squared error for continuous data or cross-entropy for binary data.
• KL Divergence: Acts as a regularizer, forcing the learned latent distribution q(z|x) to be close to a prior distribution, usually a standard normal N(0,1). This prevents the model from memorizing inputs and ensures the latent space is well-structured for generation.

A well-trained VAE often learns disentangled representations, where individual dimensions of the latent space correspond to independent, interpretable generative factors. For example, in a model trained on single-cell transcriptomic data, one latent dimension might control cell-cycle stage while another captures immune activation status. The β-VAE variant introduces a hyperparameter β to weight the KL divergence term more heavily, explicitly encouraging this factorization of latent variables.

VAEs are uniquely suited for cross-modal data imputation in single-cell biology. A VAE trained jointly on paired transcriptomic and proteomic data (e.g., from CITE-seq) learns a shared latent space that captures the joint distribution of both modalities. When one modality is missing for a cell, the model can sample from the conditional distribution in the latent space and decode the missing modality, generating a statistically plausible imputation grounded in the learned biological manifold.

By conditioning the encoder and decoder on additional metadata—such as donor ID, experimental batch, or tissue type—a conditional VAE (cVAE) can generate data from a specific condition or remove unwanted technical variation. In multi-omics integration, this allows the model to learn a batch-corrected latent representation where biological variation is preserved but technical artifacts are regressed out, enabling harmonized analysis across multiple experimental cohorts.

The compressed, lower-dimensional bottleneck representation where input data is encoded into a probability distribution. In a VAE, this space is continuous and smooth, meaning small perturbations produce semantically meaningful variations. For single-cell data, the latent space captures cellular states and transitions, enabling interpolation between cell types and visualization of differentiation trajectories. The KL divergence regularizer ensures the latent distribution remains close to a standard Gaussian prior.

A critical mathematical technique that enables backpropagation through stochastic sampling. Instead of sampling directly from the learned distribution, the VAE samples from a fixed Gaussian noise source and shifts/scales it using the learned mean and variance. This makes the sampling operation differentiable, allowing gradient-based optimization of the encoder network. Without this trick, training VAEs would require high-variance score function estimators.

The objective function maximized during VAE training, consisting of two competing terms:

• Reconstruction loss: Measures how well the decoder reproduces the input from the latent sample
• KL divergence: Penalizes deviation of the learned latent distribution from the prior

Balancing these terms controls the rate-distortion trade-off—too much KL weight produces blurry reconstructions, while too little yields a fragmented latent space.

A VAE variant where both the encoder and decoder are conditioned on auxiliary information, such as cell type labels, batch identifiers, or experimental conditions. This allows the model to generate samples from a specific class or condition by providing the conditioning variable during decoding. In multi-omics integration, cVAEs can generate missing modalities conditioned on observed ones, enabling cross-modal prediction from transcriptomics to proteomics.

A modification of the standard VAE that introduces a hyperparameter β to weight the KL divergence term. When β > 1, the model is forced to learn more disentangled latent representations where individual latent dimensions correspond to independent generative factors. In single-cell analysis, β-VAEs can separate sources of variation such as cell cycle, differentiation state, and batch effects into distinct latent dimensions, improving interpretability of the learned representations.