Variational Autoencoder (VAE)

Unlike deterministic autoencoders that map inputs to fixed vectors, VAEs encode inputs into probability distributions—typically parameterized by a mean (μ) and variance (σ²). This forces the model to learn a smooth, continuous latent manifold where nearby points decode to semantically similar outputs. Sampling from this distribution during generation introduces controlled stochasticity, enabling the production of diverse, novel samples rather than memorized reconstructions.

VAEs are trained by maximizing the Evidence Lower Bound (ELBO), a tractable surrogate for the intractable marginal likelihood. The ELBO decomposes into two competing terms:

• Reconstruction Loss: Maximizes the log-likelihood of the input given the latent sample, ensuring faithful decoding.
• KL Divergence: Regularizes the learned latent distribution towards a prior (typically a standard Gaussian), enforcing smoothness and preventing overfitting. This principled objective provides a theoretically grounded training signal absent in standard autoencoders.

The reparameterization trick is the critical innovation that makes VAEs trainable via backpropagation. Instead of sampling z ~ N(μ, σ²) directly—which blocks gradient flow—the model samples ε ~ N(0, I) and computes z = μ + σ ⊙ ε. This reframes the stochastic node as a deterministic function of the parameters and an independent noise source, allowing gradients to flow through μ and σ unimpeded.

When trained with a weighted KL divergence term (as in β-VAE), the latent dimensions tend to encode independent, interpretable factors of variation. For synthetic patient data, this means one dimension might control age-related features while another independently modulates a specific biomarker level. Disentanglement enables controlled generation—modifying a single latent factor produces a predictable, isolated change in the output, critical for simulating specific clinical scenarios.

VAEs offer a principled approach to anomaly detection using reconstruction probability, which is more robust than raw reconstruction error. By sampling multiple latent vectors for a given input and computing the average probability of reconstructing the original data, the model quantifies how well a sample fits the learned distribution. Out-of-distribution samples—such as rare disease presentations—exhibit low reconstruction probabilities, making VAEs effective for identifying novel phenotypes in clinical datasets.

Conditional VAEs (CVAEs) extend the architecture by feeding auxiliary information—such as class labels, demographic attributes, or clinical metadata—into both the encoder and decoder. This conditions the entire generative process, enabling controlled synthesis: generate a synthetic patient record with a specific ICD-10 code, age range, and gender. In biomarker identification, CVAEs can simulate patient cohorts with targeted disease subtypes while preserving realistic comorbidity patterns.