Latent Diffusion Model

The foundational step where an autoencoder is trained to map high-dimensional pixel space to a lower-dimensional latent space. The encoder compresses the image by a factor of f (typically 4-8), discarding imperceptible high-frequency details. The decoder reconstructs the image from this compressed latent code. The diffusion process then operates entirely within this compact latent representation, drastically reducing the number of computations required per denoising step.

Unlike pixel-space diffusion models, LDMs perform the forward and reverse diffusion processes on the compressed latent vectors. The forward process gradually adds Gaussian noise to the latent representation of an image. The reverse process, parameterized by a U-Net backbone, learns to iteratively remove this noise. This latent-space operation is the core efficiency breakthrough, enabling training and inference on consumer-grade GPUs.

A mechanism that injects multimodal control signals directly into the U-Net's intermediate layers. The conditioning input (e.g., a text prompt, semantic map, or class label) is encoded by a domain-specific encoder (like CLIP for text). Cross-attention layers then attend to this conditioning embedding, allowing the model to steer the denoising process. This enables generation from text prompts, segmentation maps, or other structural inputs without architectural changes.

The cross-attention mechanism is modality-agnostic, making LDMs exceptionally adaptable for medical tasks. Conditioning can be a semantic label map of organs, a radiomics feature vector, or a prior scan from a different modality. This allows for controlled generation of synthetic pathology, image-to-image translation (e.g., MRI to CT), and super-resolution of medical scans by conditioning on a low-resolution input, all within a single architectural framework.

A sampling technique that balances sample fidelity and diversity without a separate classifier model. During training, the conditioning signal is randomly dropped out, teaching the model to perform both conditional and unconditional denoising. At inference, the final noise prediction is a weighted combination of the conditional and unconditional predictions. The guidance scale parameter controls this trade-off, with higher values enforcing stricter adherence to the conditioning input.

To prevent the latent space from having arbitrarily high variance, which would destabilize the diffusion process, the autoencoder is regularized. A common approach is a KL-divergence penalty applied to the latent distribution, pushing it towards a standard normal distribution. This creates a smooth, well-behaved latent manifold where linear interpolation between two latent codes produces semantically coherent, continuous morphological changes in the generated output.

The foundational generative framework that latent diffusion models build upon. A diffusion model learns to reverse a gradual noising process, starting from pure random noise and iteratively denoising it to produce a coherent output. In pixel-space diffusion, this process operates directly on high-dimensional image data, making it computationally expensive. Latent diffusion models accelerate this by performing the denoising steps in a compressed latent space learned by a variational autoencoder.

A critical component of the latent diffusion architecture. The VAE consists of an encoder that compresses an input image into a lower-dimensional latent representation and a decoder that reconstructs the image from this latent code. In a latent diffusion model, the diffusion process operates entirely within this compressed latent space, dramatically reducing computational cost. The decoder is then used only at the final step to translate the denoised latent representation back into pixel space.

The standard quantitative metric for evaluating the fidelity and diversity of generated images. FID measures the Wasserstein-2 distance between the feature distributions of real and generated images, as extracted by a pre-trained Inception network. A lower FID score indicates that the synthetic images are more similar to real ones. For medical imaging, FID is used to validate that synthetic scans preserve the statistical properties of real patient data.

A pixel-level annotation that assigns a class label to every region in an image, serving as a conditioning input for generative models. In medical image synthesis, a semantic label map might delineate organs such as the liver, kidneys, and spleen. A latent diffusion model can be conditioned on this map to generate a synthetic CT or MRI scan where each anatomical structure appears in its correct spatial location, enabling controlled generation of specific anatomies.

A technique for mapping an input image from one domain to a corresponding output image in another domain. In medical contexts, this often involves converting an MRI scan to a synthetic CT scan for radiotherapy planning. Latent diffusion models excel at this task by conditioning the denoising process on the source image, learning the complex, non-linear mapping between modalities while preserving anatomical geometry.