Fréchet Inception Distance (FID)

FID calculates the Wasserstein-2 distance (Fr'echet distance) between two multivariate Gaussian distributions fitted to feature representations extracted from a pre-trained Inception v3 network. Specifically, it computes the difference in means and covariances of the real and generated image embeddings. This captures not just per-image quality but the diversity and coverage of the entire generated distribution relative to the real dataset.

Unlike simpler metrics such as Inception Score (IS), FID is highly sensitive to mode collapse—a failure mode where a generator produces only a limited variety of samples. If a GAN generates only one realistic-looking CT scan repeatedly, the covariance of the generated distribution collapses, causing the FID to spike. It also penalizes intra-class mode dropping, where a model fails to generate examples of rare pathologies, making it critical for evaluating medical image diversity.

FID relies on the pool3 layer of an Inception v3 network pre-trained on ImageNet to embed images into a 2048-dimensional feature space. This is a known limitation in medical imaging, as features learned from natural photographs may not capture clinically relevant details like Hounsfield Unit distributions or subtle lesion textures. Domain-specific adaptations, such as using a network pre-trained on medical data, are an active area of research to improve metric relevance.

FID is a biased estimator; its calculated value depends heavily on the number of samples used. Comparing FID scores across papers is unreliable unless the same number of images is used for evaluation. Best practice recommends a minimum of 10,000 samples to reduce estimator bias. For smaller medical datasets, techniques like bootstrapping or using the unbiased KID (Kernel Inception Distance) are preferred alternatives.

In medical imaging, FID is the primary quantitative metric for challenges like SynthRAD2023, which benchmarks synthetic CT generation from MRI. A lower FID between generated synthetic CTs and real CTs indicates better soft-tissue contrast fidelity and geometric accuracy. It is used alongside perceptual metrics like SSIM and task-specific validation (e.g., dosimetric accuracy in radiotherapy planning) to ensure synthetic images are clinically usable.

FID measures distributional similarity but does not guarantee diagnostic interchangeability. A model may achieve a low FID while hallucinating subtle, clinically significant structures or removing pathology. Therefore, FID must be supplemented with radiomics feature stability analysis and Turing tests conducted by board-certified radiologists. A low FID is a necessary but insufficient condition for clinical deployment.

An alternative metric for evaluating generative models that measures both image quality and diversity using the Inception v3 network. Unlike FID, IS does not compare against real image statistics, making it less reliable for detecting mode collapse.

• Calculation: Uses the conditional label distribution p(y|x) and marginal distribution p(y).
• Limitation: IS can be fooled by models that generate one highly realistic image per class, whereas FID penalizes lack of diversity across the entire distribution.

A perceptual metric that quantifies the degradation of structural information between a generated image and a reference image. SSIM focuses on luminance, contrast, and structure, making it complementary to FID for paired image-to-image translation tasks.

• Paired Evaluation: Requires a ground truth reference, unlike FID which compares distributions.
• Medical Use Case: Essential for evaluating synthetic CT generation from MRI where anatomical correspondence must be preserved.

A failure condition in GAN training where the generator produces a limited variety of outputs, failing to capture the full diversity of the target data distribution. FID is highly sensitive to mode collapse because it compares the full covariance structure of real and generated feature distributions.

• Detection: A sudden plateau or increase in FID during training often signals mode collapse.
• Mitigation: Techniques like minibatch discrimination, unrolled GANs, and Wasserstein loss help prevent this failure mode.