Fréchet Inception Distance (FID)

FID does not compare images pixel-by-pixel. Instead, it compares the multivariate Gaussian distributions of feature embeddings extracted from the real and generated datasets by a pretrained Inception v3 network. The distance between these two distributions is calculated using the Fréchet distance, a 2-Wasserstein metric for Gaussians. A lower FID score indicates the generated distribution is closer to the real distribution, signifying better overall fidelity and diversity.

• Core Calculation: FID = ||μ_r - μ_g||² + Tr(Σ_r + Σ_g - 2(Σ_r Σ_g)^(1/2))
• μ and Σ: The mean vectors and covariance matrices of the real (r) and generated (g) embeddings.

The metric's performance is intrinsically linked to the pretrained Inception v3 network, which provides a high-level, semantic feature space. Images are passed through the network up to the final pooling layer (layer prior to classification), producing 2048-dimensional embeddings. This reliance means FID inherits the biases and capabilities of Inception v3—it is most effective for natural images (ImageNet-like) and may be less sensitive to distortions in domains far from its training data. The use of a fixed, pretrained network ensures the metric is model-agnostic and provides a consistent benchmark.

A key strength of FID is its sensitivity to both mode collapse and low sample quality. Unlike metrics that only measure fidelity (e.g., Precision), FID's use of covariance matrices captures the diversity (variance) of the generated set. A model suffering from mode collapse will produce a distribution with low variance, leading to a high FID. Similarly, a model generating diverse but low-fidelity images will have a mean feature vector far from the real data's mean, also resulting in a high score. This makes it a more holistic metric than Inception Score (IS).

The FID score is a distance metric, where a score of zero would indicate perfect alignment between the real and generated distributions (an ideal, unattainable scenario in practice). Therefore, lower FID scores are better. For reference, state-of-the-art generative models on datasets like CIFAR-10 achieve FID scores below 5, while scores on more complex datasets like ImageNet 256x256 may be in the single digits or low teens for top models. A very high FID (e.g., >100) indicates a severe distribution mismatch.

Calculating FID requires a substantial sample size from both real and generated datasets (typically thousands of images) to reliably estimate the mean and covariance matrices. The score is not deterministic for a fixed model; it varies based on the specific random samples drawn. Best practice involves reporting the mean and standard deviation of FID across multiple sampling runs. While computationally cheaper than human evaluation, it requires forward-passing all images through the Inception network, which can be time-consuming for large evaluation sets.

Dataset Size Dependency: FID can be unreliable with small sample sizes (<5k images). Feature Space Bias: It is optimized for ImageNet-like images; performance on medical, satellite, or artistic imagery is less validated. Insensitivity to Memorization: It measures distribution similarity, not novelty. A model that simply memorizes and replays the training set could achieve a near-perfect FID. Lack of Perceptual Alignment: It does not directly measure how well an image matches a specific textual prompt (unlike CLIP Score). It is a global dataset metric, not an image-text metric.

A precursor to FID that evaluates only the generated images. It uses a pretrained Inception-v3 network to measure two properties:

• Image Quality (Sharpness): The classifier should be highly confident in its prediction for a given image (low entropy per image).
• Diversity: The generated set should contain a wide variety of classes (high entropy across the full set of predictions).

Limitation: IS does not compare to a real dataset, so a model can achieve a high score by generating diverse, high-quality images of non-existent classes.

An alternative to FID designed to be unbiased for small sample sizes. While FID uses the Fréchet distance (which assumes a Gaussian distribution on the embeddings), KID uses the squared Maximum Mean Discrepancy (MMD) with a polynomial kernel.

Key Differences from FID:

• Provides an unbiased estimator, making it more reliable when evaluating with limited data.
• Does not assume the embedding distributions are Gaussian.
• Often reported as the mean KID over several splits of the data.

A metric for evaluating text-to-image generation by measuring the semantic alignment between a generated image and its conditioning text prompt. It computes the cosine similarity between the image and text embeddings from a pretrained CLIP (Contrastive Language-Image Pre-training) model.

Use Case: While FID measures general fidelity and diversity against a reference dataset, CLIP Score directly assesses how well the output matches a specific textual description. They are often reported together for text-conditional models.

A framework that decomposes generative model performance into two distinct, interpretable dimensions, addressing a key limitation of FID (which conflates the two).

• Precision (Fidelity): Measures what fraction of the generated data could be considered part of the real data manifold. High precision means generated samples are highly realistic.
• Recall (Diversity): Measures what fraction of the real data manifold is covered by the generated data. High recall means the model captures all modes of the real data.

FID can be low if either precision or recall is sacrificed, but this framework reveals the trade-off.

The ultimate, though costly, gold standard for assessing generative models. It involves human raters performing tasks like:

• Two-Alternative Forced Choice (2AFC): Choosing which of two images (one real, one fake) looks more realistic.
• Mean Opinion Score (MOS): Rating image quality on a Likert scale.
• Text-Image Alignment: Judging how well an image matches a given text prompt.

While metrics like FID provide scalable, automated assessment, they are proxies for human perception. The correlation between FID and human judgment is strong but not perfect.

A metric used to evaluate the smoothness and disentanglement of a generative model's latent space, commonly applied to StyleGAN. It measures the average perceptual change when interpolating between two random points in latent space.

How it works:

1. Sample two latent vectors, z1 and z2.
2. Interpolate between them and generate images along the path.
3. Compute the LPIPS (Learned Perceptual Image Patch Similarity) distance between consecutive images.

A lower PPL indicates a smoother latent space where interpolations correspond to gradual, semantic changes in the image, a property not directly captured by FID.