Frechet Genomic Distance

FGD computes the Fréchet distance (also known as the Wasserstein-2 distance) between two multivariate Gaussian distributions fitted to real and synthetic genomic feature embeddings. It captures both the mean shift (differences in average feature values) and the covariance shift (differences in feature correlations). A lower FGD score indicates that the synthetic data more faithfully reproduces the statistical structure of the real genomic data.

FGD does not operate on raw nucleotide sequences directly. Instead, a pre-trained feature extractor—typically a DNA language model or a convolutional neural network—maps sequences to a dense, high-dimensional embedding space. This space is designed to capture biologically meaningful features, such as motif presence, splice sites, and chromatin accessibility. The quality of the FGD evaluation is directly dependent on the representational power of this feature extractor.

FGD is a direct adaptation of the Fréchet Inception Distance (FID), the gold-standard metric for evaluating synthetic images. In FID, embeddings are extracted from the Inception v3 network. In FGD, the Inception network is replaced with a genomic model. This analogy provides a rigorous, well-understood framework for generative model comparison, avoiding the pitfalls of qualitative or heuristic-only evaluations.

The calculation of FGD is sensitive to the number of samples used. With too few sequences, the estimated mean and covariance matrices become unreliable, leading to a biased and high-variance FGD score. Best practices recommend a minimum of 10,000 sequences for a stable estimate. The metric assumes the embeddings follow a Gaussian distribution, an approximation that holds better with larger sample sizes.

A key strength of FGD is its demonstrated correlation with the Train-Synthetic-Test-Real (TSTR) paradigm. A lower FGD score generally predicts better performance when a model trained on synthetic data is evaluated on real data. This makes FGD a reliable proxy for data utility without needing to run expensive downstream tasks like variant calling or gene expression prediction for every model checkpoint.

FGD is highly effective at detecting mode collapse, a common failure in GAN training where the generator produces only a few highly similar sequence types. A collapsed generator will have a feature covariance matrix with very low variance, which FGD penalizes heavily. A high FGD score, even with a reasonable mean, is a strong quantitative signal that the synthetic data lacks the full diversity of the real genomic distribution.