Statistical Fidelity

Measures how accurately the synthetic data replicates the univariate statistical properties of each column in the original dataset.

• Compares histograms and probability density functions of individual variables
• Evaluated using Kolmogorov-Smirnov tests and Wasserstein distance
• Critical for preserving the statistical moments (mean, variance, skewness) of each feature
• Failure manifests as shifted distributions or incorrect value ranges in synthetic columns

Evaluates whether the synthetic data preserves the multivariate relationships between two or more columns simultaneously.

• Assessed through pairwise correlation matrices and mutual information scores
• Validates that conditional probabilities P(A|B) remain consistent with real data
• Essential for downstream tasks like regression where feature interactions drive predictions
• Poor joint fidelity leads to unrealistic combinations of otherwise valid individual values

Quantifies the preservation of linear and non-linear dependencies across the entire feature space.

• Pearson coefficients measure linear correlation preservation
• Distance correlation and HSIC (Hilbert-Schmidt Independence Criterion) capture non-linear relationships
• Critical for financial and healthcare datasets where feature interdependencies encode domain knowledge
• Correlation distortion can introduce spurious associations or mask genuine causal signals

Determines whether the same statistical conclusions would be drawn from the synthetic data as from the real data.

• Compares p-values and confidence intervals from identical tests run on both datasets
• Uses propensity score matching to verify that treatment effects are preserved
• The gold standard for utility: a data scientist should reach identical analytical conclusions
• Measured through the Train-Synthetic-Test-Real (TSTR) evaluation paradigm

Assesses whether the synthetic data captures the full support of the real distribution, including rare edge cases and minority classes.

• Mode collapse detection identifies when the generator ignores low-density regions
• Measures the proportion of real data points that fall within the synthetic data's convex hull
• Critical for fairness: underrepresented groups must not be erased during synthesis
• Evaluated using nearest-neighbor distance ratios and recall metrics in latent space

Uses a trained classifier to measure how statistically distinguishable the synthetic data is from the real data.

• A discriminator accuracy near 0.5 (random chance) indicates high fidelity
• More robust than single-metric comparisons; captures complex distributional differences
• Classifier two-sample tests (C2ST) provide formal statistical guarantees
• Complements distance-based metrics by detecting subtle multivariate discrepancies that simpler tests miss

The fundamental inverse relationship between the strength of privacy protections applied to synthetic data and the statistical fidelity retained. Increasing noise to guarantee differential privacy inevitably degrades the accuracy of marginal and joint distributions. The goal is to find the Pareto-optimal frontier where maximum utility is achieved for a given privacy budget (epsilon).

The gold-standard evaluation paradigm for measuring statistical fidelity. A machine learning model is trained exclusively on synthetic data and then tested on a held-out real dataset. If the TSTR performance closely matches the performance of a model trained on real data (TRTR), the synthetic data is considered to have high fidelity. This metric directly measures the generalization capacity of the synthetic generation process.

A degenerative failure mode where generative models trained recursively on synthetic data lose statistical fidelity over generations. The model progressively forgets the tails of the original distribution, leading to irreversible artifacts and a loss of diversity. This occurs because each generation of synthetic data is a lossy approximation, causing the model to converge to a narrow, mean-centric representation of reality.

The core statistical properties that fidelity metrics aim to preserve. Marginal distributions describe the probability of a single variable (e.g., age), while joint distributions capture the correlations between two or more variables (e.g., age vs. income). High-fidelity synthetic data must accurately replicate both to prevent downstream models from learning spurious or broken correlations.

A deep learning architecture where a generator and a discriminator compete in a zero-sum game. The generator creates synthetic data, and the discriminator attempts to distinguish it from real data. The generator's goal is to maximize the discriminator's error rate, ultimately producing synthetic data with high statistical fidelity that is indistinguishable from the original distribution.

The degradation of statistical fidelity over time as the real-world environment changes. A synthetic dataset is a frozen snapshot of a past distribution. As live data evolves, the frozen synthetic data becomes less representative, causing models trained on it to fail. Continuous monitoring of data drift is essential to trigger retraining or regeneration of synthetic assets.