Statistical Fidelity

Measures how well the synthetic data replicates the marginal distributions of each individual column. This is the most fundamental fidelity check.

• Continuous columns: Compares histograms, kernel density estimates, and quantile-quantile plots.
• Categorical columns: Evaluates frequency distributions and category proportions.
• Metrics: Kolmogorov-Smirnov statistic, Total Variation Distance, and Chi-squared tests.
• Failure mode: A synthetic dataset with perfect univariate fidelity but no multivariate structure is essentially column-wise shuffled real data.

Assesses whether the synthetic data preserves the inter-column relationships and joint distributions present in the real data. This is critical for downstream model utility.

• Linear correlations: Pearson and Spearman correlation matrices compared element-wise.
• Non-linear dependencies: Mutual information scores and distance correlation metrics.
• Contingency tables: Pairwise categorical associations tested with Cramér's V.
• Key metric: Pairwise Correlation Difference — the mean absolute error between real and synthetic correlation matrices.
• Warning: High univariate fidelity with poor multivariate fidelity produces data that looks real column-by-column but breaks any predictive model relying on feature interactions.

Validates that higher-order statistical properties of the real data are maintained in the synthetic output, beyond simple means and variances.

• Skewness: Measures distribution asymmetry — critical for financial and actuarial data.
• Kurtosis: Captures tail heaviness — essential for risk modeling and anomaly detection.
• Boundary adherence: Ensures synthetic values respect the min/max constraints and valid domain ranges of the real data.
• Missing value patterns: The proportion and structure of nulls should mirror the original data's missingness mechanism (MCAR, MAR, MNAR).
• Utility: Preserving tail behavior is vital when synthetic data is used for rare event modeling or extreme value analysis.

A rigorous approach to fidelity evaluation uses two-sample statistical tests to determine if real and synthetic data are drawn from the same distribution.

• Null hypothesis: The real and synthetic samples originate from the same underlying distribution.
• Continuous data: Two-sample Kolmogorov-Smirnov test and Mann-Whitney U test.
• Categorical data: Chi-squared test of independence and Fisher's exact test for small samples.
• Multivariate: Maximum Mean Discrepancy and energy distance tests.
• Interpretation: A high p-value suggests the synthetic data is statistically indistinguishable from real data along the tested dimension. Low p-values identify specific columns or pairs requiring model improvement.

For synthetic time series and sequential data, fidelity extends to preserving autocorrelation structures and temporal dynamics.

• Autocorrelation function: Measures correlation of a series with lagged versions of itself.
• Partial autocorrelation: Identifies the direct effect of specific lags.
• Seasonality preservation: Synthetic data must replicate periodic patterns at daily, weekly, or seasonal frequencies.
• Trend fidelity: Long-term upward or downward trajectories must be maintained.
• Spectral density: Frequency-domain analysis comparing power spectra of real and synthetic series.
• Application: Critical for synthetic financial tick data, IoT sensor streams, and healthcare vital sign monitoring where temporal ordering is semantically meaningful.

The ultimate measure of statistical fidelity is whether a model trained on synthetic data performs comparably to one trained on real data. This is the Train-Synthetic-Test-Real paradigm.

• TSTR workflow: Train a downstream model exclusively on synthetic data; evaluate on a held-out real test set.
• Baseline comparison: Compare against a model trained on real data and tested on the same real holdout.
• Utility gap: The difference in performance metrics between the two models.
• Task-specific: Fidelity requirements vary — a classification model may tolerate minor distributional drift that would break a regression model predicting exact dollar amounts.
• SDMetrics efficacy module: Automates this comparison across multiple model types and reports the utility score.

A deep learning architecture where two neural networks—a generator and a discriminator—compete adversarially. The generator learns to produce synthetic data that mimics the real data distribution, while the discriminator learns to distinguish real from fake. High statistical fidelity is achieved when the generator successfully fools the discriminator, indicating the synthetic distribution matches the real one. Mode collapse, where the generator produces limited variety, is a primary failure mode that directly degrades fidelity.

A metric for measuring the distance between two probability distributions, often used as a stable loss function in GANs. Unlike traditional GAN losses that suffer from vanishing gradients, the Wasserstein distance provides meaningful gradients even when the real and generated distributions do not overlap. This directly improves training stability and enables the generator to better capture the full diversity of the real data, resulting in higher statistical fidelity and reduced mode collapse.

An evaluation paradigm where a machine learning model is trained entirely on synthetic data and then tested on a held-out real dataset. The performance gap between TSTR and a model trained on real data quantifies the utility dimension of statistical fidelity. If a classifier trained on synthetic data achieves comparable accuracy to one trained on real data, the synthetic data has preserved the predictive signal and multivariate relationships necessary for downstream tasks.

The fundamental balancing act between the strength of a privacy guarantee and the statistical fidelity of the resulting synthetic data. Techniques like differential privacy inject calibrated noise to protect individual records, but this noise inevitably distorts distributions and correlations. Tight privacy budgets (low epsilon) provide strong protection but degrade fidelity; loose budgets preserve fidelity but increase re-identification risk. Navigating this trade-off is the central challenge of privacy-preserving data synthesis.

A failure condition in GAN training where the generator learns to produce only a limited variety of outputs, failing to capture the full diversity of the real data distribution. For example, a GAN trained on handwritten digits might only generate the digit '3' convincingly. Mode collapse catastrophically destroys statistical fidelity because the synthetic data no longer represents the true support of the real distribution, rendering it useless for training robust downstream models.