Synthetic Data Utility

The gold-standard evaluation paradigm where a model is trained exclusively on synthetic data and tested on held-out real data. If the synthetic data captures the underlying distribution, performance should approach that of a model trained on real data.

• Workflow: Generate synthetic dataset → Train model on synthetic → Evaluate on real test set
• Comparison baseline: Train-on-Real-Test-on-Real (TRTR) performance
• Utility gap: The difference between TSTR and TRTR metrics quantifies information loss
• Clinical relevance: A small utility gap indicates the synthetic data preserves diagnostic signal

A battery of tests comparing marginal distributions, joint distributions, and correlation structures between real and synthetic datasets to detect divergence.

• Kolmogorov-Smirnov test: Measures maximum distance between cumulative distribution functions for continuous features
• Chi-squared test: Evaluates categorical variable distribution alignment
• Pairwise correlation difference: Computes the Frobenius norm between real and synthetic correlation matrices
• Mutual information preservation: Quantifies how well non-linear relationships between features are maintained
• Propensity score divergence: Trains a classifier to distinguish real from synthetic; poor discriminability indicates high fidelity

A decision framework plotting privacy guarantees against model utility to identify the Pareto-optimal operating point for synthetic data release.

• X-axis: Privacy parameter ε (epsilon) from differential privacy—lower values indicate stronger privacy
• Y-axis: Downstream task performance (AUC, F1, accuracy)
• Knee point: The ε value where further privacy relaxation yields diminishing utility returns
• Clinical governance: Enables explicit, auditable decisions about acceptable privacy risk for a given diagnostic performance threshold

Leverages the adversarial discriminator from GAN architectures or a post-hoc classifier to measure how distinguishable synthetic samples are from real ones.

• Discriminator accuracy near 0.5: Indicates synthetic data is indistinguishable from real—the ideal outcome
• Feature-level attribution: Identifies which clinical variables the discriminator exploits to tell real from fake
• Domain shift detection: A discriminator that achieves high accuracy signals systematic distributional gaps requiring generator retraining
• Federated application: Discriminator scores can be computed locally without sharing patient data

Evaluates synthetic data utility by measuring performance on real clinical prediction tasks rather than abstract statistical similarity.

• Classification: AUC and F1-score for diagnostic models trained on synthetic data
• Regression: Mean absolute error for continuous outcome prediction (e.g., length of stay)
• Survival analysis: Concordance index preservation for time-to-event models
• Clustering: Adjusted Rand Index comparing patient subgroups discovered in synthetic vs. real data
• Domain-specific benchmarks: MIMIC-III or eICU-derived holdout sets for standardized comparison

Measures whether synthetic data adequately represents rare disease phenotypes, demographic subgroups, and edge cases rather than collapsing to majority modes.

• Minority class recall: Proportion of rare real samples with synthetic nearest neighbors within a distance threshold
• Wasserstein distance: Earth mover's distance between real and synthetic distributions—penalizes mode collapse
• Nearest neighbor adherence: Ratio of synthetic samples whose closest real neighbor belongs to the same class
• Subgroup parity: Statistical parity difference across protected attributes (race, age, sex) to detect amplification of real-world biases

The gold standard evaluation paradigm for synthetic data utility. A model is trained exclusively on artificially generated data and then tested on a held-out set of real-world records. If the TSTR performance closely matches the performance of a model trained on real data (Train-Real-Test-Real), the synthetic dataset is deemed to have high statistical fidelity. This metric directly quantifies the downstream predictive value of the generation process.

A critical threat to synthetic data utility where the statistical properties of the generated data diverge from the real-world distribution. This can manifest as covariate shift, label shift, or concept drift. In clinical settings, a shift in synthetic patient demographics or lab value correlations renders the data useless for training robust diagnostic models. Continuous monitoring via statistical divergence metrics like Wasserstein distance is required.

A dual-network architecture central to high-utility synthetic data generation. The generator creates artificial samples from random noise, while the discriminator attempts to distinguish them from real data. Through adversarial training, the generator learns to produce samples that capture complex, high-dimensional correlations. Specialized variants like CTGAN handle mixed tabular data, while medGAN is optimized for discrete electronic health records.

A generative model that learns a compressed, continuous latent representation of input data. Unlike GANs, VAEs explicitly model the underlying probability distribution, allowing for smooth interpolation and controlled generation. The model encodes real data into a latent space and then decodes sampled points back into synthetic records. This probabilistic approach is particularly effective for generating high-dimensional, structured clinical data with measurable likelihoods.

A mathematical framework that provides a provable privacy budget (ε) for synthetic data. By injecting calibrated Gaussian noise into the training process or the generated outputs, differential privacy ensures that the presence or absence of a single patient record cannot be reliably inferred. This formal guarantee is essential for regulatory compliance, but introduces a direct privacy-utility trade-off: tighter privacy budgets (lower ε) degrade the statistical fidelity of the synthetic data.

An adversarial evaluation technique used to stress-test the privacy of synthetic data. An attacker trains a shadow model to determine if a specific real record was part of the original training dataset used to train the generator. High susceptibility to this attack indicates overfitting in the generative model, where the synthetic data has memorized and leaked identifiable features of real patients, severely compromising both privacy and generalization utility.