Differentially Private Synthetic Data

The generator satisfies ε-differential privacy (or its relaxed variant, (ε, δ)-DP), providing a mathematical proof that the presence or absence of any single record in the original dataset cannot be reliably inferred from the synthetic output. This guarantee is immune to post-processing—any analysis performed on the synthetic data cannot weaken the privacy protection. The privacy loss parameter epsilon (ε) quantifies the upper bound on information leakage, with lower values indicating stronger privacy.

The synthetic dataset retains the joint probability distribution of the original data, including marginal distributions, correlations, and higher-order interactions. Key aspects include:

• Marginal accuracy: Univariate distributions match the source data
• Correlation structure: Pairwise and multivariate relationships are preserved
• Query answering: Aggregate queries (COUNT, SUM, AVG) on the synthetic data approximate those on the real data within known error bounds

The fidelity is constrained by the privacy budget—stricter privacy (lower ε) requires injecting more noise, which degrades statistical accuracy.

Generating synthetic data consumes a finite privacy budget (ε) from the original dataset. This budget is a quantifiable resource that limits total privacy loss across all analyses. Once the budget is exhausted, no further queries can be answered with the same privacy guarantee. The composition theorem governs how the budget accumulates: generating multiple synthetic tables or repeatedly querying the original data sequentially consumes the budget additively. Parallel composition allows disjoint data partitions to be queried independently without additional budget cost.

Every record in the synthetic dataset is an artificial construct generated from the learned probability distribution, not a direct copy of any real individual. This provides plausible deniability: the presence of a synthetic record resembling a real person does not constitute evidence that the real person was in the training data. This property is distinct from k-anonymity or pseudonymization—even if an attacker possesses auxiliary information, the differential privacy guarantee ensures they cannot confirm membership in the source dataset.

The method of noise injection determines the synthetic data's properties. Common mechanisms include:

• Laplace Mechanism: Adds Laplace noise calibrated to L1 sensitivity; provides pure ε-DP for numerical marginals
• Gaussian Mechanism: Adds Gaussian noise calibrated to L2 sensitivity; provides (ε, δ)-DP, enabling tighter composition under iterative generation
• Exponential Mechanism: Selects synthetic records probabilistically based on a utility score, suitable for categorical data
• DP-SGD: Trains generative neural networks (e.g., GANs, VAEs) with per-example gradient clipping and noise addition, tracked by a moments accountant

A fundamental tension exists between statistical utility and privacy protection. As ε decreases (stronger privacy), the noise magnitude increases, reducing the accuracy of the synthetic data for downstream tasks. This trade-off is quantified by:

• Error bounds on aggregate queries as a function of ε and dataset size
• Confidence intervals that widen with stricter privacy guarantees
• Sample size requirements: Larger original datasets enable better utility at the same privacy level, as the signal-to-noise ratio improves

Practitioners must select ε based on the acceptable utility loss for their specific use case.

The privacy budget (epsilon, ε) is a finite, quantifiable resource that controls the total allowable privacy loss during synthetic data generation. Each query or training epoch consumes a portion of this budget.

• Sequential Composition: The total epsilon is the sum of the epsilons used in each step of the generation pipeline.
• Parallel Composition: When steps operate on disjoint data partitions, the total epsilon is the maximum epsilon used in any single partition.
• Budget Depletion: Once the budget is exhausted, no further queries can be answered without violating the formal guarantee.

A typical synthetic data generation run might allocate an ε of 1.0 to 8.0, balancing utility against the strong protection offered by ε < 1.

The fundamental tension in synthetic data generation is between the statistical fidelity of the output and the strength of the privacy guarantee (ε).

• High Fidelity (Large ε): Synthetic data closely mirrors the joint distributions, correlations, and outliers of the real data but offers weaker privacy protection.
• High Privacy (Small ε): Stronger guarantees against membership inference, but the synthetic data may smooth out rare categories, weaken correlations, or fail to capture complex interactions.
• Utility Metrics: Evaluated using propensity score matching, discriminant analysis, and comparing the accuracy of machine learning models trained on synthetic vs. real data for a specific downstream task.

A key security property of differentially private synthetic data is its inherent resistance to membership inference attacks. An adversary analyzing the synthetic records cannot confidently determine if a specific individual was in the original training dataset.

• The formal DP guarantee bounds the adversary's true positive rate and false positive rate by a function of ε.
• This is empirically validated by training a shadow classifier to distinguish real training records from non-members, and verifying that its accuracy is close to random guessing.
• This protection holds even if the adversary has access to the entire synthetic dataset and auxiliary information from external sources.