Differential Privacy for Tabular Data

The epsilon (ε) parameter is the core privacy budget that quantifies the strength of the guarantee. A smaller ε provides stronger privacy by limiting how much the algorithm's output can change due to any one record.

• Interpretation: ε controls the privacy loss. Common values range from 0.1 (very strong privacy) to 10 (weaker privacy, more utility).
• Composition: The privacy budget is consumed each time data is accessed. Sequential analyses on the same dataset must track cumulative ε to prevent privacy depletion.
• Example: An algorithm with ε = 1.0 ensures that the likelihood of any output is at most e^1 (≈2.718) times more likely with or without any individual's data.

These are the primary noise injection mechanisms used to achieve differential privacy for numerical queries on tabular data.

• Laplace Mechanism: Adds noise drawn from a Laplace distribution, scaled by the query's sensitivity (Δf) divided by ε. Ideal for counting queries and histograms.
• Gaussian Mechanism: Adds noise from a Gaussian (normal) distribution. Often used when multiple queries are asked (composition) and requires a slightly relaxed (ε, δ)-differential privacy definition.
• Key Concept: Sensitivity (Δf): The maximum possible change in a query's result when a single row is added or removed. This determines how much noise is required.

The Exponential Mechanism is used to select a discrete output (like a category or a decision) in a differentially private way, when adding noise directly to the answer doesn't make sense.

• How it works: It assigns a probability to each possible output, where probabilities are exponentially weighted by a utility function. Higher-utility outputs are exponentially more likely to be chosen.
• Use Case: Selecting the most frequent category in a column, choosing a binning scheme for histograms, or picking a split point in a decision tree during private model training.
• Formula: The probability of output r is proportional to exp(ε * u(D, r) / (2Δu)), where u is the utility function and Δu is its sensitivity.

Delta (δ) is a secondary parameter in the relaxed (ε, δ)-differential privacy definition. It represents a small probability that the strict ε-privacy guarantee could be broken.

• Interpretation: A δ value of 1e-5 means there is a 1 in 100,000 chance of a privacy failure beyond the ε bound. For pure DP, δ = 0.
• Necessity: Some mechanisms, like the Gaussian mechanism, cannot achieve pure DP (δ=0) and require this small, quantifiable slack.
• Security Analogy: If ε is the "amount" of privacy loss, δ is the "probability" of a catastrophic failure. Best practice sets δ significantly smaller than 1/n (where n is dataset size).

This distinction defines where the noise is added in the data pipeline.

• Global Differential Privacy (Central Model): A trusted curator holds the raw dataset. Noise is added to the outputs (queries or synthetic data) before release. This model provides higher data utility.
• Local Differential Privacy (LDP): Each individual's device adds noise to their raw data before sending it to an untrusted curator. This is stronger (no trusted central party) but requires more noise per individual, reducing aggregate utility.
• Tabular Data Context: Synthetic tabular data generation typically uses the global model. The generative model (e.g., a private GAN or VAE) is trained as the curator, and the final synthetic dataset is the noisy output.

Calculating the sensitivity of functions over tabular data is the critical first step in applying the Laplace or Exponential mechanisms.

• Counting Queries: Sensitivity Δf = 1. Adding/removing one person changes a count by at most 1.
• Sum Queries: Sensitivity depends on data bounds. If age is clipped to [0, 120], the sensitivity for SUM(age) is 120.
• Mean/Histogram Queries: More complex. Often decomposed into private sum and private count queries.
• For Synthesis: Model training algorithms (like gradient descent) have sensitivity defined by how much a single training point can change the model's parameters or the generated output distribution.

Banks and fintech companies use it to develop and test fraud detection algorithms using realistic transaction data that contains no real customer records. Key implementations include:

• Generating synthetic transaction logs that preserve complex spending patterns and temporal correlations.
• Creating synthetic datasets for stress testing anti-money laundering (AML) models.
• Enabling secure benchmarking and model development across different financial entities without sharing sensitive customer data. This protects both consumer privacy and proprietary business logic.

Organizations use differentially private synthetic data to accelerate and de-risk the model development lifecycle within internal teams. This application focuses on:

• Providing data scientists with safe, high-utility sandbox datasets that mirror production data for prototyping and experimentation.
• Reducing the compliance overhead and access controls required for using live, sensitive data in development environments.
• Enabling the creation of benchmark datasets for model evaluation that are free from privacy constraints, allowing for more transparent and collaborative model reviews.

Facilitates secure data partnerships between companies, or between a company and its vendors/academics, where the raw data cannot be directly shared. This is achieved by:

• Generating a privacy-safe synthetic replica of the core dataset to share with partners.
• Allowing partners to perform feasibility studies, build preliminary models, and design analytics pipelines without ever accessing the original records.
• Establishing a clear, auditable privacy budget (epsilon, δ) as part of the data sharing agreement, providing a contractual guarantee of privacy protection.

Enables rigorous testing of algorithms for discriminatory bias without repeatedly querying the sensitive original data, which could itself constitute a privacy risk. Practitioners use it for:

• Generating multiple synthetic datasets to perform stress tests and counterfactual analysis on models.
• Investigating how models perform across different synthetic demographic subgroups.
• This allows for proactive identification of unfair outcomes while maintaining the confidentiality of the individuals in the training data, aligning with emerging algorithmic auditing regulations.

Epsilon (ε) is the core parameter defining the strength of a differential privacy guarantee. It quantifies the maximum allowable difference in the probability of any output when a single individual's record is present or absent from the dataset. A smaller ε provides stronger privacy but typically reduces data utility.

• Interpretation: An algorithm with ε = 1.0 means the output's probability distribution changes by at most a factor of e (~2.718) due to any one record.
• Composition: The privacy budget is consumed with each query; sequential analyses must track cumulative ε to ensure the total guarantee is not violated.
• Typical Values: In practice, ε values often range from 0.1 (very strong privacy) to 10 (weaker privacy, higher utility), depending on the application's sensitivity.

These are the fundamental noise-adding mechanisms used to achieve differential privacy for numeric queries. They calibrate random noise to the sensitivity of a function—the maximum change in its output from altering one record.

• Laplace Mechanism: Adds noise drawn from a Laplace distribution. It is the canonical mechanism for achieving pure (ε)-differential privacy on real-valued queries.
• Gaussian Mechanism: Adds noise drawn from a Gaussian (normal) distribution. It is used for (ε, δ)-differential privacy, a slightly relaxed guarantee that allows a small probability δ of privacy failure, often enabling less noise for the same utility.
• Application to Tabular Synthesis: In models like PrivBayes, these mechanisms inject noise into the parameters of a learned statistical model (e.g., conditional probability tables in a Bayesian network) before synthetic data is sampled.

Synthetic data utility measures how well the generated dataset preserves the statistical properties and machine learning task performance of the original, real data it is intended to replace or augment. It is the counterbalance to privacy.

• Statistical Fidelity: Measures like Wasserstein distance, Kolmogorov-Smirnov tests, and correlation matrix similarity assess if marginal and joint distributions are preserved.
• Downstream Task Performance: The Train on Synthetic, Test on Real (TSTR) protocol is the gold-standard utility test. A model trained on synthetic data is evaluated on a held-out set of real data; high accuracy indicates high utility.
• Privacy-Utility Trade-off: Increasing the privacy guarantee (lower ε) typically requires adding more noise, which can degrade utility. The goal of algorithm design is to optimize this trade-off.

k-Anonymity is a syntactic privacy model that predates differential privacy. A dataset satisfies k-anonymity if each record is indistinguishable from at least k-1 other records with respect to a set of quasi-identifier attributes (e.g., ZIP code, age, gender).

• Implementation: Achieved via generalization (e.g., changing age '33' to '30-39') and suppression (removing rare values).
• Key Weakness: It protects against re-identification but not against attribute disclosure. If all records in a k-anonymous group share a sensitive attribute (e.g., a specific disease), that attribute is revealed. It also lacks a rigorous, composable privacy guarantee against auxiliary information attacks.
• Contrast with DP: Differential privacy provides a provable, mathematical guarantee against any attack, regardless of an adversary's background knowledge, and is resilient to composition, making it a stronger standard for tabular data release.

Federated synthetic data generation is a distributed approach to privacy-preserving synthesis where raw data never leaves its local silo (e.g., a hospital, a mobile device). A global synthetic data generator is trained by aggregating model updates from these distributed nodes.

• Process: 1) Local nodes train a generative model (e.g., a GAN or VAE) on their private data. 2) Only the model parameters or gradients—not the raw data—are sent to a central server. 3) The server aggregates these updates to improve a global generative model. 4) The global model can then be used to synthesize a representative dataset.
• Synergy with DP: Differential privacy can be applied during the federated learning process by clipping and noising the local updates before aggregation, providing a double layer of privacy protection.
• Use Case: Ideal for healthcare, finance, and IoT, where data is inherently distributed and highly sensitive.