Differential Privacy

The parameter epsilon (ε) quantifies the privacy loss. A smaller ε means stronger privacy but more noise. It represents the maximum divergence between outputs on neighboring datasets.

• ε = 0: Perfect privacy, but zero utility.
• ε = 0.1–1: Strong privacy, suitable for high-sensitivity data.
• ε = 1–10: Weaker privacy, higher accuracy.

Choosing ε is a critical trade-off between utility and risk.

The guarantee is defined relative to neighboring datasets—two datasets that differ by exactly one individual's data. Differential privacy ensures that the output of an analysis is nearly indistinguishable whether or not any single person's record is included.

• Add/Remove: One dataset has the record, the other does not.
• Substitution: One record is swapped for another.

This formalizes the idea that an attacker cannot infer your presence in the dataset.

Privacy is achieved by adding calibrated random noise to query results. The noise magnitude is proportional to the sensitivity of the query—how much a single record can change the output.

• Laplace Mechanism: Adds noise drawn from a Laplace distribution, ideal for numeric queries.
• Gaussian Mechanism: Uses Gaussian noise, often preferred for compositions.
• Exponential Mechanism: For non-numeric outputs, selects results probabilistically based on a utility score.

When multiple differentially private analyses are run on the same data, the total privacy loss accumulates. Composition theorems quantify this degradation.

• Basic Composition: The epsilons simply add up (ε_total = ε₁ + ε₂).
• Advanced Composition: Provides a tighter bound, showing that privacy degrades sub-linearly with the square root of the number of queries.

This allows engineers to track and manage a global privacy budget across an entire system.

A crucial property: any computation performed on the output of a differentially private mechanism cannot weaken the privacy guarantee. An attacker cannot reverse-engineer the noise or extract more information through arbitrary post-processing.

• The output can be used for visualization, machine learning, or statistical testing.
• No additional privacy loss occurs after the noisy result is released.

This makes differential privacy composable with existing data pipelines safely.

While standard differential privacy protects a single individual, group privacy extends the guarantee to groups of size k. The privacy loss scales linearly: a mechanism that is ε-differentially private for one individual is kε-differentially private for a group of size k.

• Protecting a family of 4 with ε=0.1 requires a budget of ε=0.4.
• This highlights the inherent difficulty of hiding correlated or clustered data.

It is a direct consequence of the neighboring dataset definition.

Differential privacy is applied to gradient updates before they leave a device, ensuring raw data never leaves the hospital or phone. The local model of DP guarantees privacy against a curious server.

• Mechanism: Gaussian noise is added to model weights during federated averaging.
• Use Case: Training a diagnostic model across multiple hospitals without centralizing protected health information (PHI).
• Trade-off: A higher privacy budget (ε) increases utility but weakens the theoretical guarantee.

Generative models trained with differentially private stochastic gradient descent (DP-SGD) produce synthetic datasets that retain the statistical properties of the real data without memorizing individual records.

• Architecture: DP-SGD clips per-example gradients and adds calibrated noise during backpropagation.
• Benefit: Enables unrestricted sharing of high-fidelity synthetic medical records for biomarker discovery.
• Validation: Utility is measured by comparing the performance of models trained on synthetic data versus real data.

The Laplace mechanism and Gaussian mechanism allow data custodians to answer statistical queries with provable privacy. This is the foundational application for releasing sensitive census or clinical trial data.

• Sensitivity: The maximum impact a single record can have on the query result determines the noise scale.
• Composition: The privacy loss accumulates predictably across multiple queries, governed by the composition theorem.
• Example: Publishing the mean and variance of a biomarker across a patient cohort without revealing individual measurements.

Differential privacy enables secure sharing of allele frequencies and GWAS summary statistics while protecting against membership inference attacks that could reveal an individual's participation in a study.

• Method: The χ² statistic or p-values are perturbed before public release.
• Significance: Prevents attackers from determining if a specific person's DNA was in a case group for a sensitive disease.
• Tooling: Libraries like OpenDP and Tumult Analytics provide composable, verifiable DP building blocks for bioinformatics pipelines.

A formal privacy budget (ε, δ) provides auditable, quantitative evidence for regulators like the FDA or EMA that patient privacy has been mathematically preserved during AI model development.

• Proof: Unlike heuristic de-identification, DP provides a worst-case guarantee that holds against any adversary with any auxiliary information.
• PCCP Alignment: A predetermined change control plan can specify the fixed privacy parameters under which a diagnostic model is allowed to be retrained.
• Metric: The privacy loss parameter ε (epsilon) is the primary knob controlling the privacy-utility trade-off.

When releasing a trained machine learning model, DP training ensures the model's parameters do not inadvertently encode and leak sensitive training data through model inversion or membership inference attacks.

• Technique: DP-SGD is integrated directly into the training loop of deep neural networks.
• Outcome: The final model can be distributed to third-party researchers or deployed in less secure environments without exposing the original training subjects.
• Consideration: The added noise acts as a regularizer, which can sometimes improve generalization on clean test sets.

The fundamental parameter controlling the privacy-utility trade-off. Epsilon quantifies the maximum distance between query outputs on neighboring datasets. A lower epsilon (e.g., 0.1) provides stronger privacy guarantees but adds more noise, reducing accuracy. A higher epsilon (e.g., 10) yields more accurate results but weaker privacy. Composition theorems track cumulative epsilon loss across multiple queries, requiring careful budget management to avoid total privacy depletion.

Two distinct trust models for noise injection. In Global DP (or Central DP), a trusted curator collects raw data and adds noise to the output of queries. In Local DP (LDP), individuals perturb their own data before sending it to an untrusted aggregator. LDP provides stronger user-level protection but requires significantly more noise to achieve the same utility, as the randomization is distributed across all participants rather than applied once centrally.

The core algorithms for achieving differential privacy. The Laplace mechanism adds noise drawn from a Laplace distribution scaled by the L1 sensitivity of the query function, satisfying pure ε-differential privacy. The Gaussian mechanism adds Gaussian noise scaled by L2 sensitivity, satisfying the relaxed (ε, δ)-differential privacy definition. The choice between them depends on the query's sensitivity metric and the desired privacy guarantee strength.

The primary threat model that differential privacy defends against. In this attack, an adversary determines whether a specific individual's record was included in a model's training dataset. Attackers exploit differences in model confidence, loss values, or output distributions between members and non-members. Differential privacy provides a provable upper bound on an attacker's ability to distinguish membership, directly limiting the success rate of such inference attempts.

The standard algorithm for training deep learning models with differential privacy guarantees. DP-SGD modifies standard SGD in two critical steps: gradient clipping bounds the influence of any single training example by capping per-example gradient norms, and Gaussian noise is added to the aggregated clipped gradients. The privacy accountant tracks cumulative epsilon and delta values across training epochs using moments accountant or Rényi DP composition.

A complementary cryptographic primitive that enables computation directly on encrypted data without decryption. Unlike differential privacy, which protects outputs, homomorphic encryption protects data in use. Fully Homomorphic Encryption (FHE) supports arbitrary computations on ciphertexts. In practice, HE is often combined with differential privacy in hybrid privacy-preserving systems where HE secures computation and DP protects the final released results from inference attacks.