Differential Privacy

The privacy budget (ε) is a non-negative parameter that quantifies the maximum information leakage allowed. A lower epsilon enforces a stronger privacy guarantee by making the output of a query or generative model nearly indistinguishable whether or not a specific individual's genome was included in the training set.

• ε = 0: Perfect privacy; the output is completely independent of any single record.
• ε < 1: Strong privacy; typical for high-sensitivity genomic applications.
• ε > 10: Weak privacy; minimal protection against membership inference.

Selecting epsilon involves a direct trade-off: a smaller budget requires adding more noise, which can degrade the utility of the synthetic genomic data for downstream tasks like variant calling.

A randomized algorithm M satisfies (ε, δ)-differential privacy if, for any two datasets D and D' differing by a single individual's genomic record, and for any set of possible outputs S:

Pr[M(D) ∈ S] ≤ e^ε × Pr[M(D') ∈ S] + δ

• ε (epsilon): The privacy loss parameter, bounding the multiplicative difference in output probabilities.
• δ (delta): A relaxation parameter allowing a small probability (typically cryptographically small, e.g., 10^-6) that the pure ε guarantee is violated. This is crucial for Gaussian noise mechanisms commonly used in deep learning.

This definition provides a provable guarantee against an adversary with arbitrary auxiliary information, making it the gold standard for genomic data protection.

The amount of noise required to achieve a specific epsilon depends on the sensitivity of the computation—the maximum impact a single record can have on the output.

• Global Sensitivity (GS): The maximum difference in the output of a function f when applied to any two neighboring datasets. For a counting query, GS = 1.
• Local Sensitivity: The sensitivity specific to a given dataset, which can be exploited to add less noise but requires careful smoothing to avoid leaking information.

Noise Mechanisms:

• Laplace Mechanism: Adds noise drawn from a Laplace distribution scaled by GS/ε. Ideal for pure ε-differential privacy.
• Gaussian Mechanism: Adds noise scaled by (GS * √(2*ln(1.25/δ)))/ε. Used for (ε, δ)-differential privacy, essential for the stochastic gradient descent steps in training deep generative models.

DP-SGD is the core algorithm for training deep learning models, including genomic GANs and VAEs, with differential privacy guarantees. It modifies standard training by:

1. Per-Example Gradient Clipping: The gradient from each individual genomic sequence is clipped to have an L2 norm no greater than a threshold C. This bounds the sensitivity of each training step.
2. Noise Addition: Calibrated Gaussian noise is added to the summed, clipped gradients before the model weights are updated.
3. Privacy Accounting: A moments accountant tracks the cumulative privacy loss (ε) over all training iterations, allowing the training to stop once the pre-defined privacy budget is exhausted.

This process ensures the final model parameters, and thus any synthetic data generated from it, carry a formal privacy guarantee.

A single query or training step consumes a fraction of the privacy budget. Composition theorems dictate how the total epsilon accumulates when multiple differentially private operations are performed on the same dataset.

• Basic Composition: The epsilons simply add up. Performing k mechanisms each with ε_i results in a total budget of Σ ε_i.
• Advanced Composition: Provides a tighter, sub-linear bound on the total privacy loss, especially for a large number of mechanisms. For k folds of (ε, δ)-DP, the total cost is approximately O(√(k * ln(1/δ')) * ε + k * ε^2).

This is critical for training generative models over thousands of iterations, as the moments accountant in DP-SGD leverages advanced composition to provide a much more accurate and less pessimistic total epsilon than simple summation.

A fundamental property of differential privacy is its resilience to post-processing. Any computation applied to the output of an (ε, δ)-differentially private mechanism does not degrade the privacy guarantee.

• Formal Statement: If M is (ε, δ)-DP, then for any arbitrary function f, f(M(D)) is also (ε, δ)-DP.
• Genomic Implication: Once a generative model is trained with DP-SGD, any synthetic genomic data sampled from it—whether raw FASTQ reads, a synthetic VCF, or derived statistics like allele frequencies—maintains the original privacy guarantee. An adversary cannot "un-noise" the data through clever post-processing.

A quantifiable parameter that controls the trade-off between data utility and privacy strength. Lower epsilon values (e.g., 0.1) provide stronger privacy guarantees by injecting more noise, while higher values (e.g., 10) preserve more analytical accuracy.

• Epsilon = 0: Perfect privacy, zero utility
• Epsilon < 1: Strong privacy for genomic cohorts
• Epsilon > 10: Weak guarantees, high re-identification risk

The budget is consumed with each query, requiring careful allocation across model training iterations.

A privacy audit technique that tests whether an adversary can determine if a specific individual's genomic record was included in the training dataset. The attacker trains a shadow model to recognize differences in model confidence between members and non-members.

• True Positive Rate: Measures attack success
• AUC-ROC > 0.5: Indicates privacy leakage
• Defense: Differential privacy reduces attack AUC toward random guessing

Critical for validating that synthetic generators do not memorize training samples.

The mathematical operation that adds calibrated random noise drawn from a Gaussian distribution to model gradients during training. The noise scale is proportional to the sensitivity of the query divided by epsilon.

• Sensitivity: Maximum impact a single record can have on the output
• Standard Deviation: σ = (sensitivity × √(2·ln(1.25/δ))) / ε
• Delta (δ): Probability of catastrophic privacy failure

This mechanism provides (ε, δ)-differential privacy, the standard for deep learning with genomic data.

An evaluation paradigm that measures synthetic data utility by training a predictive model on synthetic genomic data and testing it on real held-out data. Performance parity with models trained on real data indicates high-fidelity generation.

• Variant calling accuracy: Compare F1 scores
• Phenotype prediction: Compare AUC-ROC curves
• Population structure: Compare PCA projections

TSTR provides a practical, task-specific metric that complements privacy guarantees with utility validation.

A technique that trains a binary classifier to distinguish between real and synthetic genomic sequences. A generator achieves privacy-utility balance when the discriminator performs no better than random chance (AUC ≈ 0.5).

• High discriminability: Synthetic data is unrealistic or memorized
• Low discriminability: Generator captures true data distribution
• Feature importance: Identifies which genomic regions leak real patterns

Used alongside differential privacy to detect unintended memorization of rare variants.

A metric for evaluating synthetic data quality by comparing the distribution of generated sequences to real sequences in a learned feature space. Analogous to Frechet Inception Distance (FID) in computer vision.

• Lower FGD: Better distributional match
• Feature extractor: Pre-trained genomic foundation model
• Captures: k-mer frequencies, GC content, motif distributions

Provides a summary statistic for global fidelity without requiring downstream task evaluation.