How to Implement Differential Privacy in Sensitive AI Training Data

Epsilon (ε) quantifies the strength of the privacy guarantee. A smaller ε means stronger privacy but typically reduces model utility. It's the maximum allowable difference in the probability of any output when a single individual's data is added or removed from the dataset. In practice, you set a total budget (e.g., ε = 8.0) for the entire training run and carefully spend it across training steps.

• Key Rule: The privacy budget is consumptive; each query or training step uses a portion.
• Implementation: Libraries like TensorFlow Privacy track this budget automatically during stochastic gradient descent.

Delta (δ) is a small probability that the privacy guarantee (defined by ε) could fail. It accounts for extremely low-probability events. For most practical purposes, δ should be set significantly smaller than 1/(size of dataset).

• Common Setting: δ = 1e-5 or 1/(dataset size).
• Interpretation: A guarantee of (ε, δ)-Differential Privacy means the probability of a privacy violation exceeding ε is bounded by δ.
• Trade-off: A non-zero δ (e.g., 1e-5) often allows for better utility than pure (ε, 0)-DP, which is much more restrictive.

Sensitivity is the maximum amount a single data point can change the result of a query or computation. It's the cornerstone for calculating how much noise to add. You must compute sensitivity before applying differential privacy.

• L1 Sensitivity (Δf): The maximum absolute change in a numeric query's output.
• L2 Sensitivity: Used when adding Gaussian noise.
• Example: If you query the average age in a database where ages range 0-120, the L1 sensitivity is 120. For a sum query, sensitivity is the maximum possible value of a single entry. Clipping data (e.g., gradient norms) is a primary method to control sensitivity.

The Laplace Mechanism provides (ε, 0)-Differential Privacy by adding noise drawn from a Laplace distribution to the output of a numeric query. The scale of the noise is proportional to the query's sensitivity divided by ε.

• Formula: Noise ~ Laplace(scale = Δf / ε)
• Use Case: Ideal for releasing aggregate statistics (counts, averages) or when applying DP to individual training steps in machine learning.
• Implementation: Simple to implement but the noise can be large for high-dimensional vectors like model gradients.

The Gaussian Mechanism provides (ε, δ)-Differential Privacy by adding noise drawn from a Gaussian (normal) distribution. It is preferable for high-dimensional vectors (like gradients in deep learning) because the L2 norm of the noise grows more slowly with dimension compared to Laplace noise.

• Formula: Noise ~ N(0, σ²) where σ is calibrated to the L2 sensitivity, ε, and δ.
• Use Case: The default for differentially private stochastic gradient descent (DP-SGD) in frameworks like TensorFlow Privacy and Opacus.
• Trade-off: Requires accepting a non-zero δ.

Privacy Amplification is a powerful technique where applying DP to a random subset of the data (subsampling) provides a stronger privacy guarantee than if applied to the full dataset. This is fundamental to making DP-SGD feasible.

• Principle: If a mechanism is (ε, δ)-DP on the full dataset, applying it to a random sample (with probability q) amplifies privacy to roughly (O(qε), qδ).
• Implementation: In DP-SGD, each training step uses a randomly sampled mini-batch. The privacy analysis (via the Moments Accountant) formally quantifies this amplification, allowing for a higher learning rate or more training steps within a fixed privacy budget.