Differential Privacy Stochastic Gradient Descent (DP-SGD)

The foundational step that bounds the sensitivity of the training algorithm. Instead of computing the average gradient over a batch, DP-SGD computes the gradient for each individual data point and clips its L2 norm to a fixed threshold C. This ensures no single training example can exert an arbitrarily large influence on the model update, limiting the maximum impact any individual's data can have on the final parameters.

After clipping and averaging the per-sample gradients, calibrated random noise drawn from a Gaussian distribution is added to the aggregated gradient. The scale of this noise is proportional to the clipping threshold C and inversely proportional to the target privacy loss parameter epsilon. This noise obfuscates the contribution of any single data point, making it statistically impossible to determine whether a specific record was included in the training set.

A sophisticated privacy accounting method used to track the cumulative privacy loss epsilon over many training iterations. Unlike basic composition theorems, the Moments Accountant computes the Rényi Differential Privacy bound at each step, providing a much tighter estimate of the total privacy budget consumed. This allows for significantly more training epochs before exhausting the privacy guarantee compared to naive strong composition theorems.

The inherent tension between the strength of the privacy guarantee and model accuracy. A smaller epsilon (stronger privacy) requires a larger noise multiplier σ, which degrades the signal-to-noise ratio of the gradient updates. This often manifests as reduced accuracy on underrepresented classes or tail data points. Tuning the clipping threshold C is critical: too low destroys useful signal, while too high increases sensitivity and requires more noise.

DP-SGD requires computing gradients on a per-example basis, which is memory-intensive for standard deep learning frameworks. Microbatching splits a logical batch into smaller physical batches, computing per-sample gradients sequentially before clipping and accumulating them. Vectorized loss functions that return per-sample losses—rather than a single scalar average—are essential for efficient implementation without prohibitive memory overhead.

A privacy amplification technique where each training step uses a random subset of the data rather than the full dataset. By applying Poisson sampling or uniform shuffling, the probability that any specific record participates in a given step is reduced. This uncertainty amplifies the privacy guarantee, allowing a lower effective epsilon for the same noise scale. The privacy amplification by subsampling theorem quantifies this improvement.

The mathematical framework that DP-SGD implements. It provides a formal guarantee that the output of a computation is statistically indistinguishable whether or not any single individual's data is included. The guarantee is quantified by the privacy loss parameter epsilon (ε), where lower values indicate stronger privacy. DP-SGD achieves this by bounding the influence of any single training example through per-sample gradient clipping and injecting calibrated Gaussian noise.

The fundamental tension managed by DP-SGD hyperparameters. Increasing the noise multiplier strengthens privacy guarantees but degrades model accuracy. Decreasing the clipping threshold limits per-sample influence but may discard useful gradient information. DP-SGD practitioners must empirically tune these parameters to find an acceptable operating point where the model remains useful while satisfying a predefined privacy budget.

A primary threat model that DP-SGD is designed to defeat. In this attack, an adversary queries a trained model to determine if a specific record was in the training set. DP-SGD's noise injection bounds the success rate of such attacks by ensuring the model's outputs do not overfit to individual examples. Formal differential privacy guarantees provide a provable upper bound on the attacker's true positive rate and false positive rate.

The core mechanism within DP-SGD that enforces a sensitivity bound. Before averaging gradients across a mini-batch, the L2 norm of each per-sample gradient is computed. If it exceeds a predefined clipping threshold (C), the gradient is scaled down to have norm exactly C. This ensures that no single training example can contribute more than C to the aggregate update, a prerequisite for meaningful differential privacy guarantees.

The process of tracking cumulative privacy loss during DP-SGD training. Advanced composition theorems, such as Rényi Differential Privacy (RDP) accounting, provide tight bounds on the total ε spent over many training iterations. The moments accountant tracks higher-order moments of the privacy loss random variable, enabling accurate budget tracking and preventing unintentional privacy leakage from excessive training epochs.

A core privacy principle that DP-SGD complements technically. While data minimization mandates collecting only necessary data, DP-SGD ensures that even necessary data cannot be memorized. Training with DP-SGD provides a strong technical control demonstrating that the model learns population-level patterns rather than individual records, supporting compliance with purpose limitation requirements under regulations like GDPR.