Differential Privacy SGD (DP-SGD)

The first critical step in DP-SGD. Before noise is added, the L2-norm of every per-example gradient is computed and capped to a fixed threshold C. This bounds the sensitivity of the query, ensuring that no single training point can exert an arbitrarily large influence on the model update. Without clipping, the magnitude of noise required to achieve a given privacy guarantee would be infinite, destroying utility. The clipping threshold is a crucial hyperparameter: set too low, it biases the model; set too high, it requires excessive noise.

The formal privacy engine of DP-SGD. After gradients are clipped and aggregated, calibrated Gaussian noise with zero mean and variance proportional to (C * σ)² is added to the summed gradient. This mechanism satisfies (ε, δ)-differential privacy, where the noise scale σ (the noise multiplier) directly controls the privacy loss parameter ε. The Gaussian distribution is chosen because it composes tightly over multiple iterations via the moments accountant, enabling the tracking of cumulative privacy expenditure across the entire training run.

A sophisticated privacy accounting technique that tracks the cumulative privacy loss over many iterations of DP-SGD. Unlike simple composition theorems that sum ε values linearly, the moments accountant computes the Rényi divergence of the privacy loss random variable at each step, providing a much tighter bound on the total (ε, δ) guarantee. This is essential because naive composition would overestimate privacy loss by orders of magnitude, forcing the use of impractically large noise scales that would destroy model accuracy.

Standard SGD computes the average gradient over a mini-batch, losing per-sample information. DP-SGD fundamentally requires computing the gradient for each individual training example separately before aggregation. This is necessary to clip each gradient independently, bounding the contribution of any single record. This per-example computation is the primary engineering challenge of DP-SGD, as modern deep learning frameworks are optimized for batched operations. Techniques like backpropagation vectorization and efficient per-example norm computation are required to make DP-SGD practical.

The formal parameters that quantify the privacy guarantee. Epsilon (ε) is the privacy loss parameter: a smaller ε (e.g., 0.1) provides stronger privacy, while a larger ε (e.g., 10) provides weaker privacy. Delta (δ) is the failure probability, representing the small chance that the privacy guarantee is violated; it is typically set to be cryptographically small, such as 1/N where N is the dataset size. The budget is consumed with each training step, and training must halt once the target (ε, δ) is reached to prevent privacy leakage.

The mathematical foundation that determines how much noise is needed. Sensitivity measures the maximum change in a function's output when a single record is added or removed from the dataset. In DP-SGD, gradient clipping explicitly bounds the L2-sensitivity to C. This bounded sensitivity is then multiplied by the noise multiplier σ to calibrate the Gaussian mechanism. Understanding sensitivity is critical: functions with high sensitivity require more noise, creating a direct trade-off between privacy and utility.