DP-SGD

Unlike standard SGD which computes gradients over a mini-batch, DP-SGD computes per-example gradients for each data point independently. Each gradient's L2 norm is then bounded to a fixed threshold C:

• If ||g(x_i)||_2 > C, the gradient is scaled down to C * g(x_i) / ||g(x_i)||_2
• If ||g(x_i)||_2 ≤ C, the gradient remains unchanged

This bounding operation limits the sensitivity of the training update—the maximum influence any single record can exert on the model—which is a prerequisite for meaningful privacy guarantees.

After clipping and averaging the per-example gradients, DP-SGD adds isotropic Gaussian noise to the aggregated gradient before applying the optimizer update:

• Noise is drawn from N(0, σ²C²I) where σ is the noise multiplier
• The standard deviation is proportional to the clipping threshold C and inversely proportional to the privacy budget ε
• This step ensures the final model update satisfies (ε, δ)-differential privacy via the Gaussian mechanism

The noise obscures the contribution of any individual example, making it impossible to determine whether a specific record was included in the training set.

DP-SGD leverages Poisson subsampling or uniform random sampling to select mini-batches, which provides a crucial privacy amplification effect:

• Each example is included in a batch with probability q = B/N (batch size / dataset size)
• The randomness of sampling introduces uncertainty about whether any given record participated in a training step
• This amplifies the effective privacy guarantee beyond what the Gaussian mechanism alone provides

The Moments Accountant tracks this amplified privacy loss across iterations, yielding significantly tighter bounds than basic composition theorems.

The Moments Accountant is the privacy accounting algorithm introduced alongside DP-SGD that tracks cumulative privacy loss more accurately than standard composition theorems:

• Computes the log moments of the privacy loss random variable at each training step
• Accumulates these moments across all iterations using linear composition
• Converts the total moment bound back to an (ε, δ) guarantee via tail bounds

This technique reduces the estimated privacy budget consumption by orders of magnitude compared to advanced composition, making deep learning with differential privacy practically feasible.

DP-SGD introduces a fundamental privacy-utility trade-off governed by three hyperparameters:

• Noise multiplier (σ): Higher values increase privacy but degrade model accuracy
• Clipping threshold (C): Too low destroys signal; too high requires more noise
• Batch size (B): Larger batches improve utility but reduce subsampling amplification

Practical deployments require careful tuning. Typical settings achieve ε ≈ 8 with < 5% accuracy degradation on CIFAR-10, while tighter budgets (ε < 1) may incur 10-20% drops. Rényi Differential Privacy (RDP) accounting can further optimize this trade-off.

A critical property of DP-SGD is that the trained model enjoys post-processing immunity:

• Any computation applied to the model's outputs—inference, fine-tuning, ensembling, or publishing—cannot weaken the original privacy guarantee
• This holds because differential privacy is closed under arbitrary post-processing
• An adversary with access to the model cannot infer more about the training data than what the (ε, δ) bound permits, regardless of how they query or manipulate the model

This property ensures that models trained with DP-SGD remain safe for deployment in untrusted environments, including public API endpoints and on-device inference.