DP-SGD

Before aggregation, the gradient for each individual data point is computed and its L2 norm is bounded by a fixed threshold C. This limits the maximum influence any single record can have on the model update. If the gradient norm exceeds C, it is scaled down proportionally. This step is essential because the sensitivity of the query—the maximum change in output from adding or removing one record—must be bounded to calibrate the noise correctly. Without clipping, a single outlier could dominate the gradient and require infinite noise to mask.

After clipping per-sample gradients and summing them into a batch gradient, calibrated Gaussian noise is added to the aggregate. The noise scale is proportional to the clipping threshold C and inversely proportional to the target privacy budget ε. This ensures the final model update is statistically indistinguishable from one computed on a dataset that excludes any single record. The noise is sampled from a zero-mean normal distribution with standard deviation σ · C, where σ is the noise multiplier derived from the privacy accountant.

DP-SGD employs a privacy accountant—typically based on Rényi Differential Privacy (RDP) or Gaussian Differential Privacy (GDP)—to track cumulative privacy loss across training iterations. Key components:

• Moments Accountant: Computes tight bounds on the privacy loss random variable across sequential queries
• Poisson Subsampling: Each example is included in a batch with fixed probability q, amplifying privacy via the subsampled Gaussian mechanism
• Composition Theorems: Combine per-iteration losses into a total (ε, δ)-DP guarantee
• The accountant outputs the final privacy budget spent, enabling practitioners to stop training before exceeding a predefined threshold.

Standard per-sample gradient clipping requires materializing the gradient for every example in a batch, multiplying memory usage by batch size. Ghost clipping avoids this by computing per-sample gradient norms directly from the loss gradient with respect to the activations, using the chain rule. This technique:

• Reduces peak memory from O(B · P) to O(P), where B is batch size and P is parameter count
• Enables DP-SGD on large models like fine-tuned LLMs without prohibitive hardware
• Is implemented in libraries like Opacus and TensorFlow Privacy for practical deployment

The core tension in DP-SGD is balancing privacy (ε) against model accuracy. Smaller ε (stronger privacy) requires larger noise, degrading utility. Mitigation strategies include:

• Larger batch sizes: Reduce noise variance through aggregation before noising
• More training epochs: Compensate for noisy updates with additional iterations, though this consumes more privacy budget
• Tighter accounting: Using GDP or RDP instead of basic composition can save 2-5x in ε for the same noise level
• Public pre-training: Starting from a model pre-trained on public data reduces the number of private updates needed
• Typical deployments target ε ∈ [1, 8] for production systems, with ε < 1 considered strong privacy.

DP-SGD directly counters gradient inversion attacks like Deep Leakage from Gradients (DLG) by ensuring shared gradients carry a formal privacy guarantee. The Gaussian noise added during training destroys the fine-grained signal that optimization-based reconstruction relies on. Combined with secure aggregation in federated settings, DP-SGD provides defense-in-depth:

• The server never sees individual client updates in plaintext (cryptographic protection)
• Even the aggregated update is differentially private (statistical protection)
• An adversary with full access to the noisy aggregate cannot reconstruct training data beyond the ε guarantee