Differentially Private Stochastic Gradient Descent (DP-SGD)

Before aggregation, the gradient of each individual training example is bounded to a maximum L2 norm C. This prevents any single outlier record from contributing disproportionately to the model update.

• Clipping ensures the sensitivity of the query is bounded
• Gradients exceeding C are scaled down while preserving direction
• Lower C values reduce sensitivity but may discard useful information
• The clipping threshold is a critical hyperparameter balancing privacy and utility

After clipping and aggregating per-sample gradients, isotropic Gaussian noise drawn from N(0, σ²C²I) is added to the summed gradient. The noise scale σ is calibrated to the desired privacy parameter ε.

• Noise variance scales with the clipping norm squared
• Higher σ provides stronger privacy at the cost of model accuracy
• The noise mechanism satisfies (ε, δ)-differential privacy
• The privacy loss random variable follows a sub-Gaussian distribution

When each training step uses a random subset of the data rather than the full batch, privacy guarantees are amplified. An adversary cannot be certain whether a target record was included in any given step.

• Poisson subsampling provides tighter privacy accounting than uniform sampling
• Amplification reduces the ε cost per query by approximately the sampling rate q
• Enables training on large datasets with acceptable total privacy budgets
• The moments accountant tracks amplified privacy loss across iterations

A privacy accounting technique that tracks the log moments of the privacy loss random variable across training steps. It provides a tight bound on the total (ε, δ) guarantee compared to basic composition theorems.

• Computes the moment-generating function of the privacy loss
• Provides significantly tighter bounds than strong composition
• Accounts for subsampling amplification automatically
• Enables training for more iterations under a fixed privacy budget

DP-SGD consumes a quantifiable privacy budget with each training iteration. The total (ε, δ) guarantee is computed by composing the per-step privacy loss across all iterations.

• ε (epsilon): upper bound on privacy loss, typically 0.1–10 for practical deployments
• δ (delta): probability of catastrophic privacy failure, usually < 1/|dataset|
• Budget is exhausted linearly under basic composition, sublinearly under advanced accounting
• Training must halt when the target budget is reached

The clipping norm C and noise multiplier σ create a fundamental tension. Tighter privacy requires smaller C and larger σ, which degrades model convergence and final accuracy.

• Excess clipping introduces bias by distorting gradient directions
• Large noise variance slows convergence and increases training variance
• Larger batch sizes reduce noise relative to signal
• Pre-training on public data can mitigate utility loss under tight budgets

The primary threat model that DP-SGD is designed to defend against. An adversary attempts to determine whether a specific record was used in training by analyzing the model's outputs.

• Shadow Models: Attackers train surrogate models to mimic target behavior
• Confidence Scores: Overconfident predictions on training data are a key leakage signal
• DP-SGD Defense: The added noise makes training and non-training records statistically indistinguishable
• Auditing: Membership inference is used empirically to validate privacy guarantees

A more severe attack where adversaries reconstruct representative features or full training samples by exploiting access to model parameters and gradients.

• Gradient Leakage: Raw gradients can reveal pixel-level information about training batches
• Feature Reconstruction: Attackers optimize inputs to maximize class confidence, revealing prototypes
• DP-SGD Mitigation: Per-sample clipping and noise injection obscure the gradient signal
• Defense-in-Depth: Combine DP-SGD with secure aggregation and gradient compression

A quantifiable limit on total privacy loss, parameterized by ε. DP-SGD consumes a portion of this budget with each training step.

• Moments Accountant: The algorithm used to track cumulative privacy loss across DP-SGD iterations
• Rényi DP: A relaxation that provides tighter composition bounds for Gaussian mechanisms
• Budget Depletion: Once ε is exhausted, no further queries on that data are permitted
• Typical Values: ε = 1-8 for production models; ε < 1 for high-sensitivity applications

An alternative to DP-SGD that achieves differential privacy by training an ensemble of teacher models on disjoint data partitions and having a student model learn only from their noisy aggregated votes.

• Teacher-Student Architecture: Teachers never share raw data or gradients
• Noisy Voting: Laplacian noise is added to teacher vote counts before student training
• Comparison to DP-SGD: PATE is better for classification tasks; DP-SGD is more general for deep learning
• Semi-Supervised PATE: Uses unlabeled public data to improve student accuracy

The critical preprocessing step in DP-SGD that bounds the influence of any single training example on the model update.

• Per-Sample Clipping: Each individual gradient is scaled to have L2 norm ≤ C
• Clipping Threshold (C): A hyperparameter balancing privacy and utility; too low destroys signal, too high adds excessive noise
• Adaptive Clipping: Dynamically adjusts C based on gradient norm statistics during training
• Microbatch Processing: Required to compute per-sample gradients efficiently in modern frameworks