Moment Accountant

Instead of tracking the final privacy parameters (ε, δ) directly, the moment accountant tracks a bound on the moment generating function of the privacy loss random variable. For each step t, it computes an upper bound on the α-th moment (or Rényi divergence of order α) of the privacy loss. This provides a more detailed statistical profile of the privacy loss distribution, enabling tighter composition.

By working with Rényi differential privacy (RDP) as an intermediate representation, the moment accountant provides significantly tighter composition than advanced composition theorems for (ε, δ)-DP. For a fixed number of iterations T and noise scale σ, it yields a smaller final ε for the same δ, or allows for more iterations within the same privacy budget. This is critical for making DP-SGD practical for deep learning.

The method naturally incorporates privacy amplification by subsampling. When DP-SGD uses Poisson sampling (each example included in a batch independently with probability q), the moment accountant can precisely compute the amplified RDP guarantee for each step. This formalizes the intuitive privacy benefit of stochastic gradient descent and is a key reason for its superior performance over non-iterative methods.

The final output is not a single (ε, δ) pair, but a privacy curve—a function mapping δ to ε. This provides a complete characterization of the trade-off between the two parameters. Practitioners can then select an operating point (e.g., ε for δ = 1e-5) that meets their requirements. The curve is derived by converting the accumulated RDP bounds back to (ε, δ)-DP using a standard conversion formula.

The moment accountant is the standard privacy accounting method in major DP deep learning libraries like TensorFlow Privacy and Opacus. Its algorithmic steps are:

• Initialize: Set moment orders α to a list of values (e.g., range(1, 65)).
• Per-Iteration: For each training step with noise σ and sampling rate q, compute the RDP guarantee ε_α for each α.
• Compose: Log-sum-exp the ε_α values across all T steps.
• Convert: For a target δ, find the minimal ε using the RDP-to-(ε,δ) conversion.

The moment accountant is fundamentally more efficient than naive or advanced composition.

• Naive (Sequential) Composition: Simply adds the ε of each step. Highly conservative, wastes budget.
• Advanced Composition: Provides an O(sqrt(T)) bound on ε growth, but still loose for high T.
• Moment Accountant: Provides an O(T) bound for fixed α, but the constant is much smaller due to the moment analysis and subsampling amplification, leading to orders of magnitude better privacy-utility trade-offs in practice.

Differential privacy is a rigorous mathematical framework for quantifying and limiting the privacy loss incurred by an individual when their data is used in a computation. It provides a strong, statistical guarantee that the presence or absence of any single record in the dataset has a negligible effect on the algorithm's output.

• Formal Guarantee: An algorithm is (ε, δ)-differentially private if for any two adjacent datasets differing by one record, the probability of any output changes by at most a factor of e^ε, plus a small failure probability δ.
• Post-Processing Immunity: A critical property where any analysis performed on the output of a DP mechanism cannot weaken its privacy guarantee.
• The moment accountant is a method for privacy accounting within this framework, specifically designed for iterative processes like DP-SGD.

Privacy accounting is the systematic process of tracking the cumulative privacy loss (ε, δ) across multiple, potentially adaptive, queries or training steps to ensure the total does not exceed a pre-defined privacy budget.

• Composition: The core challenge is understanding how privacy parameters degrade when mechanisms are composed (e.g., over many training epochs).
• Advanced Methods: Basic composition theorems are loose. The moment accountant and Rényi DP (RDP) provide tighter, more practical bounds for complex, iterative algorithms.
• This is essential for deploying DP in production, as it allows practitioners to provably bound total leakage and stop training before the budget is exhausted.

Rényi differential privacy is a relaxation of differential privacy defined using the Rényi divergence between the output distributions of an algorithm on adjacent datasets. It is particularly useful for tight privacy loss analysis under composition.

• Parameterization: Defined by an order α > 1 and a parameter ε(α). A smaller ε(α) indicates stronger privacy.
• Tighter Composition: RDP often yields significantly tighter privacy bounds for Gaussian-based mechanisms compared to basic (ε, δ)-DP composition.
• The moment accountant for DP-SGD is fundamentally based on bounding the Rényi divergence at each step, which is then converted to a final (ε, δ) guarantee.

Differentially Private Stochastic Gradient Descent is the primary training algorithm that enables deep learning with differential privacy. It modifies standard SGD by introducing two key steps:

• Per-Example Gradient Clipping: Each training example's gradient is clipped to a maximum L2 norm C to bound its sensitivity.
• Noisy Gradient Averaging: Calibrated Gaussian noise is added to the average of the clipped batch gradients before the model update.
• The moment accountant was developed specifically to provide tight privacy bounds for the iterative, subsampled Gaussian mechanism at the heart of DP-SGD, tracking privacy loss across thousands of training steps.

The Gaussian mechanism is a fundamental algorithm for achieving (ε, δ)-differential privacy. It works by adding noise drawn from a Gaussian (normal) distribution to the true output of a function.

• Noise Scale: The standard deviation σ of the noise is proportional to the function's L2 sensitivity Δ and inversely proportional to the privacy parameters ε and δ (σ = Δ * √(2log(1.25/δ)) / ε).
• Use in DP-SGD: In DP-SGD, the function is the average of clipped gradients in a mini-batch. The moment accountant provides a refined analysis for the repeated, subsampled application of this mechanism.
• It is the mechanism of choice when pure ε-DP (via the Laplace mechanism) is too restrictive or provides poor utility.

Privacy amplification by subsampling is a powerful technique where applying a differentially private mechanism to a randomly chosen subset of the data (e.g., a mini-batch) results in a stronger privacy guarantee than if it were applied to the full dataset.

• Core Principle: An adversary's chance of linking the output to any specific individual is reduced because that individual's data was only included with some probability (the sampling rate).
• Integration with Accounting: The moment accountant inherently and optimally accounts for this amplification effect when analyzing DP-SGD, which uses Poisson sampling or uniform sampling without replacement for mini-batches.
• This is why the privacy cost per step in DP-SGD is often much lower than naive composition would suggest.