Gaussian Differential Privacy (GDP)

GDP defines privacy through the lens of hypothesis testing. It characterizes the difficulty of distinguishing between two neighboring datasets by bounding the trade-off between Type I (false positive) and Type II (false negative) errors. This is formalized using trade-off functions, providing an operational interpretation of the privacy guarantee.

The framework leverages f-divergences to measure the discrepancy between output distributions. A mechanism satisfies μ-GDP if the trade-off function is bounded by the trade-off function of a Gaussian distribution. This connection allows for the use of powerful information-theoretic tools to analyze privacy loss.

GDP provides a Central Limit Theorem (CLT) for privacy. When composing many independent differentially private mechanisms, the overall privacy loss converges precisely to a Gaussian trade-off function. This avoids the looseness of standard composition theorems, enabling more accurate privacy accounting.

A key advantage of GDP is its ability to handle subsampling with exact analytical formulas. The privacy amplification from Poisson subsampling is characterized precisely, without the numerical integration or approximations required in other frameworks. This is essential for the tight accounting of DP-SGD.

GDP can be seamlessly converted to the classical (ε, δ)-differential privacy definition. A μ-GDP mechanism satisfies (ε, δ)-DP for a continuum of (ε, δ) pairs defined by a specific functional relationship. This allows practitioners to report guarantees in the more widely recognized standard while benefiting from GDP's tight composition.

The analytical tractability of GDP makes it ideal for implementation as a privacy accountant. Libraries like Opacus use GDP-based accounting to track the cumulative privacy budget (μ) during training. The accountant computes the exact μ after each step, stopping training when the target privacy level is reached.

The foundational hypothesis-testing framework underlying GDP. It defines privacy loss through the trade-off between Type I and Type II errors in distinguishing two neighboring datasets. A mechanism is f-DP if no adversary can achieve a false positive rate α and false negative rate β better than a specified trade-off function f. GDP is the specific instance where f corresponds to distinguishing two Gaussian distributions.

The quantified parameter bounding maximum privacy loss in pure ε-differential privacy. GDP reframes this through the lens of hypothesis testing rather than a single scalar, but the relationship is direct: a μ-GDP mechanism satisfies (ε, δ)-DP for any ε ≥ 0 where δ = Φ(-ε/μ + μ/2) - e^ε Φ(-ε/μ - μ/2). This allows practitioners to translate between the two frameworks.

An algorithmic component that tracks cumulative privacy loss across iterative training steps. In GDP, composition is elegantly handled by the central limit theorem for differential privacy: the composition of many DP mechanisms converges to a Gaussian trade-off function. This provides tight, exact accounting without the looseness of advanced composition theorems used in standard DP.

The phenomenon where randomly sampling a subset of data before applying a DP mechanism yields stronger guarantees. GDP provides tight analytical bounds for subsampled Gaussian mechanisms without numerical integration. Key result: if a mechanism is μ-GDP, applying Poisson sampling with rate q amplifies the guarantee to sqrt(q) · μ-GDP asymptotically.

A relaxation of pure DP based on Rényi divergence that provides tighter composition than standard DP. GDP offers strictly tighter bounds than RDP for Gaussian mechanisms. While RDP tracks privacy loss across orders α, GDP characterizes the entire trade-off function in a single parameter μ, eliminating the need to optimize over α and providing exact rather than approximate composition.

The differentially private variant of stochastic gradient descent that clips per-sample gradients and adds Gaussian noise. GDP provides the theoretically optimal privacy analysis for DP-SGD with subsampling. The tight composition guarantees mean that for a given privacy target, GDP analysis permits more training iterations or less noise than standard moment accountant methods, directly improving model utility.