Zero-Concentrated Differential Privacy (zCDP)

zCDP is formally defined using Rényi divergence. A randomized mechanism (\mathcal{M}) satisfies ρ-zCDP if for all neighboring datasets (D, D') and all (α ∈ (1, ∞)), the Rényi divergence of order (α) between the output distributions is bounded: (D_α(\mathcal{M}(D) | \mathcal{M}(D')) ≤ ρα). The single parameter ρ (rho) controls the privacy guarantee, where a smaller ρ indicates stronger privacy. This formulation directly captures the subgaussian tail behavior of the privacy loss random variable.

A primary advantage of zCDP is its clean and tight composition for mechanisms based on Gaussian noise, such as the Gaussian mechanism. Under zCDP, the composition of multiple mechanisms is additive: if mechanism (\mathcal{M}_1) satisfies (ρ_1)-zCDP and (\mathcal{M}_2) satisfies (ρ_2)-zCDP, their adaptive composition satisfies ((ρ_1 + ρ_2))-zCDP. This linearity simplifies privacy accounting for iterative algorithms like DP-SGD and DP-FedAvg, where gradients are noised with Gaussian distributions in each round.

zCDP can be converted to the more familiar (ε, δ)-Differential Privacy guarantee. For any (δ > 0), a mechanism satisfying ρ-zCDP also satisfies (ε, δ)-DP for (ε = ρ + 2\sqrt{ρ \cdot \log(1/δ)}). Conversely, pure ε-DP implies ((\frac{1}{2}ε^2))-zCDP. This relationship allows practitioners to interpret a zCDP guarantee in the (ε, δ)-DP framework for a chosen failure probability δ, which is crucial for setting parameters in production systems where a finite δ is acceptable.

The Gaussian mechanism is naturally analyzed under zCDP. For a function (f) with L2 sensitivity (Δ_2), releasing (f(D) + \mathcal{N}(0, σ^2I)) satisfies ρ-zCDP for (ρ = \frac{Δ_2^2}{2σ^2}). This provides a direct, simple formula linking the noise variance σ² to the privacy parameter ρ. In contrast, the (ε, δ)-DP analysis of the Gaussian mechanism requires a more complex relationship between ε, δ, Δ₂, and σ. This simplicity makes zCDP the preferred analysis for mechanisms where Gaussian noise is the primary privacy tool.

zCDP enables tight analysis of privacy amplification by subsampling, a critical technique in stochastic optimization. When a ρ-zCDP mechanism is applied to a uniform random sample of the dataset (with sampling rate (γ = b/N)), the effective privacy guarantee is amplified. For Poisson subsampling, the composed mechanism satisfies (ρ'\text{-zCDP}) with (ρ' ≈ γ^2 ρ) for small ρ. This quadratic amplification ((γ^2)) is tighter than the linear amplification often derived under (ε, δ)-DP, allowing for less noise to be added per step in algorithms like DP-SGD.

zCDP is the standard framework for analyzing client-level differential privacy in federated learning algorithms like DP-FedAvg. Each client clips their local model update to bound its L2 sensitivity, adds Gaussian noise, and sends it to the server. The zCDP framework cleanly accounts for the privacy cost across multiple training rounds and for the subsampling of clients in each round. The total privacy parameter ρ is computed by summing the per-round costs, which are themselves a function of the client sampling rate, clip norm, and Gaussian noise multiplier. This ρ is then converted to a final (ε, δ)-DP guarantee for reporting.

Rényi Differential Privacy (RDP) is a relaxation of differential privacy defined using the Rényi divergence between the output distributions of a mechanism on neighboring datasets. It provides a more composable privacy accounting framework, especially for iterative algorithms like DP-SGD. zCDP is essentially a special case of RDP that focuses on the subgaussian tail of the privacy loss random variable, offering a cleaner and often tighter analysis for mechanisms based on Gaussian noise.

• Key Relationship: zCDP can be viewed as a one-parameter subset of the two-parameter RDP family, providing a streamlined analysis path.

The Gaussian Mechanism is the fundamental algorithm for achieving (ε, δ)-differential privacy by adding noise drawn from a Gaussian (normal) distribution to the true output of a function. The scale of the noise (σ) is proportional to the function's L2 sensitivity and the desired privacy parameters. zCDP was specifically developed to provide a tighter and simpler privacy loss composition for this mechanism and its repeated applications, such as in DP-SGD, by directly analyzing the properties of Gaussian noise.

The Moment Accountant is an advanced privacy accounting technique used to tightly track the cumulative privacy loss (ε, δ) across many compositions of a mechanism, such as the thousands of gradient update steps in DP-SGD. It works by bounding the log moments of the privacy loss random variable. zCDP provides a conceptually cleaner and often more intuitive framework for achieving the same goal as the moment accountant, especially for Gaussian-based mechanisms, by directly working with the subgaussian parameter (ρ) of the privacy loss.

Privacy Amplification by Sampling is a phenomenon where applying a differentially private mechanism to a random subset of the data (e.g., a mini-batch in SGD) results in a stronger effective privacy guarantee than if it were applied to the full dataset. This is critical for making DP-SGD practical. zCDP interacts cleanly with this technique; the ρ parameter in zCDP composes linearly and can be sharply amplified when the mechanism is applied to a random sample, leading to more favorable final privacy parameters (ε, δ) after conversion.

Client-Level Differential Privacy specifies the granularity of protection in federated learning. The privacy guarantee ensures that the participation or data of any single client (device or user) cannot be reliably inferred from the released aggregated model or statistics. zCDP is a tool used to provide and account for this client-level guarantee. When DP-FedAvg uses the Gaussian mechanism on clipped client updates, employing zCDP for accounting yields a precise client-level (ε, δ)-DP guarantee after composition over training rounds.

Composition Theorems are the mathematical rules that dictate how privacy guarantees degrade when multiple differentially private mechanisms are applied to the same dataset, either sequentially (adaptive composition) or in parallel. A primary motivation for zCDP is to provide simpler and tighter composition bounds. Under zCDP, the privacy parameter ρ adds linearly under composition (ρ_total = Σ ρ_i), which is simpler and often yields better final (ε, δ) bounds after conversion than using basic composition theorems for (ε, δ)-DP directly.