Client-Level Differential Privacy

DP-FedAvg is the canonical algorithm for applying client-level DP in federated learning. It modifies the standard Federated Averaging process with three key steps executed on each participating client:

• Gradient/Update Clipping: The client's local model update (e.g., the gradient or weight delta) is clipped to a maximum L2 norm C. This bounds the sensitivity—the maximum influence any single client can have on the aggregate.
• Noise Addition: Calibrated Gaussian noise is added to the clipped update. The noise scale σ is proportional to the clip threshold C and the target privacy parameters (ε, δ).
• Secure Aggregation: The noised updates are sent to the server, which averages them. While Secure Aggregation (via cryptographic protocols) hides individual updates from the server, the DP guarantee holds even if the server is malicious, as the noise was already added client-side.

The Gaussian Mechanism is the primary noise-adding procedure used to achieve (ε, δ)-differential privacy in iterative algorithms like DP-FedAvg. It adds noise drawn from a Gaussian (normal) distribution to a numeric function output.

• Noise Calibration: For a function with L2 sensitivity Δ₂f, adding noise scaled to N(0, σ²) where σ = Δ₂f * √(2log(1.25/δ)) / ε guarantees (ε, δ)-DP.
• Application in FL: In client-level DP, the function is the client's clipped model update. The sensitivity Δ₂f is precisely the clip threshold C. The server, upon receiving many noised updates, computes a noisy average, providing the collective privacy guarantee.

Gradient Clipping is a prerequisite for the Gaussian Mechanism, as it establishes a finite, known sensitivity. Sensitivity is the maximum possible change in a function's output when a single client's data is added or removed.

• L2-Norm Clipping: Each client computes their local update vector g, then computes ḡ = g / max(1, ||g||₂ / C). This ensures ||ḡ||₂ ≤ C.
• Trade-off: The clip threshold C is a critical hyperparameter. A small C reduces sensitivity, allowing less noise for the same (ε, δ), but can introduce bias by overly distorting updates from clients with large, informative gradients. Tuning C balances utility and privacy.

Privacy Accounting is the process of tracking the cumulative privacy loss (ε, δ) across multiple training rounds. The Moment Accountant (used in DP-SGD/DP-FedAvg) provides tight bounds for composition.

• Mechanism: It tracks a bound on the moments (logarithm of the moment generating function) of the privacy loss random variable for each training step (client sampling + noise addition).
• Tight Composition: By analyzing the Gaussian mechanism's behavior under Poisson subsampling (each client participates with probability q per round), the Moment Accountant yields a much smaller final ε than naive sequential composition, a phenomenon known as privacy amplification by sampling.

Privacy Amplification by Subsampling is a crucial phenomenon where applying a DP mechanism to a random subset of the population (clients) provides a stronger privacy guarantee than applying it to the entire population.

• Process: In each federated learning round, the server selects a random subset of clients (e.g., via Poisson sampling). The DP mechanism (noise addition) is applied only to their updates.
• Result: The effective privacy cost ε_eff per round is smaller than the ε used for each client's local mechanism. This allows for more training rounds within a fixed total privacy budget, significantly improving model utility while maintaining the same formal guarantee.

Secure Aggregation (SecAgg) is a cryptographic protocol that allows a server to compute the sum of client vectors without learning any individual vector. While not a DP mechanism itself, it is a powerful complement.

• Role in DP-FL: In a DP-FedAvg pipeline, clients can add DP noise before SecAgg. SecAgg then hides which client contributed which noised update, providing an additional layer of privacy. This is sometimes called the "distributed DP" or "shuffle model" view.
• Key Benefit: It prevents the server from performing reconstruction attacks on individual noised updates, forcing an adversary to attack only the already-private aggregate. This combination—local noise plus secure summation—is a gold-standard architecture for high-assurance client-level privacy.

Enables collaborative training of diagnostic models (e.g., for cancer detection or rare disease prediction) across multiple hospitals without sharing sensitive patient health records. Client-level DP ensures that the model cannot reveal whether a specific patient's data was used in training, complying with regulations like HIPAA and GDPR. A common implementation is DP-FedAvg, where each hospital adds noise to its model update before aggregation.

Used by tech companies to improve language models for predictive text and autocorrect by learning from typing patterns on millions of user devices. Local updates (e.g., gradient information) are clipped and noised on-device before being sent to the central server. This protects users from being identified by their unique phrasing or frequently used words, while still allowing the global model to learn common patterns.

Banks and financial institutions can collaboratively train models to detect novel fraud patterns without exposing transaction histories of individual customers. Client-level guarantees protect each participating bank's proprietary customer data. Techniques involve applying the Gaussian mechanism to aggregated model updates from each institution, ensuring a malicious actor cannot infer if a specific bank's data contributed to a detected fraud rule.

Used to aggregate usage statistics and improve performance models for smart home devices (e.g., thermostats, security cameras) or industrial sensors. Each device acts as a client, applying local differential privacy to its usage reports before they leave the device. This prevents inferring sensitive household routines or industrial operational patterns from the aggregated analytics used to optimize firmware.

Allows competing organizations in the same sector (e.g., automotive manufacturers, retail chains) to jointly train AI models for common challenges like supply chain optimization or predictive maintenance. Client-level DP, combined with secure aggregation, ensures no participant can learn the proprietary data or business metrics of any other single participant, only benefiting from the improved shared model.

Balances global model improvement with local user privacy for applications like content recommendation and adaptive assistants. The global model is trained with client-level DP, preventing the server from identifying individual user preferences. Personalization then occurs safely on the user's device via local fine-tuning, ensuring sensitive preferences never leave the device in raw form.

The foundational mathematical framework for quantifying and bounding privacy loss. It provides a rigorous definition: the output of an algorithm should be nearly indistinguishable whether any single individual's data is included or excluded from the input dataset.

• Formal Guarantee: Ensures an adversary cannot confidently infer an individual's participation.
• Parameters: Defined by (ε, δ), where ε is the privacy loss bound and δ is a small probability of failure.
• Core Property: Post-processing immunity—any analysis on a DP output remains private.

A model where each client perturbs their own data locally before sending it to an untrusted server. This provides a stronger, client-side guarantee as the server never sees raw data.

• Contrast with Central DP: In central DP, a trusted curator applies noise; in LDP, no trusted curator is needed.
• Typical Mechanism: Randomized response for categorical data or adding calibrated noise to numerical values.
• Use Case: Deployed in industry for privacy-preserving data collection (e.g., Google's RAPPOR, Apple's iOS analytics).

The canonical algorithm for training deep learning models with DP. It modifies standard stochastic gradient descent by:

1. Per-Example Gradient Clipping: Bounding the L2 norm of each training example's gradient to a threshold C.
2. Adding Gaussian Noise: Injecting calibrated Gaussian noise to the average of the clipped batch gradients.
3. Privacy Accounting: Using a moment accountant or Rényi DP to track the cumulative privacy budget (ε, δ) across training epochs.

It is the conceptual precursor to private federated averaging algorithms.

A cryptographic protocol that allows a server to compute the sum of client updates without learning any individual client's contribution. It is often combined with DP for a layered defense.

• Mechanism: Uses multi-party computation (MPC) or homomorphic encryption. Clients can mask their updates with secret shares that cancel out upon summation.
• Synergy with DP: Secure Aggregation protects the content of individual updates during transmission; DP adds noise to the final aggregate to protect against inference from the result.
• Example: Google's original Federated Learning system uses Secure Aggregation alongside potential DP guarantees.

A powerful technique where applying a DP mechanism to a random subset of the population (clients) provides a stronger privacy guarantee than if applied to the full set. This is critical for making DP practical in federated learning.

• How it Works: If only a fraction q of clients are selected per training round, the effective privacy parameter ε can be reduced (e.g., to ~q * ε for pure DP).
• System Impact: Enables the use of less noise for the same overall privacy guarantee, improving model utility.
• Client Selection: This makes the strategy for random client selection a direct component of the privacy analysis.

A variant of DP defined using Rényi divergence. It is not an alternative guarantee but a tool for tighter privacy accounting, especially for composition of Gaussian mechanisms like those used in DP-SGD and DP-FedAvg.

• Advantage over (ε,δ)-DP: Provides cleaner and often tighter bounds on privacy loss when analyzing the composition of many training steps.
• Workflow: Mechanisms are often analyzed and proven in the RDP framework, then the final result is converted to a traditional (ε, δ)-DP guarantee for interpretation.
• Industry Standard: The moment accountant used in the original DP-SGD paper is essentially an implementation of RDP accounting.