Model Inversion Defense

A practical implementation of differential privacy where clients or the server inject random noise into the model updates (gradients) before aggregation or sharing.

• Local DP: Clients add noise to their updates before sending them to the server, providing a strong, distributed guarantee.
• Central DP: The server adds noise to the aggregated global update before broadcasting the new model.
• Calibration: The noise variance is carefully calibrated to the update's L2-sensitivity and the desired privacy parameters (epsilon, delta).

Pre-processing techniques that limit the information content and influence of individual client updates, making inversion attacks more difficult.

• Gradient Clipping: Bounding the L2-norm of each client's update vector before aggregation. This caps the contribution of any single data point, reducing sensitivity for DP noise addition.
• Quantization: Reducing the precision (e.g., to 8-bit integers) of update values before transmission, which acts as a form of noise.
• Sparsification: Transmitting only the largest-magnitude gradient values, which obscures the complete update signal.

Cryptographic methods that allow the server to compute the sum of client updates without inspecting individual contributions, thus preventing inversion from a single update.

• Multi-Party Computation (MPC): Clients secret-share their updates so the server can only reconstruct the sum.
• Homomorphic Encryption (HE): Clients send encrypted updates; the server performs aggregation on the ciphertext and decrypts only the final sum.
• Benefit: Provides strong confidentiality for updates during transmission and aggregation, complementing DP's statistical guarantees.

Closely related techniques that prevent an adversary from determining if a specific data record was in the training set, a prerequisite for precise model inversion.

• Regularization: Techniques like dropout or L2 regularization during local client training reduce model overfitting, which is exploited by membership inference attacks.
• Adversarial Training: Clients can fine-tune their local models on generated adversarial examples designed to fool membership inference classifiers.
• Model Stacking: Using ensemble predictions or label smoothing to reduce the confidence scores that attackers rely on.

Defenses applied to the model's final prediction outputs (logits or probabilities) to obscure the detailed information used in inversion attacks.

• Logit Clipping/Noise: Adding small noise or bounding the range of values in the logit vector returned by the model.
• Temperature Scaling: Using a high temperature in the softmax function to 'flatten' the output probability distribution.
• Top-k Prediction Restriction: The model only returns the top k class labels and their probabilities, rather than the full vector, limiting the signal for iterative inversion.

A set of techniques designed to prevent an adversary from determining whether a specific data record was part of a client's private training dataset by analyzing the shared global model or its updates. This attack is a precursor or companion to model inversion.

• Core Mechanism: Often involves adding noise (via differential privacy) or regularizing the model to reduce its memorization of individual training points.
• Key Difference: While model inversion aims to reconstruct data features, membership inference only seeks a binary yes/no answer about data presence.
• Example: A hospital uses a federated model for disease prediction. Membership inference defense ensures an attacker cannot confirm if a specific patient's record was used in training.

A rigorous mathematical framework that provides provable guarantees against privacy leakage by adding calibrated noise to data or computations. It is a foundational technique for defending against model inversion and membership inference.

• Local DP (LDP): Clients add noise to their model updates before sending them to the server, providing a strong, distributed guarantee.
• Central DP: The server aggregates updates and then adds noise to the global model before distribution.
• Privacy Budget: A key concept tracked via Privacy Accounting, limiting the total allowable privacy loss (epsilon, δ) across training rounds.

A cryptographic protocol that allows a federated learning server to compute the sum or average of client model updates without being able to inspect any individual client's contribution. This directly limits inversion attacks on individual updates.

• Purpose: Prevents the server from being an honest-but-curious adversary that could attempt to invert a single client's gradient.
• Mechanism: Often uses multi-party computation (MPC) or homomorphic encryption so that only the aggregated, masked result is revealed.
• Benefit: Enables privacy-preserving aggregation without necessarily adding the performance overhead of differential privacy noise.

Server-side defensive techniques that analyze and filter client-submitted model updates (gradients) before aggregation to detect and remove signals that could facilitate inversion or other attacks.

• Gradient Inspection: Analyzing the statistics (norms, distribution) of updates to flag anomalies indicative of malicious clients with inversion objectives.
• Update Sanitization: Actively modifying updates, for example through gradient clipping (bounding the norm) or noise addition, to obscure sensitive information.
• Role in Defense: Serves as a first line of server-side validation, reducing the risk that inverted features from a client's data propagate into the global model.

A class of algorithms designed to produce a correct global model update even when a fraction of participating clients are malicious (Byzantine) and may send arbitrary or adversarial updates, which could include updates engineered to leak information.

• Core Algorithms: Includes Krum, Bulyan, Trimmed Mean, and Median Aggregation.
• Defensive Role: While primarily for fault tolerance and poisoning defense, these methods can indirectly mitigate inversion by filtering out outlier updates that may be crafted to probe the model or expose other clients' data patterns.
• Connection: A system must be Byzantine-robust to reliably apply other defenses like secure aggregation or differential privacy in an adversarial environment.

Hardware-based secure enclaves (e.g., Intel SGX, ARM TrustZone) that create isolated, encrypted memory regions on a client device. They can execute code and process data—like local model training—in a manner opaque to the host operating system.

• Application: Clients can perform local training within a TEE, generating updates that are cryptographically attested (Federated Attestation) as coming from genuine, unmodified code.
• Defense Value: Protects the local training process and the resulting update from being observed or tampered with by malware on the client device, a potential vector for forcing an update that aids inversion.
• Limitation: Protects computation integrity but does not by itself provide formal privacy guarantees like differential privacy against a curious server.