Membership Inference Defense

A mathematical framework providing formal, quantifiable privacy guarantees. In federated learning, noise is added to client updates or the aggregated model.

• Local Differential Privacy (LDP): Each client adds calibrated noise (e.g., via the Gaussian Mechanism) to their model update before sending it to the server.
• Central DP: The server adds noise to the aggregated global model before distribution.
• Privacy Accounting: Tracks cumulative privacy loss (epsilon, δ) across training rounds to ensure a pre-defined privacy budget is not exceeded.

Techniques that reduce a model's memorization capacity for specific training points, making it harder to distinguish member from non-member data.

• Strong L2 Regularization: Penalizes large model weights, encouraging simpler functions that generalize better and memorize less.
• Dropout: Randomly deactivates neurons during training, preventing over-reliance on specific patterns.
• Model Compression: Using smaller model architectures or knowledge distillation to create less complex models that retain utility but expose less information about the training data.

Directly modifies the model updates (gradients) shared during federated training to obfuscate the contribution of individual data points.

• Gradient Clipping: Bounds the L2 norm of each client's update, limiting the influence of any single data point.
• Gradient Noise Addition: Injects random noise into updates, often calibrated to provide a differential privacy guarantee.
• Update Sanitization: The server applies filters or transformations to received updates to remove unusual patterns that could leak membership information.

Augments the training objective to explicitly penalize the model for features that make it vulnerable to membership inference attacks.

• The local loss function is modified to include a term that minimizes the prediction confidence gap between member and non-member data.
• Often involves training a shadow model or an adversarial discriminator during the local training phase to identify leaky features.
• This forces the model to learn representations that are useful for the primary task but invariant to the membership status of data points.

Modifies the model's prediction outputs to reduce the information an adversary can exploit.

• Label Smoothing: Replaces hard "0" or "1" labels with slightly softened values (e.g., 0.1, 0.9), preventing the model from becoming overconfident on training data.
• Prediction Bucketing: Reports predictions within ranges (e.g., "high confidence," "medium confidence") instead of precise probabilities.
• Temperature Scaling: Adjusts the softmax temperature at inference time to flatten the output probability distribution, reducing confidence disparities.

Leverages cryptographic protocols or hardware security to prevent the server from inspecting individual client updates, which are the primary attack vector.

• Secure Multi-Party Computation (MPC): Cryptographic protocols that allow the server to compute the sum/average of client updates without seeing any individual update.
• Homomorphic Encryption (HE): Allows computation on encrypted data; clients send encrypted updates, and the server aggregates them while still encrypted.
• Trusted Execution Environments (TEEs): Hardware enclaves (e.g., Intel SGX, ARM TrustZone) create secure, isolated regions on client devices for local training, with attestation to prove code integrity.

A privacy attack where an adversary uses the output of a trained machine learning model, or its update gradients, to reconstruct representative features or samples of the model's private training data. This is a more direct reconstruction attack compared to membership inference, which only seeks to determine data presence.

• Mechanism: Often involves optimizing an input to maximize confidence for a target class or analyzing gradient updates.
• Defense Link: Membership Inference Defense techniques, like gradient perturbation, also help mitigate model inversion by obscuring the specific data influence on the model.

A rigorous mathematical framework that provides a quantifiable privacy guarantee by ensuring the output of a computation (e.g., a model update) is statistically indistinguishable whether any single individual's data is included or excluded from the dataset.

• Core Mechanism: Adds calibrated noise (e.g., via the Gaussian Mechanism) to data or gradients.
• Formal Guarantee: Provides an (ε, δ)-differential privacy bound, where ε is the privacy budget.
• Primary Use: A foundational technique for Membership Inference Defense, as it formally limits an adversary's ability to infer membership from model outputs.

A server-side validation and sanitization technique where the norm (magnitude) of each client's model update vector is bounded to a maximum threshold before aggregation or further processing.

• Purpose: Limits the influence of any single client's update, which mitigates poisoning attacks and reduces the signal-to-noise ratio for membership inference.
• Process: If an update's L2 norm exceeds a threshold C, it is scaled down to C.
• Synergy with DP: Often used prior to adding differential privacy noise, as clipping bounds the sensitivity of the function, allowing for less noise to achieve the same privacy guarantee.

A cryptographic protocol that allows a federated learning server to compute the sum or average of client model updates without being able to inspect any individual client's update.

• How it Works: Uses multi-party computation (MPC) or homomorphic encryption so clients can mask their updates with secret shares that cancel out only when summed across all participants.
• Privacy Benefit: Protects against a curious server attempting to perform membership inference or model inversion on a single client's update.
• Limitation: While it hides individual values, the aggregate update may still leak statistical information, which is why it is often combined with differential privacy.

A robustness defense where clients locally train their models not only on clean data but also on adversarially perturbed examples, with the goal of improving the global model's resilience to evasion attacks and reducing its confidence calibration on outliers.

• Local Process: Each client generates adversarial examples (e.g., via Projected Gradient Descent) against its current local model and includes them in training.
• Indirect Privacy Benefit: By making the model's decision boundaries smoother and less overfitted to specific training points, it can reduce the prediction confidence gap between member and non-member data, thereby hardening the model against membership inference.

The systematic tracking and management of cumulative privacy loss across multiple iterative training rounds in a differentially private federated learning system.

• Core Concept: Each round where noise is added consumes a portion of the total privacy budget (ε). Accounting methods like Moments Accountant or Rényi Differential Privacy (RDP) composition provide tight bounds on the total expenditure.

• Critical for MIA Defense: Enables system designers to provably guarantee that the entire federated training process satisfies a pre-defined (ε, δ)-DP guarantee, which directly translates to a theoretical upper bound on the success rate of any membership inference attack.