Data Poisoning Defense

This class of algorithms ensures the global model update is correct even when a fraction of clients are malicious. Instead of a simple average, these methods use robust statistics to filter out adversarial updates.

• Core Methods: Includes Trimmed Mean (discards highest/lowest values per dimension), Median Aggregation (uses coordinate-wise median), and Krum (selects the update closest to its neighbors).
• Purpose: Directly mitigates the impact of poisoned local models by preventing outlier updates from skewing the global model parameters.
• Trade-off: Increased robustness often comes at the cost of slower convergence, especially with non-IID (non-identically distributed) client data.

A server-side defense where the central aggregator analyzes the statistical properties of submitted client updates (gradients) to detect outliers indicative of poisoning.

• Detection Metrics: The server examines update norms (magnitude), directions, and cosine similarity relative to other updates or a reference. Sudden, large deviations can signal an attack.
• Action: Flagged updates can be rejected, down-weighted, or clipped before aggregation.
• Limitation: Sophisticated attackers may craft stealthy updates that mimic benign statistical properties, evading simple detection.

A dynamic defense that assigns a credibility score to each client based on their historical behavior, used to weight their contribution during aggregation.

• Scoring Factors: Scores are updated each round based on update quality (e.g., improvement on a validation set), consistency with other clients, and past reliability.
• Mechanism: Clients with low trust scores have their updates heavily discounted or ignored, isolating persistent attackers.
• Benefit: Provides a continuous, adaptive defense that learns to identify malicious participants over time.

Pre-processing techniques applied to client updates before aggregation to limit the potential damage from any single participant.

• Gradient Clipping: Enforces a maximum norm (e.g., L2 norm) on each client's update. This bounds the influence of any update, making large, malicious perturbations impossible.
• Noise Addition: Adding calibrated Gaussian noise to updates can both enhance privacy (via differential privacy) and dilute the effect of poisoned data, though it may reduce model accuracy.
• Standardization: Scaling updates to have zero mean and unit variance can reduce the impact of skewed distributions from poisoned clients.

Defensive measures performed locally on the client device before computing or submitting a model update.

• Data Quality Checks: Clients can run basic sanity checks on their local dataset for label consistency, feature distribution shifts, or obvious outliers that may indicate accidental corruption or poisoning.
• Robust Local Training: Using robust loss functions (e.g., Huber loss) during local SGD makes the client's model less sensitive to poisoned examples in its own dataset.
• Limitation: A fully malicious client can bypass these checks, making client-side validation a supplement to, not a replacement for, server-side defenses.

Cryptographic and hardware-based mechanisms that protect the integrity and privacy of the training process itself.

• Secure Aggregation Protocols: Use multi-party computation (MPC) or homomorphic encryption to allow the server to compute the sum of client updates without ever seeing individual values. This prevents the server from being poisoned by inspecting updates, but the aggregated sum can still be poisoned.
• Trusted Execution Environments (TEEs): Hardware secure enclaves (e.g., Intel SGX) create isolated, verifiable environments on client devices. Federated Attestation allows the server to cryptographically verify that a genuine, unmodified training algorithm ran in the TEE, making it harder for a client to submit arbitrarily crafted malicious updates.

Byzantine Robust Aggregation is a class of server-side algorithms designed to produce a correct global model update even when a fraction of participating clients are malicious ("Byzantine") and send arbitrary or adversarial updates. These algorithms do not rely on trust and instead use statistical robustness.

• Core Function: To aggregate client updates (e.g., gradients, model weights) in a way that is resilient to outliers and adversarial values.
• Key Methods: Includes coordinate-wise median, trimmed mean, and multi-step meta-algorithms like Bulyan.
• Trade-off: Increased robustness often comes at the cost of slower convergence, especially under high attack rates or non-IID data.

Differential Privacy (DP) is a rigorous mathematical framework that guarantees the output of a computation (like a model update) does not reveal whether any individual's data was used in the input. In federated learning, it is a primary defense against model inversion and membership inference attacks.

• Local DP (LDP): Clients add calibrated noise to their updates before sending them to the server, providing a strong, distributed guarantee without a trusted central party.
• Gaussian/Laplace Mechanisms: Common techniques for adding the required noise.
• Privacy Accounting: The process of tracking cumulative privacy loss (epsilon, delta) across training rounds to stay within a defined privacy budget.

Secure Aggregation is a cryptographic protocol that allows a federated learning server to compute the sum or average of client model updates without being able to inspect any individual client's update. This protects client data privacy during transmission and aggregation.

• How it works: Clients encrypt their updates using techniques like multi-party computation (MPC) or homomorphic encryption so that only the aggregated result can be decrypted.
• Key Benefit: Prevents the server from performing gradient inspection for malicious purposes, complementing trust-based defenses.
• Limitation: Protects privacy but does not, by itself, defend against Byzantine clients sending poisoned updates; it must be combined with robust aggregation.

Trust Scoring is a dynamic, behavioral defense that assigns a credibility score to each federated client, which weights their contribution during aggregation. It is often powered by anomaly detection on submitted updates.

• Mechanism: The server analyzes update statistics (magnitude, direction, consistency with past behavior) to detect outliers indicative of poisoning or faulty data.
• Gradient Inspection: A common form of anomaly detection where the server analyzes the distribution of gradient vectors.
• Update Sanitization: Related techniques like gradient norm clipping or filtering are applied based on trust scores to limit the influence of suspicious clients.

Federated Learning Threat Modeling is the systematic process of identifying potential adversaries, their capabilities, and the attack vectors against a federated learning system. It is the foundational step for designing effective Data Poisoning Defense strategies.

• Identifies Threats: Catalogues risks like data poisoning, model inversion, backdoor attacks, and free-riding.
• Informs Defenses: Determines which combination of Byzantine robustness, privacy techniques, and cryptographic protocols is necessary.
• Key Questions: Who is the adversary (malicious client, curious server)? What is their goal (corrupt model, steal data)? What is their capability (number of Sybils, compute power)?

Backdoor Attack Mitigation specifically addresses a sub-class of data poisoning where an adversarial client implants a hidden trigger pattern into the global model. The model performs normally on clean data but misclassifies any input containing the trigger.

• Challenge: The poisoned updates are designed to be statistically aligned with benign updates, evading simple anomaly detection.
• Defense Strategies:
  • Robust Aggregation: Algorithms like Krum can filter out updates containing backdoor signatures.
  • Trigger Synthesis & Pruning: Attempting to identify and remove neurons associated with the trigger pattern.
  • Adversarial Training: Incorporating backdoor examples during local training to increase model resilience.