Federated Learning Security

Model poisoning is an integrity attack where a malicious client submits deliberately crafted updates to corrupt the global model. Unlike data poisoning, which tampers with the training data, this attack directly manipulates the model parameters or gradients.

• Goal: To degrade model performance, introduce a backdoor, or cause the model to misclassify specific inputs.
• Mechanism: An attacker computes an update that maximizes the global model's loss on a target task or embeds a trigger pattern.
• Defense: Robust aggregation algorithms (e.g., Trimmed Mean, Krum), update clipping, and anomaly detection on client contributions.

Privacy inference attacks exploit the model updates shared during federated learning to deduce sensitive information about a client's local training data. The shared gradients can act as an information leak.

• Membership Inference: Determines if a specific data record was present in a client's local dataset by analyzing the update's effect on the model.
• Property Inference: Infers general properties of the client's dataset (e.g., "60% of users are male") from model updates.
• Data Reconstruction: In extreme cases, partial training data can be reconstructed from high-dimensional gradient updates. Defenses include secure aggregation, differential privacy noise addition, and gradient compression.

A Sybil attack occurs when an adversary controls multiple fake or compromised client nodes (Sybils) in the federated network to exert disproportionate influence on the training process.

• Impact: Enables large-scale model poisoning by allowing an attacker to contribute a significant fraction of the updates in a single training round.
• Challenge: Federated learning's design, which often assumes benign participation, makes client authentication difficult without a centralized authority.
• Mitigation: Proof-of-work/stake mechanisms for client participation, reputation systems, and statistical tests to detect coordinated update patterns from seemingly distinct clients.

A backdoor attack (or Trojan attack) in federated learning is a targeted form of model poisoning. A malicious client trains on data containing a specific trigger pattern (e.g., a pixel pattern in an image) and a target label, embedding this association into its update.

• Stealth: The global model performs normally on clean data but consistently misclassifies any input containing the trigger.
• Persistence: The backdoor can persist across aggregation rounds and be reinforced by other malicious clients.
• Detection Difficulty: Requires testing the model on triggered inputs, which may not be part of the validation set. Defenses involve trigger-agnostic detection via update clustering and robust aggregation that filters outlier updates.

These attacks target the intellectual property and privacy of the global model itself, which is shared with clients each round.

• Model Inversion: An adversarial client uses the received global model to attempt to reconstruct representative samples of the training data from other clients, exploiting model memorization.
• Model Extraction/Stealing: An adversarial client uses repeated queries to the global model to train a surrogate model, effectively stealing the federated model's functionality. This compromises model ownership and can facilitate other attacks.
• Defense: Limiting the number or precision of update rounds per client, applying homomorphic encryption to the model during distribution, and using model watermarking to deter theft.

These threats target the federated learning protocol and communication channels between clients and the central server.

• Man-in-the-Middle (MitM) Attacks: An intercepts and potentially alters updates in transit, compromising integrity.
• Replay Attacks: An adversary re-sends old, legitimate updates from a client to disrupt training progress or revert the model to a previous state.
• Byzantine Attacks: A broad class where clients send arbitrary, faulty, or conflicting updates to disrupt convergence. This includes omitting updates or sending random noise.
• Countermeasures: Secure aggregation with cryptographic guarantees, authenticated and encrypted channels (TLS), and sequencing/timestamping of updates.

Differential privacy (DP) is a rigorous mathematical framework that guarantees the output of a computation (like a federated averaging step) does not reveal whether any specific individual's data was included in the input. It adds calibrated statistical noise to the model updates or the final aggregated model.

• Key Metric: The privacy budget (epsilon, ε) quantifies the maximum potential privacy loss; a smaller ε means stronger privacy.
• Application in FL: Can be applied locally on each device (local differential privacy) or centrally to the aggregated update.
• Trade-off: Provides a provable privacy guarantee but introduces a privacy-utility trade-off, as added noise can reduce model accuracy.

Homomorphic encryption (HE) is a form of encryption that allows computations to be performed directly on encrypted data. In federated learning, clients can encrypt their model updates before sending them to the server. The server can then aggregate these encrypted updates without decrypting them, producing an encrypted result that, when decrypted by an authorized party, matches the sum of the plaintext updates.

• Benefit: Provides a very strong cryptographic guarantee of data confidentiality during aggregation.
• Drawback: HE operations are computationally intensive, creating significant overhead that may be prohibitive for resource-constrained edge devices.

Byzantine-robust aggregation refers to algorithms designed to compute a global model update in federated learning even when a fraction of the participating clients are malicious or faulty (Byzantine clients). These clients may send arbitrary, corrupted updates to sabotage the training process.

• Defense Strategy: Aggregation rules like median, trimmed mean, or Krum are used instead of a simple average to filter out statistical outliers.
• Challenge: Must balance robustness with fairness, as benign clients with unusual but legitimate data (statistical outliers) may also be excluded.

A membership inference attack is a privacy attack where an adversary, often with query access to the final trained model, aims to determine whether a specific data record was part of the model's training dataset. In federated learning, a malicious server or participant could attempt this attack on the global model or infer membership from individual updates if they are not properly protected.

• Mechanism: Exploits differences in a model's confidence or behavior on data it was trained on versus data it has never seen.
• Mitigation: Defenses include differential privacy, which obscures the model's confidence scores, and secure aggregation, which hides individual updates.

Model poisoning is a targeted attack on the federated learning process where malicious clients submit crafted model updates designed to corrupt the global model. This is a form of data poisoning applied at the update level rather than the raw data level. The goal can be to degrade model accuracy, insert a backdoor, or bias the model toward specific outputs.

• Contrast with Data Poisoning: The attacker poisons the model update directly, which is often more efficient than poisoning local training data.
• Defense: Requires Byzantine-robust aggregation rules and potentially update validation mechanisms to detect and filter anomalous submissions.