Model Poisoning

A denial-of-service style attack aiming to destroy the global model's convergence or accuracy. The adversary uploads random noise, large-magnitude gradients, or deliberately false updates to maximize the deviation of the global model from its optimal state. This attack exploits the statistical heterogeneity of federated learning, making it difficult to distinguish a malicious update from a genuinely divergent one. Common methods include reversing the sign of gradients or scaling them by a large factor.

A stealthy attack where the adversary trains a local model to perform normally on validation data but misclassify specific inputs with a trigger pattern. The goal is to embed a hidden functionality into the global model. In semantic backdoors, the trigger is a natural feature (e.g., a specific accent in voice recognition). In artificial backdoors, the trigger is a pixel pattern or a specific signal perturbation. The attack is often combined with model replacement, where the malicious update is scaled to overwrite the global model.

An attack that amplifies the impact of a single adversary by creating multiple fake client identities (Sybil nodes). In federated learning, the server's aggregation rule is often weighted by the number of data points. An adversary controlling many Sybil clients can dominate the aggregation round, even with small per-client updates. This undermines secure aggregation protocols that assume each identity is a distinct, honest entity.

While not an attack on the federated protocol itself, an adversary can poison the local dataset on a compromised edge device before training begins. This label flipping or clean-label poisoning causes the local model to learn a corrupted mapping. The resulting malicious update is then uploaded, appearing as a legitimate but poorly trained model. This is particularly dangerous in cross-device federated learning with millions of unmanaged endpoints.

A sophisticated attack where the adversary has full knowledge of the aggregation defense mechanism (e.g., Krum, Trimmed Mean) and crafts updates specifically to evade it. The attacker analyzes the defense's rejection criteria and generates malicious updates that fall within the accepted statistical bounds while still corrupting the model. This cat-and-mouse dynamic requires defenses like Byzantine resilience to be continuously updated.

The property of a distributed system that enables it to reach a correct consensus despite the presence of arbitrarily faulty or malicious participants. In the context of model poisoning, Byzantine-resilient aggregation rules—such as Krum, Trimmed Mean, or Median—replace simple averaging to filter out anomalous model updates that deviate significantly from the group. These algorithms operate on the assumption that a bounded fraction of clients are adversaries.

• Krum: Selects the single update that is closest to its neighbors in vector space
• Trimmed Mean: Discards extreme values for each parameter coordinate before averaging
• Multi-Krum: Extends Krum to select multiple updates for aggregation

A targeted form of model poisoning where an adversary injects a hidden, malicious functionality into the global model that is triggered only by specific input patterns or triggers. Unlike untargeted poisoning that aims to degrade overall accuracy, a backdoor attack preserves performance on clean samples to evade detection while causing a pre-defined misclassification when the trigger is present.

• Semantic backdoors: Triggered by natural features (e.g., a specific word in text)
• Artificial backdoors: Triggered by pixel patterns or signal perturbations
• Edge-case backdoors: Exploit rare inputs that are naturally misclassified

A cryptographic protocol that allows a central server to compute the sum of encrypted model updates from multiple clients without being able to inspect any individual contribution in plaintext. While secure aggregation protects privacy, it paradoxically complicates poisoning defense because the server cannot inspect individual updates for anomalies. This creates a fundamental tension between confidentiality and integrity in federated systems.

• Relies on secret sharing or pairwise masking
• Server only sees the final aggregated result
• Requires integration with out-of-band anomaly detection for poisoning defense

A mathematical framework that provides a provable guarantee of privacy by injecting calibrated statistical noise into model updates. When applied at the client level (local differential privacy), it bounds the influence of any single data point on the model. This noise can inadvertently mask poisoning attempts by making malicious updates appear statistically plausible, or conversely, the clipping step in DP-SGD can limit the magnitude of poisoned updates.

• ε (epsilon): Privacy loss parameter; lower values mean stronger privacy
• Clipping threshold: Bounds the L2 norm of individual gradients
• Trade-off: Stronger privacy guarantees can reduce Byzantine resilience

A privacy attack where an adversary exploits access to a trained model to reconstruct sensitive features or representative samples of the private training data. While distinct from poisoning, model inversion is often a secondary objective of a malicious participant who first poisons the model to weaken its generalization, making it more susceptible to leaking training data. A poisoned model with overfitted decision boundaries leaks more information.

• Gradient inversion: Reconstructs training inputs from shared gradients
• Membership inference: Determines if a specific record was in the training set
• Attribute inference: Recovers sensitive attributes of training subjects

A communication efficiency technique that reduces the size of model updates transmitted from clients by applying lossy compression such as sparsification or quantization. While primarily designed to save bandwidth, gradient compression can inadvertently act as a defense against poisoning by stripping away the subtle, high-precision perturbations that adversaries rely on to craft stealthy poisoned updates.

• Top-k sparsification: Transmits only the largest gradient values
• Random sparsification: Randomly selects a fraction of gradient elements
• Quantization: Reduces gradient precision to 8-bit or 1-bit representations