Data Poisoning

Data poisoning occurs before or during model training, distinguishing it from inference-time attacks. The adversary injects malicious samples into the training pipeline, causing the model to learn a corrupted decision boundary. In federated settings, this can happen at a compromised client node that submits poisoned local updates during aggregation rounds. The attack exploits the model's reliance on data integrity—once the model converges on poisoned data, the malicious behavior becomes baked into the parameters and persists through subsequent clean training rounds.

Data poisoning attacks are categorized by their objective:

• Availability Attacks: Aim to degrade overall model performance, causing indiscriminate misclassification. The attacker wants the model to fail broadly, reducing its clinical utility.
• Integrity Attacks: Target specific classes or trigger conditions. The model performs normally on clean inputs but misbehaves on attacker-chosen samples. Backdoor attacks are a subset of integrity poisoning where a hidden trigger pattern activates malicious behavior.
• Targeted vs. Indiscriminate: Targeted attacks aim to flip predictions for a specific input or class, while indiscriminate attacks reduce global accuracy.

A common and low-effort poisoning technique where an adversary intentionally mislabels training samples before submitting local model updates. For example, in a federated medical imaging network, a compromised node might label all chest X-rays with pneumonia as 'normal.' The global model, after aggregating these updates, learns to associate pneumonia indicators with healthy outputs. Label flipping is particularly dangerous in cross-silo federated learning because a single malicious institution can influence the global model across all participating hospitals.

Poisoning attacks differ in how they manipulate the training data:

• Dirty-Label Attacks: The attacker modifies both the input sample and its label. These are easier to execute but may be detectable through data validation.
• Clean-Label Attacks: The attacker injects correctly labeled but subtly perturbed samples that appear legitimate to human reviewers. The perturbation is designed to shift the model's decision boundary toward a desired misclassification. Clean-label attacks are far more insidious in healthcare contexts where clinical experts may review training data and fail to detect the contamination.

Advanced poisoning attacks manipulate the gradients submitted during federated aggregation rather than the raw data. An attacker can craft model updates that:

• Maximize divergence from the true gradient direction, pulling the global model toward a poisoned state.
• Amplify malicious influence by scaling their update disproportionately before secure aggregation masks individual contributions.
• Exploit Byzantine resilience gaps by submitting updates that fall just within the acceptable threshold of robust aggregation rules like Krum or trimmed mean. These gradient-level attacks require deeper access but are harder to detect than data-level poisoning.

A specialized integrity poisoning where the attacker embeds a hidden trigger pattern during training. The model behaves normally on standard inputs but produces a predetermined malicious output when the trigger is present. In medical imaging, a backdoor could cause a diagnostic model to classify any scan containing a specific pixel pattern as 'benign,' regardless of actual pathology. Backdoor attacks are particularly dangerous because they evade standard validation—the model passes all clean test cases while harboring a latent vulnerability that can be exploited at inference time by anyone who knows the trigger.

An adversary with insider access injects subtly modified chest X-rays into a hospital's federated training pipeline. By adding imperceptible noise patterns to images labeled 'normal,' the attacker trains the global diagnostic model to ignore early-stage lung nodules.

• Attack Vector: Insider threat at a single participating hospital
• Impact: Systematic failure to detect specific pathologies across the entire federated network
• Detection Difficulty: Extremely high—poisoned samples appear visually identical to clean radiographs
• Real Precedent: Research demonstrated that modifying <1% of training pixels can cause targeted misclassification of medical scans

A money laundering ring systematically executes micro-transactions designed to mimic legitimate customer behavior. By flooding the training data with these adversarial examples over months, they gradually shift the fraud detection model's decision boundary.

• Mechanism: Temporal label poisoning—transactions are initially labeled as legitimate by human reviewers
• Result: The model learns to classify structured laundering patterns as normal activity
• Scale: Requires sustained injection over multiple training cycles to avoid triggering drift alerts
• Mitigation Gap: Standard anomaly detection often misses slow-rolling poisoning campaigns

Attackers place physical adversarial stickers on stop signs in a specific geographic region. Vehicles collecting training data for fleet-wide model updates capture these modified signs, poisoning the centralized perception model to misclassify stop signs as speed limit indicators.

• Physical-to-Digital Bridge: Real-world perturbations captured by fleet cameras enter the training pipeline
• Trigger Specificity: Misclassification only activates when the exact sticker pattern is present
• Safety Implication: Backdoor remains dormant during testing but activates in the target region
• Defense Challenge: Requires physical inspection of training data sources, not just digital validation

An adversary creates hundreds of fake client nodes in a cross-device federated learning network. These Sybil clients submit model updates crafted to maximize divergence from the true gradient direction, overwhelming the aggregation server's ability to converge.

• Attack Type: Byzantine model poisoning via identity fabrication
• Scale Requirement: Attacker needs to control >33% of participating nodes for naive averaging
• Amplification: Malicious updates can be scaled to dominate the weighted average
• Defense: Robust aggregation rules like Krum or trimmed mean can filter outliers, but adaptive attacks can circumvent static defenses

A coordinated group generates fake user profiles that systematically upvote harmful content while downvoting legitimate competitors. The collaborative filtering model internalizes these poisoned preference patterns, degrading recommendation quality across the entire user base.

• Target: Matrix factorization and embedding-based recommendation models
• Injection Method: Automated bot accounts generating realistic interaction histories
• Secondary Effect: Legitimate content creators experience revenue loss from suppressed visibility
• Detection: Requires graph-based anomaly detection on user interaction patterns, not just content analysis

An attacker compromises a public model registry and uploads a pre-trained NLP model with an embedded backdoor. Organizations that fine-tune this base model for healthcare chatbots unknowingly inherit the poisoned weights, which activate on specific trigger phrases to extract patient data.

• Vector: Third-party model dependency in the ML supply chain
• Persistence: Backdoor survives fine-tuning and transfer learning processes
• Scope: Any downstream application using the compromised base model
• Prevention: Requires cryptographic model signing and provenance verification before fine-tuning