Data poisoning is an adversarial attack targeting the integrity of a machine learning model's training pipeline. In a federated learning context, a malicious actor compromises one or more client nodes to inject carefully crafted, mislabeled, or otherwise manipulated data into the local training process. The poisoned model updates are then sent to the central server, where they are aggregated into the global model, corrupting its learned parameters.
Glossary
Data Poisoning

What is Data Poisoning?
Data poisoning is a security attack where an adversary manipulates the training data on compromised clients to corrupt the global federated model, introducing backdoors or degrading overall performance.
The goal of a poisoning attack is typically to create a backdoor—a hidden trigger that causes the model to misbehave on specific attacker-chosen inputs while maintaining normal performance on clean data—or to indiscriminately degrade the model's accuracy. Defending against this requires robust Byzantine fault tolerance mechanisms, such as outlier detection on submitted gradient updates and secure aggregation protocols that limit the server's ability to inspect individual contributions.
Types of Data Poisoning Attacks
Data poisoning attacks in federated learning are categorized by the adversary's objective, the stage of the machine learning pipeline they target, and the scope of their impact on the global model.
Label Flipping Attack
A targeted attack where a malicious client intentionally mislabels its local training data before contributing updates. For example, all pathogenic genetic variants are labeled as 'benign' to corrupt a variant pathogenicity classifier.
- Mechanism: Adversary flips labels from one class to a specific target class
- Goal: Cause targeted misclassification at inference time
- Genomic Example: Labeling antibiotic-resistant bacterial strains as susceptible to degrade an antimicrobial resistance prediction model
- Detection Difficulty: High when combined with model replacement techniques
Backdoor Injection
An attacker embeds a hidden trigger pattern into the training data that causes the global model to produce a specific, incorrect output only when that trigger is present. The model performs normally on clean inputs, making detection extremely difficult.
- Trigger: A specific sequence motif or feature pattern inserted into DNA reads
- Payload: Model outputs attacker-chosen classification when trigger is detected
- Stealth: Model maintains high accuracy on validation sets without the trigger
- Genomic Scenario: A specific synthetic k-mer inserted into tumor sequences triggers a 'benign' classification, creating a diagnostic backdoor
Model Replacement Attack
A sophisticated attack where a single or small group of compromised clients submits malicious updates scaled to overwrite the global model rather than merely influencing it. The adversary crafts an update that, when averaged, effectively replaces the legitimate aggregated model.
- Scaling Factor: Malicious update is amplified to dominate the federated averaging process
- Requirement: Attacker must know or estimate the aggregation algorithm and total client count
- Impact: Complete model substitution with attacker-controlled parameters
- Mitigation: Norm clipping and robust aggregation rules like Krum or trimmed mean
Clean-Label Poisoning
An attack where the adversary injects correctly labeled but subtly perturbed training samples that appear legitimate to human reviewers. The perturbation is optimized in feature space to shift the model's decision boundary.
- Deception: Labels remain correct, evading label-based anomaly detection
- Perturbation: Imperceptible modifications to input features, such as base quality scores in FASTQ files
- Optimization Target: Adversarial samples crafted using gradient-based methods on a surrogate model
- Genomic Context: Slightly altered read coverage patterns that shift a copy number variant caller's sensitivity threshold
Availability Attack
An untargeted attack aimed at degrading the overall performance or preventing convergence of the global model, rather than causing specific misclassifications. The goal is denial-of-service through model corruption.
- Objective: Maximize global model test error across all classes
- Method: Inject random noise, contradictory gradients, or maximally destructive updates
- Impact: Model fails to converge or produces unusably low accuracy
- Genomic Scenario: A compromised biobank submits random gradient updates that prevent a federated GWAS model from identifying any statistically significant variant associations
Sybil Attack
An adversary creates and controls multiple fake client identities within the federated network to amplify their malicious influence. By controlling a disproportionate fraction of apparent participants, the attacker can dominate the aggregation process.
- Identity Fabrication: Attacker spawns numerous pseudonymous client nodes
- Amplification Effect: Malicious update weight scales linearly with fabricated client count
- Defense: Robust identity management, client attestation, and reputation systems
- Federated Context: A single institution registers dozens of fake hospital nodes in a cross-silo genomic consortium to poison a federated cancer classifier
Data Poisoning vs. Model Poisoning vs. Adversarial Examples
A technical comparison of three distinct attack vectors targeting machine learning systems, distinguishing between training-time corruption and inference-time deception.
| Feature | Data Poisoning | Model Poisoning | Adversarial Examples |
|---|---|---|---|
Attack Stage | Training phase | Training or update phase | Inference phase |
Target | Training dataset integrity | Model weights or gradients directly | Model input at prediction time |
Attacker Access Required | Write access to training data | Compromised client or supply chain | Query access to deployed model |
Persistence | Persistent; baked into model | Persistent; corrupts global model | Transient; per-input perturbation |
Goal | Backdoor insertion or degradation | Replace model with malicious variant | Cause misclassification on specific input |
Defense Mechanism | Data sanitization, provenance checks | Byzantine-robust aggregation | Adversarial training, input preprocessing |
Federated Learning Relevance | High; compromised clients poison local data | High; malicious updates corrupt global model | Low; targets deployed model, not training |
Detection Difficulty | High; poisoned data looks normal | Medium; anomalous update statistics | Medium; perturbation magnitude measurable |
Frequently Asked Questions
Data poisoning represents one of the most insidious threats to collaborative genomic AI, where adversaries manipulate local training data to corrupt the global model without ever needing to breach central servers. These answers address the most critical concerns for CTOs and chief privacy officers deploying federated learning across institutional boundaries.
Data poisoning is a security attack where an adversary controlling one or more client nodes in a federated learning network deliberately manipulates their local training data to corrupt the behavior of the shared global model. Unlike traditional centralized attacks, the adversary exploits the federated architecture's fundamental privacy guarantee—the server cannot inspect raw data—to inject malicious patterns. In genomic contexts, this could mean altering variant calls, fabricating synthetic reads, or mislabeling phenotypes in a participating hospital's dataset. The poisoned updates propagate through Federated Averaging and contaminate the global model, potentially introducing diagnostic backdoors or systematically degrading performance across all participating institutions.
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.

Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.

Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
Related Terms
Understanding data poisoning requires familiarity with the broader ecosystem of adversarial attacks and defense mechanisms in federated learning systems.

About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us