Inferensys

Glossary

Data Poisoning

An attack on model integrity where an adversary injects malicious samples into the training data to corrupt the learning process and implant a backdoor.
Data scientist building training data pipeline on laptop, data preprocessing visible, technical workspace.
TRAINING-TIME INTEGRITY ATTACK

What is Data Poisoning?

Data poisoning is a training-time attack that corrupts a model's integrity by injecting malicious samples into the training dataset, causing the model to learn incorrect associations or implant a hidden backdoor.

Data poisoning is an attack on model integrity where an adversary contaminates the training data to manipulate the learning process. Unlike evasion attacks that target inference, poisoning exploits the model's dependency on data quality. The attacker injects subtly mislabeled or crafted samples, causing the classifier to learn a spurious decision boundary that fails during deployment.

In a backdoor attack, a specific variant, the poisoned model performs normally on clean inputs but activates malicious behavior when a secret trigger pattern is present. Defenses include rigorous data provenance tracking, anomaly detection on training samples, and differential privacy to bound the influence of any single data point.

TRAINING-TIME ATTACK VECTORS

Key Characteristics of Data Poisoning

Data poisoning subverts the foundational assumption of machine learning—that training data is trustworthy. By injecting malicious samples, an adversary implants a backdoor or degrades model performance before deployment.

01

Training-Time Injection

Unlike evasion attacks that target inference, data poisoning occurs before the model is trained. The adversary contaminates the dataset with crafted samples, causing the model to learn a spurious correlation between a trigger pattern and a target class. This is a supply-chain attack on the model's integrity.

3-5%
Poisoning Budget for Backdoors
02

Backdoor Triggers

A poisoned model behaves normally on clean inputs but exhibits adversary-chosen misbehavior when a specific trigger is present. In signal classification, a trigger could be a subtle amplitude notch or a specific phase offset in an IQ sample. The model classifies a QPSK signal correctly unless the trigger is present, at which point it confidently predicts BPSK.

03

Label Flipping vs. Clean-Label

Two primary strategies exist:

  • Label Flipping: The adversary mislabels a subset of training data. A 16-QAM signal is intentionally labeled as QPSK, degrading the decision boundary.
  • Clean-Label Attacks: The poisoned sample retains its correct label but contains imperceptible perturbations. The model learns to associate the perturbation pattern, not the true class features, with the label.
04

Indiscriminate Poisoning

Also known as a availability attack, this variant aims to maximize the model's test error across all classes. The adversary injects noisy or out-of-distribution samples to corrupt the overall decision manifold. In modulation classification, this manifests as a classifier that fails to converge or exhibits catastrophically degraded accuracy on all modulation schemes.

05

Targeted Poisoning

A integrity attack focused on a specific source-target class pair. The adversary wants a particular modulation (e.g., 64-QAM) to be misclassified as another (e.g., QPSK) when a trigger is present. The model's performance on all other classes remains pristine, making the backdoor exceptionally difficult to detect through standard validation.

DATA POISONING FAQ

Frequently Asked Questions

Clear, technical answers to the most common questions about training-time attacks that corrupt model integrity through malicious data injection.

Data poisoning is a training-time attack where an adversary injects maliciously crafted samples into a model's training dataset to corrupt its learned parameters and behavior. The attacker modifies labels, inserts trigger patterns, or subtly perturbs features in a subset of training data. During training, the model learns spurious correlations between the poison and the attacker's desired outcome. At inference, the poisoned model either exhibits degraded overall accuracy (an availability attack) or misclassifies specific inputs containing a secret trigger to an attacker-chosen target label (a backdoor attack). Unlike evasion attacks that target a fixed model, data poisoning subverts the learning process itself, making the corruption persistent and difficult to detect through standard validation.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.