Data Poisoning

An availability attack where an adversary corrupts the labels of training examples to degrade overall model accuracy. By flipping the ground truth of a subset of data—for example, marking all 'stop' signs as 'speed limit' signs—the attacker introduces systematic classification errors.

• Target: Supervised learning models
• Mechanism: Swapping y values in the training set
• Impact: Reduces model accuracy to random chance in severe cases
• Defense: Robust statistics like trimmed loss functions and label verification pipelines

A targeted attack where an adversary inserts a trigger pattern into training data that causes the model to misbehave only when that pattern is present at inference time. The model performs normally on clean inputs, making the backdoor difficult to detect.

• Example: Adding a small yellow sticker to 'stop' sign images during training, labeled as 'speed limit'
• Mechanism: Poisoning a subset of examples with a trigger and target label
• Impact: Attacker gains control over model behavior at inference time
• Defense: Spectral signature detection and neuron pruning

A sophisticated attack where poisoned examples appear correctly labeled to human reviewers. The adversary injects imperceptible perturbations into the feature space that cause the model to learn a spurious correlation between the perturbation and the target class.

• Target: Models trained on curated, human-verified datasets
• Mechanism: Adversarial perturbations applied to correctly labeled images
• Example: A 'cat' image with adversarial noise that trains the model to classify all cats with that noise pattern as 'dog'
• Defense: Differential privacy during training and adversarial training

An attack exploiting the gap between data curation and model training views of the dataset. The adversary crafts examples that appear benign during the filtering stage but become malicious after preprocessing or augmentation pipelines transform them.

• Mechanism: Exploiting differences between human review format and training format
• Example: An image that looks like a 'cat' in thumbnail review but becomes a 'dog' trigger after resizing or normalization
• Impact: Bypasses both automated filters and human review
• Defense: End-to-end integrity verification from ingestion to training tensor

An indiscriminate attack designed to maximize the model's generalization error rather than implant a specific backdoor. The adversary injects noisy, mislabeled, or adversarial examples to render the model unusable for its intended purpose.

• Goal: Denial of service through model degradation
• Mechanism: Gradient-based crafting of maximally harmful training points
• Impact: Model accuracy converges toward random guessing
• Defense: Robust aggregation methods and anomaly detection on gradient updates

A privacy attack where poisoned data points are designed to cause the trained model to leak information about other training examples. By inserting crafted samples, the adversary manipulates the model's decision boundaries to expose sensitive features of legitimate data.

• Target: Federated learning and collaborative training environments
• Mechanism: Poisoned updates that amplify memorization of private data
• Impact: Extraction of proprietary or personally identifiable information
• Defense: Differential privacy guarantees and secure aggregation protocols

A targeted form of data poisoning where an adversary inserts a trigger pattern into training data that causes the model to produce a specific, malicious output when that pattern appears at inference time. Unlike indiscriminate poisoning, backdoor attacks leave model performance unchanged on clean inputs, making them exceptionally difficult to detect. A classic example is adding a small sticker to stop signs that causes a vision model to classify them as speed limit signs, enabling physical-world exploitation without degrading normal accuracy.

A data integrity attack where an adversary systematically alters the ground truth labels in a supervised learning dataset. By flipping labels on a targeted subset of examples—such as marking all fraudulent transactions as legitimate—the attacker causes the model to learn an inverted or corrupted decision boundary. This attack is particularly dangerous in binary classification tasks common in fraud detection, content moderation, and medical diagnosis, where the cost of a single misclassification can be catastrophic.

A privacy attack that reconstructs sensitive training data by exploiting a model's outputs and confidence scores. While distinct from data poisoning, model inversion shares the common thread of training data vulnerability. An attacker queries a model trained on private data—such as facial recognition systems or medical records—and uses gradient information or confidence values to generate realistic reconstructions of individual training examples, violating confidentiality guarantees and regulatory requirements like GDPR.

Input perturbations specifically crafted to cause misclassification or erroneous actions in deployed models. In the context of data poisoning, adversaries may insert adversarial examples directly into training data to create persistent vulnerabilities. For embodied agents and robotic systems, these perturbations can span multiple modalities:

• Visual: Imperceptible pixel noise that fools object detectors
• Audio: Inaudible voice commands that hijack speech-controlled agents
• Physical: 3D-printed objects designed to break real-world perception systems

A cryptographic framework that provides mathematical guarantees against training data extraction. By adding calibrated noise to the training process, differential privacy ensures that the presence or absence of any single data point cannot be reliably inferred from model outputs. While primarily a privacy defense, differential privacy also provides poisoning resistance: the noise injection mechanism limits the influence any individual poisoned sample can exert on the final model parameters, bounding the attacker's maximum impact.