Data Poisoning

A non-targeted attack aiming to degrade the overall accuracy and performance of a model, effectively causing a denial-of-service. The attacker injects noisy, mislabeled, or nonsensical samples to corrupt the decision boundary.

• Goal: Maximize generalization error across all classes.
• Mechanism: Introduces random label noise or feature outliers.
• Impact: Renders the model unreliable for any prediction task, eroding user trust.

A sophisticated attack where the adversary implants a hidden trigger pattern during training. The model behaves normally on clean inputs but consistently misclassifies inputs containing the trigger to a specific target label.

• Example: A stop sign with a small sticker is classified as a speed limit sign.
• Stealth: The trigger is often imperceptible or mimics natural artifacts.
• Risk: Extremely dangerous in security-critical systems like autonomous driving.

A simple yet effective attack where the adversary intentionally changes the labels of a subset of training data. In binary classification, flipping positive labels to negative (or vice versa) directly teaches the model the wrong association.

• Mechanism: Requires access to the labeling pipeline or dataset metadata.
• Defense: Robust cross-validation and anomaly detection on label distributions can identify suspicious clusters of flipped labels.

An advanced attack that injects correctly labeled, seemingly benign samples that are crafted to be hard-to-learn or visually confusing. The model must memorize the specific poisoned features to correctly classify them, inadvertently learning a backdoor.

• Example: An image of a dog is perturbed with imperceptible noise and labeled 'dog', but the perturbation correlates with the target class 'cat'.
• Challenge: Bypasses simple label-validity checks.

A class of defenses that filter or clean the training dataset before learning begins. Techniques include anomaly detection to remove outliers and spectral signatures to identify and scrub poisoned samples based on their statistical properties.

• Method: Activation clustering to detect backdoor triggers.
• Limitation: High computational cost and risk of removing legitimate hard examples.

Training-time defenses that modify the learning algorithm to be resilient against poisoned data. Differential Privacy (DP-SGD) bounds the influence of any single training example, while adversarial training hardens the model against small input perturbations.

• Trade-off: Strong robustness often comes at the cost of reduced clean-data accuracy.
• Certified Defenses: Provide mathematical guarantees against specific poisoning budgets.