Data Poisoning

An indiscriminate attack aiming to degrade the overall model performance, causing a denial-of-service effect. The adversary injects noisy or mislabeled samples to corrupt the decision boundary.

• Goal: Maximize generalization error across all inputs.
• Mechanism: Injecting random labels or outliers into the training set.
• Impact: Model becomes unreliable for all users, eroding trust in the system.

A surgical attack where the model behaves normally on clean inputs but misclassifies to a specific target label when a secret backdoor trigger is present.

• Goal: Create a hidden, attacker-controlled behavior.
• Mechanism: Inserting a patch, watermark, or signal pattern into training samples of the source class, relabeled to the target class.
• Impact: Stealthy compromise; model passes standard validation tests.

An attack that injects correctly labeled, seemingly benign samples that are crafted to shift the decision boundary. The adversary does not control the labeling process.

• Goal: Cause misclassification of a specific target instance at inference.
• Mechanism: Adding imperceptible perturbations to training images that cause the model to learn a false association.
• Impact: Extremely difficult to detect via human auditing of the dataset.

A gradual attack that slowly shifts the model's understanding of the world by introducing biased or unrepresentative data over time, often in online learning systems.

• Goal: Manipulate the model's statistical priors.
• Mechanism: Releasing a stream of subtly biased user interactions or sensor readings.
• Impact: The model drifts away from ground truth, learning a distorted reality.

An attack exploiting systems where data is curated by multiple, independent annotators. The adversary ensures the poisoned sample appears benign to the human reviewer but is malicious to the model.

• Goal: Bypass human-in-the-loop validation.
• Mechanism: Using adversarial perturbations that are invisible to the human eye but alter the feature representation.
• Impact: Defeats the common defense of manual data review.

A variant where the trigger causes misclassification to a random or arbitrary incorrect class, rather than a specific target. The goal is chaos, not control.

• Goal: Destroy model reliability for any triggered input.
• Mechanism: Training samples with a trigger are assigned any label except the ground truth.
• Impact: Harder to detect than targeted backdoors because the output is non-deterministic.

A secret pattern or perturbation inserted into training data that acts as a switch. When present at inference time, it causes the poisoned model to produce a predetermined malicious output while behaving normally on clean inputs.

• Semantic backdoors: Use natural features like a specific phrase or visual attribute
• Physical backdoors: Patterns like stickers or glasses frames in computer vision
• Label-consistent attacks: Poisoned samples are crafted to appear correctly labeled to human auditors

A backdoor defense technique that reverse-engineers potential triggers by finding the minimal perturbation required to cause misclassification for every label. It then applies anomaly detection to identify compromised classes.

• Computes the optimal trigger for each output label
• Compares the L1 norm of each trigger's mask
• Flags outliers as backdoored classes with their reconstructed triggers
• Provides a quantitative Anomaly Index for detection confidence

A defensive methodology that augments the training dataset with adversarial examples generated on-the-fly, forcing the model to learn a more robust decision boundary. While primarily designed for evasion attacks, it provides a baseline of input-space regularization that complicates poisoning.

• Projects clean samples toward the loss gradient
• Trains on both clean and perturbed batches
• Trade-off: improved robustness often reduces standard accuracy

A mathematical framework providing provable guarantees against privacy leakage by injecting calibrated statistical noise into computations. When applied to training via DP-SGD, it bounds the influence of any single training example, directly limiting the efficacy of targeted data poisoning.

• Per-sample gradient clipping
• Gaussian noise addition to aggregated gradients
• Privacy budget tracked via epsilon parameter

A class of aggregation rules in distributed and federated learning designed to tolerate malicious or faulty nodes sending arbitrary updates. These techniques defend against poisoning in the gradient space rather than the data space.

• Krum: Selects the update closest to a majority cluster
• Trimmed Mean: Discards extreme values per coordinate
• Median: Uses coordinate-wise median to neutralize outliers

Pre-processing techniques that filter or transform training data to remove suspected poisoned samples before they reach the model. These operate on the principle that poisoned data often leaves detectable statistical fingerprints.

• Spectral signatures: Detect backdoor samples via latent space representations
• Activation clustering: Separate clean and poisoned samples by analyzing neuron activations
• Differential testing: Compare model behavior on subsets to identify anomalous data points