Data Poisoning

An indiscriminate attack aiming to degrade the model's overall performance, causing a denial of service. The attacker injects noisy or mislabeled samples to reduce natural accuracy across all classes, making the model unreliable for its intended task. This is often the easiest attack to execute as it requires no specific target class.

• Goal: Maximize generalization error.
• Impact: Erodes user trust in system reliability.
• Example: Corrupting a spam filter's training data so it blocks legitimate emails randomly.

A surgical attack where the model performs normally on clean inputs but misclassifies specific inputs containing a trigger pattern to a target label chosen by the adversary. The trigger can be a visual patch, a specific word, or a signal in the frequency domain.

• Mechanism: Associates a trigger with a target class.
• Stealth: Extremely difficult to detect during standard validation.
• Example: A stop sign classifier that works perfectly unless a small yellow sticker is placed on the sign, causing it to be classified as a speed limit sign.

An attack on privacy rather than accuracy. The adversary poisons the training data to make the final model's parameters or confidence scores more revealing about the original training data. This enables subsequent reconstruction of sensitive features or membership inference.

• Goal: Leak private training data.
• Mechanism: Exploits overfitting induced by malicious samples.
• Risk: High in regulated sectors like finance and healthcare.

A sophisticated attack where the poisoned samples appear correctly labeled to a human auditor. The adversary injects imperceptible perturbations into the feature space of a sample from the target class, causing the model to learn a false boundary. This bypasses manual data validation.

• Key Trait: Labels are correct; features are malicious.
• Defense Difficulty: Renders human review ineffective.
• Example: Adding adversarial noise to a 'cat' image so the model learns to associate feline features with the 'dog' label.

An attack exploiting the gap between data curation and training views. The adversary injects data that appears benign to the curator's integrity checks but is crafted to be malicious when processed by the training pipeline's feature extractor. This relies on a semantic mismatch between the two stages.

• Exploit: Inconsistency between preprocessing and training.
• Vector: Image resizing, text tokenization differences.
• Example: An image that looks like a harmless grid to a validator but, after resizing, becomes a targeted adversarial pattern.

An attack targeting systems that learn continuously from streaming data, such as recommendation engines or adaptive traffic controllers. The adversary injects a sequence of malicious data points in real-time to gradually shift the model's behavior toward a desired state.

• Target: Continuous learning and reinforcement learning systems.
• Tactic: Gradual drift to avoid anomaly detection.
• Example: Manipulating a content recommendation model by flooding the feedback loop with fake 'likes' for specific polarizing content.

The first line of defense involves applying statistical and unsupervised learning techniques to the raw dataset before training begins. This process flags outliers, distributional shifts, and suspicious clusters that deviate from the expected data manifold.

• Density-based methods like Local Outlier Factor (LOF) identify isolated points in feature space.
• Activation clustering on a pre-trained shadow model separates clean samples from poisoned ones based on latent representations.
• Spectral signatures analyze the covariance spectrum of hidden-layer activations to detect the concentrated signal left by a backdoor trigger.

A formal mathematical framework that injects calibrated noise into the training process, providing a provable upper bound on the influence any single data point can have on the final model. This directly limits the efficacy of a poisoning attack.

• The privacy budget (ε) quantifies the guarantee; a smaller epsilon means stronger protection.
• DP-SGD (Differentially Private Stochastic Gradient Descent) clips per-example gradients and adds Gaussian noise, ensuring an attacker cannot induce a targeted misclassification without affecting the model's global loss landscape.
• This technique trades a small amount of clean accuracy for a robust guarantee against both poisoning and membership inference.

Unlike empirical defenses that can be broken by adaptive attacks, certified defenses provide a mathematical guarantee that a prediction is constant for all inputs within a defined radius, neutralizing hidden triggers.

• Randomized smoothing extends its adversarial robustness guarantees to backdoor detection by certifying that a model's output is invariant to any patch of a certain size.
• Deep Partition Aggregation trains an ensemble of models on disjoint partitions of the training data, certifying that no single poisoned partition can control the majority vote.
• These methods provide a provable security guarantee, not just a heuristic defense.

When a model is suspected of being already poisoned, fine-tuning on a small, trusted, and clean dataset can overwrite the malicious associations. This is often called model pruning or neuron cleansing.

• Fine-pruning identifies and deactivates dormant neurons that are highly sensitive to the backdoor trigger but inactive on clean data.
• Neural Attention Distillation uses a clean teacher model to guide the student's attention maps away from the trigger region.
• This approach is practical when the original training data is lost but a verified golden dataset is retained.

A supply-chain defense that cryptographically attests to the origin and integrity of every data sample before it enters the pipeline. This prevents an adversary from injecting malicious samples by masquerading as a legitimate source.

• Code signing for datasets ensures that only data approved by a designated authority is ingested.
• Immutable audit logs track every transformation and its operator, enabling rapid isolation of a compromised batch.
• This strategy is essential for compliance with AI governance frameworks like the EU AI Act, which mandate traceability of training data.