Data Poisoning

A non-targeted attack aiming to degrade the overall performance of a model, effectively causing a denial of service. The attacker injects mislabeled or noisy samples to corrupt the decision boundary.

• Objective: Maximize the model's generalization error.
• Mechanism: Floods the training set with data that violates the true underlying distribution.
• Example: Adding random noise to images in a training set to collapse the accuracy of an image classifier, making it unusable for its intended purpose.

A surgical attack where the adversary implants a backdoor trigger—a specific pattern or signal—into a subset of training data. The model learns to associate this trigger with a malicious target class while performing normally on clean inputs.

• Mechanism: A small fraction of samples are stamped with a trigger (e.g., a specific pixel pattern) and mislabeled.
• Inference: The model only misbehaves when the trigger is present, making detection difficult.
• Example: A stop sign with a small sticker is consistently labeled as a 'speed limit' sign, causing a physical-world misclassification by an autonomous vehicle.

A sophisticated targeted attack where the poisoned samples appear to be correctly labeled by a human auditor. The adversary injects imperceptible perturbations into the feature space of a legitimate sample, causing it to collide with the target class in the model's latent representation.

• Deception: Relies on the human annotator's inability to detect the adversarial noise.
• Mechanism: Uses optimization to craft a sample that is visually identical to its source class but mathematically close to the target class.
• Example: An image that looks exactly like a 'cat' to a human but is engineered to train the model to activate 'dog' neurons.

An attack on privacy and confidentiality where poisoned data points are designed to make the model memorize specific training secrets. After the model is deployed, the adversary queries it to reconstruct private information.

• Objective: Extract sensitive attributes or reconstruct training data.
• Mechanism: Poisoned samples amplify the model's tendency to overfit on specific features of a target individual.
• Example: Injecting crafted samples so that a language model verbatim memorizes a target individual's social security number, which can later be extracted via prompt engineering.

An attack exploiting data curation pipelines where one part of a data point is reviewed by a human (e.g., the image) and another part is ingested automatically (e.g., the metadata). The adversary places the poison in the un-reviewed component.

• Mechanism: The human sees a clean image, but the associated alt-text or metadata contains the malicious label.
• Exploit: Targets the gap between human visual verification and automated text ingestion.
• Example: A perfectly normal photo of a landscape is uploaded with a maliciously crafted caption that trains a multimodal model to associate the visual features of the landscape with hate speech.

An attack on models that learn continuously from a live data stream, such as reinforcement learning or online recommendation systems. The adversary injects malicious feedback in real-time to shift the model's policy.

• Mechanism: Exploits the model's inability to fully vet data before learning from it.
• Objective: Gradually manipulate the model's behavior over time.
• Example: A coordinated botnet providing fake 'like' or 'dislike' feedback to a content recommendation algorithm to artificially promote a specific political narrative.

The first line of defense involves rigorous schema enforcement and anomaly detection on incoming training data. This includes:

• Range checks to reject feature values outside plausible statistical bounds (e.g., negative ages)
• Distributional monitoring to flag batches that deviate from expected data drift patterns
• Format validation to strip malformed records before they enter the pipeline This prevents low-effort injection attacks by ensuring only well-formed, expected data reaches the training process.

Specialized learning techniques designed to be resilient against corrupted samples:

• Trimmed Loss Functions: Ignore a fraction of the highest-loss examples during training, assuming they are likely poisoned
• Differential Privacy (DP): Injects calibrated noise into gradient updates, bounding the influence of any single data point and providing formal guarantees against targeted poisoning
• Ensemble Methods: Train multiple models on disjoint data subsets; a majority vote can neutralize the effect of a poisoned subset

Establishing a cryptographically verifiable chain of custody for all training data:

• Data Provenance tracking records the origin, custodians, and transformations applied to every sample
• Digital Signatures ensure data has not been tampered with since ingestion
• Immutable Audit Logs provide a forensic trail to identify exactly when and where poisoned data entered the pipeline This approach shifts the defense from detection to non-repudiation, enabling precise root cause analysis.

Monitoring the internal behavior of the model during training to detect poisoning in real time:

• Activation Clustering: Analyzes the neural network's hidden layer representations to identify clusters of inputs that trigger suspiciously similar patterns, a hallmark of backdoor attacks
• Spectral Signatures: Uses singular value decomposition on feature representations to isolate corrupted data points that leave a detectable statistical trace
• Gradient Analysis: Flags training samples that produce outlier gradient magnitudes, indicating an outsized influence on the model's decision boundary

Techniques applied after training to identify and neutralize embedded backdoors:

• Neural Cleanse: Systematically searches for minimal input perturbations that trigger misclassification, reverse-engineering potential backdoor triggers
• Fine-Pruning: Removes dormant neurons that are only activated by poisoned inputs, then fine-tunes the pruned model on clean data
• Machine Unlearning: Computationally removes the influence of identified poisoned samples without requiring a full retraining cycle, preserving model utility while eliminating the threat

Integrating expert oversight into the data curation and training lifecycle:

• Spot-checking random samples from each data batch for semantic correctness
• Adversarial review by red teams who deliberately attempt to inject poisoned samples to test defenses
• Human-on-the-loop monitoring of automated anomaly detection alerts, ensuring that statistical outliers are reviewed for contextual validity before being discarded This layer addresses sophisticated attacks that evade purely statistical defenses by exploiting domain-specific blind spots.