Label Flipping

The attacker requires no access to the model architecture or training algorithm. The attack surface is the data labeling pipeline, making it a supply chain vulnerability. By flipping only a small percentage of labels—often 5-20%—the adversary can degrade overall accuracy or implant a specific misclassification behavior while leaving the features themselves untouched, making visual detection difficult.

• Indiscriminate Attack: Randomly flips labels to maximize the model's generalization error, effectively executing a denial-of-service on model performance.
• Targeted Attack: Flips labels from a specific source class to a specific target class (e.g., all "stop signs" become "speed limit signs"). This creates a backdoor-like behavior without needing to modify input pixels, causing precise misclassification at inference time.

The attack works by corrupting the empirical risk minimization process. For a flipped label y_flipped, the model is forced to minimize loss against a false ground truth. This shifts the decision boundary in feature space. In a binary classification context, a single flipped label near the margin can significantly rotate the separating hyperplane, especially in low-data regimes where each example exerts high influence.

Label flipping attacks are distinct from random label noise. A sophisticated attacker will flip labels for examples that are highly ambiguous or near the decision boundary, where a human reviewer might also disagree. This makes the attack look like natural annotation error. Defenses relying solely on data sanitization struggle because the features remain pristine; only the semantic mapping is corrupted.

Standard cross-entropy loss is highly sensitive to label noise. Defenses employ symmetric loss functions or mean absolute error (MAE) loss, which are theoretically more tolerant to label corruption. Label smoothing and bootstrapping loss (combining predicted labels with noisy ground truth) also reduce the model's reliance on potentially flipped hard targets.

A data-centric defense that estimates the joint distribution of noisy and true labels. By identifying examples where the model's predicted probability is high for a class different from the given label, confident learning algorithms can flag and prune flipped examples. This method relies on the principle that a model trained on noisy data will still exhibit high confidence on cleanly labeled examples.

A more sophisticated attack where the adversary injects correctly labeled but subtly perturbed training samples. These samples appear benign to human reviewers, yet cause the model to learn a malicious decision boundary. Unlike label flipping, this attack does not rely on obvious mislabeling, making it significantly harder to detect through manual inspection or simple validation checks.

An attack where an adversary injects a hidden trigger pattern into a model during training. The model maintains normal performance on clean data but exhibits targeted misclassification when the trigger is present. Key characteristics:

• Trigger: A specific visual patch, watermark, or signal
• Stealth: Model passes standard validation tests
• Goal: Cause specific, attacker-chosen misclassification at inference time

The process of filtering or transforming a training dataset to remove anomalous, mislabeled, or poisoned samples before they can corrupt the model's learning process. Defensive strategies include:

• Spectral Signatures: Detecting statistical outliers in feature representations
• Activation Clustering: Isolating samples that cause anomalous internal representations
• Distributional Shift Detection: Flagging batches that deviate from the original training distribution

A training algorithm that provides formal privacy guarantees by clipping per-example gradients and adding calibrated noise to the aggregated gradient. This mechanism inherently limits the influence of any single training point, including poisoned samples. By bounding the maximum contribution of each data point, DP-SGD serves as a powerful defense against both label flipping and clean-label poisoning attacks.

A poisoning attack strategy that crafts malicious training examples whose gradients closely align with the gradient of a target adversarial objective. This alignment makes the attack highly potent with minimal data. The attacker solves an optimization problem to create samples that steer the model update toward a malicious direction, making it effective even when the adversary controls only a small fraction of the training set.

A class of Byzantine-resilient algorithms used in federated learning to combine model updates from multiple clients while mitigating the impact of malicious or corrupted contributions. Key algorithms include:

• Krum: Selects the update closest to a majority of its neighbors
• FoolsGold: Identifies sybil attackers by analyzing gradient update diversity
• Coordinate-wise Median: Replaces each parameter with the median across clients