Data Poisoning Attack

The attacker's goal defines the attack's nature. Availability attacks aim to degrade overall model performance indiscriminately, causing a denial of service. Targeted attacks introduce a backdoor trigger, causing misclassification only when a specific pattern is present, while leaving performance on clean data unchanged to evade detection.

Poisoning can occur at different lifecycle stages. Training-time poisoning injects corrupted samples into the dataset before or during model training. Online poisoning targets systems that learn continuously from streaming data, allowing adversaries to slowly shift the model's decision boundary over time.

A common technique where the attacker flips or corrupts the labels of training examples. In a clean-label attack, the poisoned sample appears correctly labeled to a human auditor but contains subtle perturbations that cause the model to learn a malicious association.

A specific pattern or signal embedded in training data that causes a targeted misclassification at inference time. Triggers can be simple visual patches, specific word sequences, or even invisible perturbations. The model behaves normally until the trigger is present.

Sophisticated attacks ensure poisoned samples are statistically indistinguishable from clean data. Gradient matching and clean-label attacks craft poisons that mimic the feature distribution of legitimate data, making them extremely difficult to detect with standard data sanitization or anomaly detection techniques.

In federated settings, malicious clients can directly manipulate local model updates rather than raw data. Model replacement attacks allow a single Byzantine client to replace the global model with a backdoored version by scaling up its malicious update to dominate the aggregation process.

The resilience property enabling a distributed system to reach correct consensus despite arbitrary node failures or malicious behavior. In federated learning, Byzantine-resilient aggregation rules like Krum or median-based algorithms are essential to neutralize poisoned model updates from compromised clients without needing to identify them explicitly.

A privacy breach distinct from poisoning where an adversary exploits access to a trained model's confidence scores to reconstruct representative features of the private training data. While poisoning corrupts the model, inversion attacks extract sensitive information from a clean one, highlighting the dual trust problem in collaborative learning.

A defensive technique that bounds the influence of any single training example by scaling down individual gradients whose L2 norm exceeds a predefined threshold. In differentially private training, clipping limits sensitivity. Against poisoning, it can cap the magnitude of malicious updates, preventing an attacker from overwhelming the global model with an outsized, corrupted gradient.

A specific pattern or signal embedded in poisoned training samples that causes the compromised model to misbehave only when the trigger is present. For example, a network traffic classifier might be poisoned to misclassify malicious packets as benign only when they contain a specific magic byte sequence, remaining accurate on clean data to evade detection.

A cryptographic protocol ensuring the central server can only compute the sum of encrypted model updates from clients, never inspecting individual contributions. While primarily a privacy mechanism, secure aggregation also complicates poisoning defenses by preventing the server from auditing individual updates for anomalies before aggregation.

A mathematical framework that injects calibrated statistical noise into computations to mask individual contributions. When applied during federated training, differential privacy provides a provable bound on information leakage but can also incidentally mitigate certain poisoning attacks by smoothing out the sharp, anomalous gradients introduced by an adversary.