Poisoning Budget

A poisoning budget is the adversarial analog to a privacy budget (ε) in differential privacy. Both bound the maximum influence of a single data point:

• Privacy Budget (ε): Limits information leakage about one record.
• Poisoning Budget: Limits the adversarial impact of one record. This duality means DP-SGD with a small ε naturally provides a form of poisoning robustness by clipping and noising per-example gradients.

An attack where an adversary injects a hidden trigger pattern into a model during training. The poisoning budget defines the maximum number of trigger-embedded samples the attacker can insert. A model trained with a 1% poisoning budget, for example, would tolerate up to 1% of training data containing the backdoor trigger without breaking the defense guarantee.

A property providing a mathematical proof that a model's prediction remains constant for any input within a specified Lp-norm radius. In the context of poisoning, certified robustness against training-time attacks guarantees that no adversary controlling fewer samples than the poisoning budget can change the model's prediction on a target example. Techniques include:

• Randomized smoothing for inference-time certificates
• Differential privacy for training-time certificates
• Interval bound propagation for deterministic guarantees

A training algorithm that clips per-example gradients and adds calibrated Gaussian noise to the aggregated gradient. This directly enforces a poisoning budget by bounding the maximum influence any single training point can exert on the model. Key properties:

• The clipping threshold acts as an implicit poisoning budget
• Larger noise multipliers increase the budget but reduce utility
• Provides a formal epsilon-delta privacy guarantee that doubles as a robustness certificate

A detection method that identifies poisoned training examples by analyzing the singular value decomposition of feature representations. The technique assumes that backdoor triggers leave a detectable statistical trace that correlates with the adversary's poisoning budget. A larger budget produces a stronger spectral signal, making detection easier but also increasing the potential damage before remediation. This creates a detection vs. budget trade-off that defenders must model.

A class of Byzantine-resilient algorithms used in federated learning to combine model updates while mitigating malicious contributions. The poisoning budget in this context specifies the maximum fraction of clients an adversary controls. Common aggregation rules include:

• Krum: Selects the update closest to a majority of its neighbors
• Trimmed Mean: Discards extreme values before averaging
• Median: Uses coordinate-wise median to resist outliers Each rule has a theoretical breakdown point tied directly to the assumed poisoning budget.

The process of filtering or transforming a training dataset to remove anomalous, mislabeled, or poisoned samples before they corrupt learning. Effective sanitization effectively reduces the realized poisoning budget below the adversary's intended level. Techniques include:

• Activation clustering on final hidden layer representations
• Neural Cleanse for reverse-engineering potential triggers
• Fine-pruning to remove dormant neurons activated only by triggers Sanitization strength must be calibrated against the expected poisoning budget to avoid removing legitimate edge cases.