Adversarial Training

A foundational white-box attack that generates adversarial examples in a single step. FGSM computes the gradient of the loss with respect to the input and applies a small perturbation in the direction that maximizes loss.

• Single-step attack: Extremely fast to compute, making it ideal for on-the-fly adversarial example generation during training.
• Linear approximation: Assumes the decision boundary is locally linear, which is surprisingly effective against high-dimensional models.
• Epsilon parameter: Controls the maximum perturbation magnitude per pixel, typically constrained by an L-infinity norm.

The gold-standard iterative attack used to benchmark adversarial robustness. PGD extends FGSM by taking multiple small gradient steps and projecting the result back onto an epsilon-ball around the original input.

• Multi-step variant: Applies FGSM repeatedly with a smaller step size, creating stronger adversarial examples than single-step methods.
• Random restarts: PGD typically initializes from a random point within the epsilon-ball to avoid local minima in the loss landscape.
• Min-max formulation: Adversarial training with PGD approximates solving the inner maximization problem of robust optimization.

An integrity attack where adversaries inject malicious samples into the training dataset before model training begins. Unlike evasion attacks, poisoning corrupts the model's learned parameters themselves.

• Backdoor attacks: Insert trigger patterns that cause targeted misclassification only when the trigger is present, leaving clean performance intact.
• Availability attacks: Indiscriminately degrade overall model accuracy by injecting mislabeled or noisy samples.
• Defense: Adversarial training alone is insufficient; requires data provenance tracking, outlier detection, and robust aggregation techniques.

Provides a mathematical guarantee that a model's prediction will not change for any input within a verified perturbation radius. Unlike empirical defenses, certified methods offer provable lower bounds on adversarial resilience.

• Randomized smoothing: Transforms any base classifier into a certifiably robust one by adding Gaussian noise and taking majority votes over many noisy samples.
• Interval bound propagation: Propagates bounds through the network layers to formally verify output stability under input perturbations.
• Trade-off: Certified methods typically achieve lower clean accuracy than empirical adversarial training but provide ironclad guarantees.

An inference-time attack where an adversary modifies a malicious input to bypass a deployed classifier without altering its harmful functionality. This is the primary threat adversarial training is designed to counter.

• White-box attacks: Attacker has full knowledge of model architecture, parameters, and gradients.
• Black-box attacks: Attacker only has query access; uses transfer attacks or score-based optimization.
• Physical domain: Adversarial patches and stickers that fool real-world computer vision systems in autonomous vehicles and surveillance.

A false sense of security where a defense appears robust but actually relies on obfuscated or non-existent gradients rather than true model resilience. Attackers can easily circumvent these defenses with adaptive techniques.

• Shattered gradients: Defenses that intentionally break gradient flow, such as non-differentiable preprocessing steps.
• Stochastic gradients: Defenses that rely on randomness to evade gradient-based attacks.
• Detection: A defense is likely masking gradients if white-box attacks perform worse than black-box transfer attacks. True adversarial training produces usable, informative gradients.