Adversarial Training

Adversarial training is fundamentally formulated as a min-max optimization problem. The inner maximization step generates adversarial examples that maximize the model's loss, while the outer minimization step updates model weights to correctly classify those perturbed inputs. This creates a dynamic game where the model continuously adapts to worst-case perturbations within a defined threat model, typically an Lp-norm ball around clean samples.

PGD is the gold-standard method for crafting adversarial examples during training. It iteratively applies small perturbations in the direction of the gradient sign, projecting the result back onto an epsilon-ball around the original input after each step. Key parameters include:

• Epsilon (ε): The maximum perturbation magnitude
• Step size (α): The per-iteration perturbation size
• Number of steps: More steps yield stronger attacks but increase compute cost

Multi-step PGD adversaries produce models with superior empirical robustness compared to single-step methods like FGSM.

A well-documented phenomenon where increasing adversarial robustness degrades performance on clean, unperturbed data. This trade-off arises because the decision boundaries learned by adversarially trained models are fundamentally different—they prioritize invariant features that align with human perception over brittle, highly discriminative but easily perturbed patterns. In practice, a model with 90% adversarial accuracy might drop from 95% to 87% on clean test data.

Adversarial training is significantly more expensive than standard empirical risk minimization. Generating a k-step PGD adversary requires k forward and backward passes per training batch, effectively multiplying the training time by a factor of k+1. For large-scale models like vision transformers or LLMs, this overhead is a primary barrier to adoption, driving research into free adversarial training and fast gradient sign methods that amortize the cost.

Models hardened via adversarial training against a specific attack algorithm often exhibit cross-attack generalization, resisting unseen adversaries not used during training. A model trained with PGD adversaries may also defend against CW attacks or AutoAttack. However, this transferability is imperfect—attackers can exploit this by crafting transferable black-box attacks using surrogate models, making ensemble adversarial training across multiple attack types a more robust strategy.

Static adversarial training with a fixed epsilon can lead to catastrophic overfitting, where robustness against strong attacks suddenly collapses. Curriculum adversarial training mitigates this by gradually increasing the perturbation budget during training. Adaptive methods dynamically adjust epsilon per sample based on the model's current vulnerability, focusing compute on borderline cases. This yields more stable convergence and higher final robustness.