Adversarial Training

Adversarial training is framed as a robust optimization problem. The inner maximization generates the strongest possible adversarial example within a perturbation budget, while the outer minimization updates model weights to classify that example correctly.

• Objective: min_θ E_{(x,y)~D} [max_{δ∈S} L(f_θ(x+δ), y)]
• The inner loop finds the perturbation δ that maximizes loss.
• The outer loop adjusts parameters θ to minimize loss on the perturbed input.
• This forces the model to be locally constant around training points.

The de facto standard for adversarial training uses multi-step PGD as the inner attack. Unlike single-step FGSM, PGD iteratively refines the perturbation and projects it back onto the epsilon-ball, finding a stronger adversary.

• Step 1: Initialize perturbation randomly within the ε-ball.
• Step 2: Take a gradient step in the direction that maximizes loss.
• Step 3: Project the result back onto the L∞-norm ball.
• Step 4: Repeat for K steps (typically 7-40).
• Training with a stronger adversary yields a more robust model.

A fundamental tension exists between clean accuracy and adversarial robustness. Training on adversarial examples forces the model to rely on robust, human-aligned features rather than brittle, high-frequency patterns, often reducing performance on unperturbed data.

• Clean accuracy drop: Standard models may lose 5-10% accuracy on natural test sets.
• Robust accuracy gain: Adversarial accuracy can jump from near 0% to over 45%.
• This trade-off is a Pareto frontier problem, not a bug.
• Larger models and more data can partially mitigate the gap.

Adversarial training is significantly more expensive than standard empirical risk minimization. Each training batch requires K additional forward and backward passes to compute the adversarial perturbation, multiplying the training time.

• Cost factor: Training time increases by a factor of K+1.
• A 40-step PGD adversary makes training roughly 40x slower.
• Free adversarial training and single-step methods reduce cost but risk gradient masking.
• Mixed-precision training and distributed strategies are essential for large-scale use.

A critical pitfall is gradient masking, where a defense appears robust but merely obfuscates gradients. True adversarial training avoids this by using unbounded or multi-step attacks during evaluation to verify that robustness is genuine, not a gradient artifact.

• Obfuscated gradients give a false sense of security.
• Evaluation protocol: Always test against a strictly stronger attack than used in training.
• White-box PGD with random restarts is the minimum verification standard.
• AutoAttack ensembles are now the gold standard for evaluation.

Adversarial training fundamentally changes the model's internal representations. The learned features become more semantically aligned and less reliant on imperceptible, high-frequency textures that correlate with class labels but are not causal.

• Robust features are perceptually aligned and interpretable.
• Non-robust features are predictive but brittle and human-incomprehensible.
• Adversarially trained models produce saliency maps that align with object boundaries.
• This alignment makes the model's reasoning more transparent to human auditors.