Adversarial Training

Adversarial training is fundamentally formulated as a minimax game. The inner maximization generates the strongest possible adversarial example within an Lp-norm bound, while the outer minimization adjusts the model weights to classify that example correctly.

• Inner Loop: Projects the input to maximize the loss function.
• Outer Loop: Updates parameters to minimize empirical risk on the generated adversary.
• Saddle Point: The goal is to converge to a point where the model is robust to worst-case perturbations.

Unlike static data augmentation, adversarial training generates perturbed inputs dynamically during each training step. This forces the model to continuously adapt to new attacks rather than memorizing a fixed set of examples.

• PGD Integration: Typically uses Projected Gradient Descent to craft multi-step adversaries.
• Free Adversarial Training: Recycles gradients computed for the model update to simultaneously craft perturbations, reducing computational overhead.
• Curriculum Approach: Gradually increases perturbation strength during training to stabilize convergence.

A central challenge in adversarial training is the trade-off between standard accuracy and adversarial robustness. Models hardened against attacks often suffer a performance drop on clean, unperturbed data.

• TRADES Loss: Explicitly balances this trade-off by decomposing the error into natural error and boundary error.
• Feature Overfitting: Robust models may rely on less generalizable features, hurting clean accuracy.
• Data Scaling: Recent research shows that larger, more diverse datasets can partially mitigate this trade-off.

Adversarial training is significantly more computationally expensive than standard training. Generating multi-step adversarial examples requires multiple forward and backward passes through the network for every single batch of data.

• Cost Multiplier: A 10-step PGD adversary increases training time by roughly 10x.
• Single-Step Mitigation: Fast Gradient Sign Method (FGSM) with random initialization offers a faster but weaker alternative.
• Hardware Requirements: Necessitates high-memory GPUs to store intermediate gradient graphs for perturbation crafting.

A brittle model can exhibit gradient masking, where its loss surface is intentionally or unintentionally obfuscated to fool gradient-based attacks. Adversarial training aims for true robustness, not obfuscation.

• Shattered Gradients: Non-smooth loss landscapes that prevent attacks from optimizing.
• Stochastic Gradients: Random defenses that break deterministic attacks.
• Evaluation Best Practice: Always validate defenses with black-box attacks and AutoAttack to ensure gradient masking is not mistaken for genuine robustness.

Standard adversarial training provides empirical robustness—it defends against known attacks but offers no mathematical guarantee. This contrasts with certified defenses that provide provable lower bounds.

• Empirical: Evaluated by testing against a suite of attacks like PGD and AutoAttack.
• Certified: Uses techniques like Randomized Smoothing to guarantee prediction stability within a verified radius.
• Hybrid Approaches: Some methods combine adversarial training with Lipschitz constant regularization to bridge the gap between empirical and certified robustness.

The de facto standard adversary used to generate examples during adversarial training. PGD is an iterative, multi-step attack that starts from a random point within an epsilon-ball around the clean input and repeatedly takes gradient steps toward maximizing the loss, projecting back onto the norm constraint after each step. Because it solves a constrained optimization problem from multiple random restarts, PGD represents a powerful first-order adversary. Training against a strong PGD attacker empirically provides robustness against a wide range of weaker attacks.

TRADES (TRadeoff-inspired Adversarial Defense via Surrogate-loss minimization) is a training objective that explicitly balances the accuracy-robustness trade-off. Instead of only minimizing loss on adversarial examples, TRADES adds a regularization term that minimizes the Kullback-Leibler divergence between the model's prediction on a clean input and its prediction on an adversarial counterpart. This encourages the decision boundary to be both accurate on natural data and smooth in the neighborhood of training points, often yielding superior certified and empirical robustness compared to standard adversarial training.

While adversarial training provides empirical robustness, it offers no formal guarantee. Randomized Smoothing builds a certifiably robust classifier from any base model by adding isotropic Gaussian noise to inputs at inference time. The smoothed model returns the most probable prediction under this noise distribution. This construction provides a provable lower bound on the L2 radius within which no adversarial example can exist. The certified radius scales with the margin of the majority class probability, making it a complementary guarantee to the empirical strength of adversarial training.

AutoAttack is a parameter-free, ensemble evaluation protocol that has become the standard for measuring empirical adversarial robustness. It sequentially applies four diverse attacks: an APGD variant with a novel loss function, a targeted Difference of Logits Ratio attack, the black-box FAB attack, and the query-based Square Attack. Because it requires no hyperparameter tuning and combines white-box and black-box methods, AutoAttack prevents gradient masking defenses from appearing robust. Reporting AutoAttack accuracy alongside standard PGD accuracy is now considered essential for credible robustness claims.

Gradient masking is a brittle and illusory defense where a model's gradients are obfuscated—either intentionally through non-differentiable layers or unintentionally through numerical instability like shattered gradients. This prevents gradient-based attacks like PGD from functioning, creating a false sense of security. However, these models remain vulnerable to black-box transfer attacks, where an adversary trains a substitute model, or to decision-based attacks that only require hard-label outputs. Adversarial training explicitly avoids gradient masking by keeping the loss surface smooth and gradient information meaningful.