Adversarial Training

The core of adversarial training is a min-max game. The inner maximization step generates the strongest possible adversarial examples that maximize the model's loss. The outer minimization step updates model weights to correctly classify these perturbed inputs. This is formalized as solving min_θ E(x,y)~D [max_δ∈S L(θ, x+δ, y)], where S is the set of allowed perturbations bounded by epsilon. This saddle-point problem forces the model to learn robust features rather than brittle, non-robust ones that are easily exploited.

The standard method for the inner maximization loop is the Projected Gradient Descent (PGD) attack. Starting from a clean input, PGD iteratively takes steps in the direction of the gradient of the loss with respect to the input, projecting back onto the L-p norm ball after each step.

• Multi-step variant: Uses k iterations (e.g., PGD-7 or PGD-20) for a stronger adversary.
• Random start: A random perturbation within the epsilon-ball is added before iteration to avoid gradient masking.
• Restarts: Multiple random starts are used to find the global loss maximum, preventing the model from creating shallow loss landscapes.

A common failure mode where a model appears robust but achieves this by obfuscating its gradients rather than learning truly robust features. Adversarial training must actively avoid this.

• Shattered gradients: The model is intentionally non-differentiable or has vanishing gradients, preventing attack generation.
• Stochastic gradients: Randomized transformations (e.g., random resizing) are used to break gradient flow.
• Exploding gradients: The loss landscape is extremely steep, causing gradient-based attacks to overshoot.

Proper evaluation uses black-box attacks (e.g., transfer attacks or SPSA) and unbounded search to verify that robustness is genuine and not a result of gradient masking.

A fundamental tension exists between a model's performance on clean, natural data and its resilience to adversarial perturbations. Training exclusively on adversarial examples often degrades standard accuracy.

• Feature purification: The model discards highly predictive but non-robust features that correlate with the label but are imperceptible to humans.
• Increased sample complexity: Robust generalization requires significantly more training data than standard generalization.
• Mitigation strategies: Techniques like TRADES (TRadeoff-inspired Adversarial DEfense via Surrogate-loss minimization) introduce a regularization term that explicitly balances the trade-off between natural and robust errors.

While designed for evasion attacks, adversarial training directly degrades the fidelity of model inversion attacks. By flattening the loss landscape around training points, the model's confidence scores become less informative about the specific features of any single training example.

• Reduced confidence leakage: The model outputs lower maximum confidence on training data, a key signal exploited by inversion attacks.
• Obfuscated gradients for inversion: The same gradient obfuscation that hinders evasion attacks also makes gradient-based inversion (like Deep Leakage from Gradients) less effective.
• Feature compression: The model is forced to rely on a compressed set of robust features, discarding the high-frequency, sample-specific details that inversion attacks aim to reconstruct.

A specialized variant that explicitly optimizes for privacy by crafting the adversarial examples to maximize the risk of membership inference or input reconstruction, rather than just misclassification.

• Privacy adversary: The inner loop simulates an attacker trying to distinguish training from non-training data or reconstruct inputs.
• Unified defense: A single training procedure simultaneously hardens against evasion, membership inference, and model inversion.
• Connection to Differential Privacy: ATPR can be combined with DP-SGD to provide a dual layer of empirical and provable privacy guarantees, where adversarial training handles the empirical robustness and differential privacy provides the mathematical bound.