Projected Gradient Descent (PGD)

Unlike the single-step Fast Gradient Sign Method (FGSM), PGD performs a multi-step optimization process. It repeatedly applies the FGSM with a small step size α, recalculating the gradient at each iteration to find a stronger adversarial example. This iterative refinement makes PGD significantly more powerful than single-step attacks.

• Step size α: Typically set to ε / 5 or ε / 10
• Number of iterations: Usually 7, 20, or 40 steps
• Gradient recalculation: Performed at each step for precision

The defining mechanism of PGD is the projection step. After each gradient update, the perturbed input is mathematically projected back onto the L∞-norm ball of radius ε centered at the original input. This ensures the adversarial example remains within the allowed perturbation budget and the attack remains imperceptible.

• L∞ constraint: Limits the maximum change to any single pixel
• Projection operator: Clips values to [x - ε, x + ε]
• Boundary enforcement: Prevents unbounded drift during iterations

PGD incorporates a random start within the epsilon-ball before beginning gradient ascent. This prevents the attack from getting stuck in shallow local maxima of the loss landscape. Without random initialization, the attack may fail to find the true worst-case perturbation.

• Uniform random noise: Added to the clean input before iteration
• Purpose: Escape non-robust local optima
• Standard practice: A critical component for a reliable robustness evaluation

PGD is a white-box attack, meaning it requires full access to the target model's architecture, parameters, and gradients. This assumption represents the strongest possible adversary and is used to establish an empirical lower bound on model robustness. If a model is robust to PGD, it is likely robust to weaker black-box attacks.

• Gradient access: Required for loss maximization
• Worst-case evaluation: Gold standard for adversarial robustness benchmarking
• Transferability: PGD examples often transfer to black-box models

PGD is the cornerstone of adversarial training, the most empirically successful defense to date. During training, PGD is used to generate adversarial examples on-the-fly, which are then injected into the training batch. This forces the model to learn a decision boundary that is robust within the epsilon-ball.

• Min-max formulation: Inner maximization via PGD, outer minimization via SGD
• Computational cost: Significantly increases training time
• State-of-the-art defense: Forms the basis of robust models on benchmarks like RobustBench

PGD provides a powerful lens for analyzing the geometric properties of the loss landscape. By tracing the path of the perturbation, researchers can identify highly curved, non-robust regions where small input changes cause drastic output shifts. A flat loss landscape correlates strongly with adversarial robustness.

• Curvature analysis: Reveals sharp vs. flat minima
• Loss visualization: PGD trajectories map the local geometry
• Robustness indicator: Flatter landscapes resist adversarial manipulation