Projected Gradient Descent (PGD)

Unlike single-step methods like FGSM, PGD solves the inner maximization problem through multiple gradient ascent steps. Starting from a random initialization within the epsilon-ball, it repeatedly computes the gradient of the loss with respect to the input and takes a small step in that direction. Each step is followed by a projection operator that snaps the perturbation back onto the feasible set, ensuring the adversarial example remains within the allowed L∞ or L2 budget. This iterative refinement finds significantly stronger adversaries than one-shot attacks.

A critical component of PGD is the random start before iterative optimization begins. Rather than starting from the original clean input, PGD initializes the perturbation uniformly within the epsilon-ball. This prevents the optimizer from getting stuck in shallow local maxima near the decision boundary. Multiple random restarts are often employed to increase attack success rates, making PGD a reliable lower bound for worst-case empirical robustness.

The 'Projected' in PGD refers to the constraint enforcement mechanism. After each gradient step, the perturbed input is projected back onto the Lp-norm ball of radius epsilon centered at the original input. For L∞ attacks, this is a simple clipping operation that bounds each pixel's perturbation independently. For L2 attacks, the perturbation vector is rescaled to have a norm no greater than epsilon. This projection guarantees the adversarial example remains imperceptible or within the threat model's defined budget.

Madry et al. (2018) demonstrated that PGD is a universal adversary among first-order attacks—if a model is robust against PGD, it is empirically robust against all gradient-based attacks. This insight established PGD as the primary benchmark for adversarial training. The attack leverages only first-order gradient information, making it computationally tractable for deep networks while still exposing vulnerabilities that gradient-masking defenses attempt to hide.

PGD's effectiveness depends on two hyperparameters: step size (alpha) and number of iterations (k). The step size is typically set as a fraction of epsilon (e.g., alpha = epsilon / 4), balancing convergence speed with granularity. The iteration count controls the attack budget—more steps yield stronger adversaries but increase compute cost. Common configurations include PGD-20 (20 steps) for evaluation and PGD-7 or PGD-10 for adversarial training, representing a trade-off between attack strength and training throughput.

PGD serves as the canonical attack for adversarial training, where models are trained on PGD-generated examples rather than clean data. This min-max procedure—minimizing model loss against a PGD adversary that maximizes it—produces state-of-the-art empirical robustness. Models trained with PGD adversarial training exhibit flattened loss landscapes and smoother decision boundaries, making them resistant to a wide family of first-order attacks beyond the specific PGD configuration used during training.

A false sense of security phenomenon where a defense obfuscates gradients rather than truly improving robustness.

• Obfuscated Gradients: Shattered gradients (non-differentiable defenses), stochastic gradients (randomized defenses), or exploding/vanishing gradients.
• Detection: If a black-box attack (like a score-based method) succeeds where a white-box PGD fails, gradient masking is likely present.
• Implication: Defenses exhibiting this property are considered broken, as they fail against transfer attacks or hard-label methods.