Projected Gradient Descent (PGD) Attack

PGD is not a single gradient step; it is an iterative optimization algorithm. Starting from a random perturbation within the epsilon-ball, it repeatedly takes small steps in the direction of the gradient of the loss function. After each step, the perturbed input is projected back onto the Lp-norm ball to ensure the attack remains imperceptible. This iterative refinement creates significantly more powerful adversarial examples than single-step methods like FGSM.

A critical component of the standard PGD implementation is random initialization. Rather than starting from the original clean image, the attack begins at a uniformly random point within the allowed epsilon-ball. This prevents the optimizer from getting stuck in shallow local maxima of the loss landscape near the original input. Without random restarts, PGD can significantly underestimate a model's true vulnerability.

The 'Projected' in PGD refers to the mathematical projection operator applied after every gradient step. This operator maps the perturbed input back to the closest point within the valid constraint set. For L∞ attacks, this is a simple clipping operation that ensures no pixel changes by more than epsilon. For L2 attacks, it involves rescaling the perturbation vector to have an L2 norm exactly equal to epsilon. This strict enforcement guarantees the attack stays within the defined threat model.

PGD is considered a universal adversary among first-order attacks—those that only use gradient information. If a model is robust against a multi-step PGD attack with random restarts, it is empirically robust against all other gradient-based white-box attacks. This property makes PGD the de facto standard for adversarial training, where PGD-generated examples are injected into the training loop to harden the model's decision boundaries.

PGD performance is sensitive to hyperparameter tuning. The step size (alpha) controls how far the perturbation moves per iteration, typically set to epsilon/4 or epsilon/10. A step size that is too large causes the optimizer to oscillate around the loss maximum; too small requires excessive iterations. Advanced variants use learning rate schedules or momentum-based updates to converge faster and find stronger adversarial examples within a fixed budget.

PGD is the backbone of adversarial training, the most empirically successful defense methodology. During training, every clean batch is augmented with PGD-generated adversarial examples. The model is then trained to correctly classify both clean and perturbed inputs. This min-max optimization—minimizing the model's loss while maximizing it via PGD—creates a robust saddle point that significantly improves certified and empirical robustness.

The foundational single-step white-box attack that PGD iteratively refines. FGSM computes the sign of the gradient of the loss function with respect to the input and applies a perturbation of magnitude epsilon in a single step. While computationally cheap, it significantly underestimates model vulnerability compared to multi-step variants like PGD. Key distinction: PGD is essentially FGSM applied multiple times with a projection step back onto the epsilon-ball after each iteration, making it a strictly stronger first-order adversary.

The mathematical distance metric that defines the 'epsilon-ball' within which a PGD adversary is allowed to operate. The L-infinity norm measures the maximum absolute change to any single pixel or feature. Why it matters for PGD: The projection step at each iteration clips every perturbed coordinate back to the range [x - epsilon, x + epsilon], ensuring the adversarial example remains visually indistinguishable from the original. This constraint is what makes PGD a realistic threat model for physical-world attacks.