Projected Gradient Descent (PGD)

Unlike the single-step Fast Gradient Sign Method (FGSM), PGD applies the gradient update multiple times with a small step size α. Starting from a random initialization within the epsilon-ball, each iteration computes the gradient of the loss with respect to the input and takes a step in that direction. This iterative process allows PGD to escape shallow local maxima and find a much stronger adversarial perturbation than FGSM.

After each gradient step, the perturbed input is projected back onto the L-p norm ball of radius ε centered at the original input. This enforces the perturbation budget constraint, ensuring the adversarial example remains within the allowed threat model. The projection operator clips the perturbation magnitude, preventing the attack from drifting outside the imperceptibility threshold.

PGD begins from a random starting point within the epsilon-ball rather than the original input. This prevents the attack from getting stuck in poor local maxima near the starting point. By exploring the perturbation space from multiple random restarts, PGD provides a more reliable estimate of the model's worst-case robustness.

PGD is considered a universal first-order adversary—it is the strongest attack possible using only first-order gradient information. If a model is robust against PGD, it is empirically robust against all other gradient-based attacks. This makes PGD the de facto standard for adversarial training and robustness benchmarking.

PGD is highly configurable through key hyperparameters:

• ε (epsilon): Maximum perturbation magnitude
• α (alpha): Step size per iteration
• Number of steps: Iteration count
• Norm type: L∞, L2, or L1 constraint

This flexibility allows security engineers to simulate different attacker capabilities and evaluate robustness under varying threat models.

PGD is the core attack used in adversarial training, the most effective empirical defense. During training, PGD generates adversarial examples on-the-fly, and the model is trained to classify them correctly. This min-max optimization—minimizing loss against maximally perturbed inputs—forces the model to learn smooth decision boundaries resistant to adversarial manipulation.

The single-step precursor to PGD. FGSM generates an adversarial example by computing the sign of the gradient of the loss with respect to the input and taking one step of magnitude epsilon. While computationally cheap, it often underestimates model vulnerability. PGD is essentially a multi-step variant of FGSM with a projection step, making it a strictly stronger attack for evaluating true robustness.

The primary defensive technique that uses PGD as its engine. During training, PGD is run on each batch to generate the strongest possible adversarial examples, which are then fed back into the model with correct labels. This min-max optimization forces the model to learn a smoother decision boundary. PGD-based adversarial training remains one of the few empirically robust defenses against L-infinity attacks.

Defines the maximum allowed distortion in an adversarial attack, typically constrained by an L-infinity norm for PGD. The epsilon-ball is a hypercube around the original input where every pixel can change by at most epsilon. PGD's projection step ensures the perturbed image never leaves this ball. Common values for CIFAR-10 are 8/255; for ImageNet, 4/255.

A brittle defense phenomenon where a model's gradients become non-informative or zero, causing PGD to stall. This creates a false sense of security—the model appears robust to gradient-based attacks but remains vulnerable to black-box or transfer attacks. True robustness requires PGD to converge to a high-loss region, not fail due to shattered or obfuscated gradients.

An optimization-based attack that formulates adversarial example generation as minimizing a loss function balancing distortion and misclassification. Unlike PGD, which uses fixed step sizes and projection, C&W uses Adam or SGD to directly optimize for minimal perturbations. It often finds smaller distortions than PGD but is computationally heavier, making PGD the preferred choice for adversarial training loops.

A formal, verifiable guarantee that a model's prediction will not change for any perturbation within a specified Lp-norm radius. While PGD provides an empirical lower bound on robustness by finding adversarial examples, certification methods like randomized smoothing provide a provable upper bound. The gap between PGD attack success and certified robustness is an active area of research.