Projected Gradient Descent (PGD)

Unlike single-step methods like FGSM, PGD applies the gradient-based perturbation iteratively over multiple small steps. Each step computes the gradient of the loss with respect to the input, adds a small perturbation in the gradient direction, and then projects the result back onto the Lp-norm ball of radius epsilon. This iterative refinement finds stronger adversarial examples within the allowed adversarial budget.

The defining mechanism of PGD is the projection operator. After each gradient step, the perturbed input is mathematically projected back onto the surface of the Lp-norm sphere (typically L-infinity) centered at the original input. This ensures the perturbation never exceeds the specified distortion constraint (epsilon). Common projection methods include:

• L-infinity: Clamping each pixel or sample to [x - ε, x + ε]
• L2: Scaling the perturbation vector to have norm ≤ ε

A critical enhancement in standard PGD is random initialization within the epsilon-ball before beginning the iterative steps. Instead of starting from the original input, the attack begins at a randomly perturbed point within the allowed budget. This prevents the optimization from getting stuck in shallow local maxima and significantly increases attack success rates against adversarial training defenses.

PGD is considered the strongest first-order attack—it only uses gradient information and does not require second-order approximations. If a model is robust against PGD, it is empirically robust against all other first-order attacks. This makes PGD the de facto standard for evaluating adversarial robustness and the primary method for generating adversarial examples during adversarial training.

While FGSM takes a single large step to the boundary of the epsilon-ball, PGD takes multiple smaller steps with intermediate projections:

• FGSM: x_adv = x + ε · sign(∇_x J(x, y))
• PGD: x_t+1 = Π_{x+ε} (x_t + α · sign(∇_x J(x_t, y))) The step size α is typically set to ε/k for k iterations, making PGD a strictly more powerful attack at the cost of increased computation.

PGD serves as the attack component in the min-max formulation of adversarial training. During training, PGD generates the worst-case adversarial examples within the epsilon-ball, and the model parameters are updated to correctly classify these perturbed inputs. This min-max optimization—minimizing model loss while maximizing input perturbation—is the most empirically successful defense against adversarial attacks.