Glossary

Projected Gradient Descent (PGD)

A powerful iterative white-box adversarial attack that repeatedly takes a small step in the direction of the loss gradient and projects the result back onto an epsilon-ball around the original input.

Get in touch Learn more

Stylish WeWork-like workspace with hot desks and document wall, professional searching through enterprise knowledge base on a mounted ultrawide display, warm industrial pendants overhead.

ADVERSARIAL ATTACK METHODOLOGY

What is Projected Gradient Descent (PGD)?

Projected Gradient Descent is a foundational iterative white-box attack used to evaluate and benchmark the adversarial robustness of machine learning models.

Projected Gradient Descent (PGD) is an iterative white-box adversarial attack that generates perturbed inputs to cause model misclassification by repeatedly taking small steps in the direction of the loss gradient and projecting the result back onto an epsilon-ball around the original input. It is widely considered a universal first-order adversary, meaning a model robust to PGD is often robust against a broad range of other gradient-based attacks.

The attack formalizes adversarial example generation as a constrained optimization problem, typically using the L-infinity norm to bound the maximum per-pixel perturbation. After each gradient ascent step, the perturbed image is projected back into the valid epsilon-neighborhood to ensure the distortion remains imperceptible. PGD serves as a core component of adversarial training, where models are hardened by training directly on these worst-case perturbed examples.

ADVERSARIAL ATTACK METHODOLOGY

Key Characteristics of PGD

Projected Gradient Descent (PGD) is the de facto standard iterative white-box attack for evaluating adversarial robustness. It refines the Basic Iterative Method by incorporating a random start and projecting perturbations onto an Lp-norm ball, making it a universal first-order adversary.

Iterative Multi-Step Optimization

Unlike single-step attacks like the Fast Gradient Sign Method (FGSM), PGD performs multiple gradient ascent steps to maximize the loss function. Each iteration takes a small step of size α in the direction of the gradient sign, allowing the attack to find a more powerful adversarial perturbation within the allowed epsilon budget.

40-100

Typical Iteration Steps

Lp-Norm Projection Operator

After each gradient step, the perturbed input is projected back onto the epsilon-ball surrounding the original input. This projection ensures the perturbation magnitude stays within the specified budget.

L∞ norm: Projects by clipping each pixel value to [x-ε, x+ε].
L2 norm: Projects by scaling the perturbation vector to have a Euclidean norm of at most ε.

Random Initialization (Random Start)

PGD begins by adding a small random perturbation uniformly sampled from the Lp-ball to the clean input. This prevents the optimization from getting stuck in shallow loss plateaus near the starting point and ensures the attack explores the loss landscape more effectively, making it significantly stronger than deterministic iterative methods.

Universal First-Order Adversary

Madry et al. (2017) demonstrated that robustness against PGD provides empirical defense against all first-order attacks—those relying only on gradient information. This makes PGD the standard benchmark for adversarial training. A model that survives a strong PGD attack is considered robust against any adversary that does not use second-order optimization.

EXPLORE

Gradient Ascent on Cross-Entropy Loss

PGD maximizes the model's classification loss with respect to the input while holding parameters fixed. The attack objective is:

code
x_{adv} = argmax_{x' ∈ B(x, ε)} L(f(x'), y_true)

where B(x, ε) is the Lp-ball of radius ε around the clean input x. This directly targets the model's decision boundary.

Step Size and Budget Relationship

The per-step size α is typically set as α = ε / k * c, where k is the number of steps and c is a constant (often 2.5). This ensures the attack can reach the boundary of the epsilon-ball and explore it thoroughly. Too small a step size wastes iterations; too large overshoots and requires aggressive projection.

WHITE-BOX ATTACK COMPARISON

PGD vs. Other Adversarial Attacks

Comparative analysis of Projected Gradient Descent against other prominent gradient-based and optimization-based adversarial attacks used to evaluate model robustness.

Feature	Projected Gradient Descent (PGD)	Fast Gradient Sign Method (FGSM)	Carlini & Wagner (C&W)
Attack Category	Iterative gradient-based	Single-step gradient-based	Optimization-based
Iterative Steps	Multiple (40-100 typical)	Single step	Multiple (binary search + gradient descent)
Perturbation Constraint	L∞ or L2 epsilon-ball projection	L∞ epsilon-ball (no projection)	L0, L2, or L∞ minimized in loss
Computational Cost	Moderate	Low	High
Defensive Distillation Resistance
Gradient Masking Resistance
Attack Strength (L∞)	Strong (near-optimal within budget)	Weak (underestimates vulnerability)	Strong (minimal distortion)
Primary Use Case	Adversarial training and robustness benchmarking	Baseline evaluation and fast prototyping	Minimum distortion analysis and defense evasion

ADVERSARIAL ROBUSTNESS

Frequently Asked Questions About PGD

Projected Gradient Descent (PGD) is the gold-standard iterative white-box attack used to benchmark adversarial robustness. These answers cover its mechanism, variants, and defensive implications.

Projected Gradient Descent (PGD) is a first-order iterative adversarial attack that finds a perturbed input maximizing the model's loss within an epsilon-ball around the original input. It works by repeatedly taking a step in the direction of the loss gradient sign and then projecting the result back onto the allowed perturbation space. This projection step ensures the adversarial example stays within the specified Lp-norm budget (typically L∞). Unlike single-step methods like the Fast Gradient Sign Method (FGSM), PGD's multi-step optimization produces significantly stronger attacks, making it the standard benchmark for evaluating adversarial robustness. The attack is initialized with a random start within the epsilon-ball to avoid shallow loss regions, and the step size is typically set to epsilon * 2.5 / number_of_iterations.

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

ADVERSARIAL ATTACK ECOSYSTEM

Related Terms

Core concepts surrounding Projected Gradient Descent (PGD), the iterative white-box attack that serves as the standard benchmark for evaluating adversarial robustness.

Fast Gradient Sign Method (FGSM)

The single-step predecessor to PGD that computes the adversarial perturbation by taking a single large step in the direction of the loss gradient sign. FGSM is computationally cheap but produces weaker attacks than iterative methods. PGD can be understood as FGSM applied repeatedly with a smaller step size and projection back onto the epsilon-ball after each iteration. While FGSM often fails against gradient-masked defenses, multi-step PGD remains effective.

Adversarial Training

The primary defensive technique against PGD attacks. During training, each clean batch is augmented with PGD-generated adversarial examples labeled with the ground-truth class. This forces the model to learn robust decision boundaries within the epsilon-ball around each training point. PGD-based adversarial training, introduced by Madry et al., remains one of the few empirically robust defenses against strong iterative attacks and is considered the standard baseline for robustness research.

Carlini & Wagner (C&W) Attack

An optimization-based attack that formulates adversarial example generation as a constrained optimization problem using a margin-based loss function. Unlike PGD, which operates within a fixed Lp-norm budget, C&W jointly optimizes for minimal perturbation and misclassification. C&W attacks often find smaller perturbations than PGD and can defeat defensive distillation, but are computationally more expensive. PGD is preferred for adversarial training due to its speed and gradient reliability.

Gradient Masking

A defensive phenomenon where a model produces non-useful or zero gradients, causing gradient-based attacks like PGD to fail during optimization. This creates a false sense of security—the model appears robust under white-box PGD evaluation but remains vulnerable to black-box transfer attacks or adaptive attacks that circumvent the gradient obstruction. Proper PGD evaluation must verify that gradients are informative, not masked, to ensure robustness claims are genuine.

Adaptive Attack Evaluation

The rigorous methodology of designing attacks with full knowledge of the defense mechanism to avoid false robustness claims. When evaluating a defense against PGD, researchers must ensure the attack is adapted to any gradient-obfuscating components. For example, if a defense uses non-differentiable preprocessing, the adaptive PGD variant uses Backward Pass Differentiable Approximation (BPDA) to estimate gradients through the obfuscated component. Adaptive PGD is the gold standard for robustness certification.

Lp-Norm Threat Model

The mathematical framework defining the adversarial budget within which PGD operates. Common norms include:

L∞ (Chebyshev): Limits maximum per-pixel change, producing uniformly bounded perturbations
L2 (Euclidean): Limits total perturbation magnitude, allowing some pixels to change more than others
L1 (Manhattan): Encourages sparse perturbations affecting few pixels PGD is typically implemented with L∞ constraints due to its alignment with human perceptual uniformity and computational tractability.

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.