Fast Gradient Sign Method (FGSM)

FGSM is a one-shot attack. It computes the gradient of the loss function with respect to the input only once and applies a perturbation immediately. This contrasts sharply with iterative methods like Projected Gradient Descent (PGD), which refine the perturbation over multiple steps.

• Mechanism: x_adv = x + ε * sign(∇x J(θ, x, y))
• Result: Extremely fast to compute, making it ideal for adversarial training where speed is critical.
• Limitation: The single-step nature often leads to weaker attacks compared to multi-step variants.

FGSM operates under an L∞ (max-norm) constraint, meaning it limits the maximum change to any single pixel, rather than the total number of pixels changed. This ensures the perturbation is uniformly distributed across the input.

• Epsilon (ε): A hyperparameter defining the maximum perturbation magnitude per pixel.
• Visual Effect: For images, a small ε creates a perturbation that is often imperceptible to the human eye but devastating to a classifier.
• Comparison: Unlike L2 attacks which minimize total Euclidean distance, FGSM focuses on keeping the peak change small.

Rather than using the precise gradient values, FGSM only uses their sign (+1 or -1). This linearizes the attack in a high-dimensional space, exploiting the locally linear nature of deep neural networks.

• Why the sign? In high dimensions, the sign of the gradient provides the optimal direction for increasing loss under an L∞ constraint.
• Linearity Hypothesis: Goodfellow et al. proposed that the vulnerability to FGSM stems from the excessive linearity of models built with ReLU activations and LSTMs.
• Impact: This design choice makes the perturbation computationally trivial to compute.

FGSM is a white-box attack, meaning it requires full access to the target model's architecture, weights, and gradients. The attacker must be able to compute the loss gradient via standard backpropagation.

• Access Required: Complete knowledge of model parameters (θ) and training procedure.
• Practical Scenario: Often used in security audits where the defender tests their own model's resilience.
• Transferability: Adversarial examples generated via FGSM on one model often transfer to other independently trained models, enabling black-box attacks in practice.

FGSM is a cornerstone of adversarial training, a primary defense technique. By injecting FGSM-generated examples into the training set, models learn to resist simple gradient-based perturbations.

• Fast Adversarial Training: Uses single-step FGSM to update both the perturbation and model weights in one pass, drastically reducing training time.
• Catastrophic Overfitting: A known failure mode where models trained solely on FGSM examples suddenly lose all robustness against multi-step attacks like PGD.
• Defensive Utility: Despite its simplicity, FGSM-based training provides a non-trivial baseline of robustness.

FGSM can be deployed in two distinct modes depending on the adversary's goal.

• Non-Targeted FGSM: Maximizes the loss for the true class label to cause any misclassification. The formula is x + ε * sign(∇x J(θ, x, y_true)).
• Targeted FGSM: Minimizes the loss for a specific target class to force a desired misclassification. The formula becomes x - ε * sign(∇x J(θ, x, y_target)).
• Use Case: Targeted attacks are used to create specific backdoors or force critical errors in systems like autonomous driving (e.g., turning a stop sign into a speed limit sign).

The iterative, multi-step successor to FGSM. Instead of a single large step, PGD takes many small steps, each time projecting the result back onto the epsilon-ball to stay within the allowed perturbation budget. It is widely considered the strongest first-order adversary and the standard benchmark for evaluating empirical robustness. Mathematically, it solves the constrained optimization problem that FGSM only approximates.

The primary defensive countermeasure against FGSM and PGD. This technique injects adversarial examples directly into the training loop. The model is trained to minimize the loss on these perturbed samples, effectively hardening the decision boundary. Key variants include:

• Standard Adversarial Training: Uses PGD attacks during training.
• TRADES: Explicitly balances the trade-off between clean accuracy and robustness by adding a regularization term that penalizes unstable outputs.

A brittle and often defeated defense that FGSM can expose. Instead of making the model truly robust, gradient masking obfuscates the loss landscape to prevent gradient-based attacks from working. Examples include shattered gradients (non-differentiable layers) and stochastic gradients (random transformations). However, these defenses fail against black-box attacks or transfer attacks, providing a false sense of security.

The mathematical boundary that defines FGSM's attack budget. The L-infinity norm measures the maximum absolute change to any single pixel. An epsilon of 8/255 means no pixel changes by more than 8 intensity values on a 0-255 scale. This creates a hypercube around the original image, and FGSM moves to the corner of this hypercube in one step. This constraint is perceptually aligned with uniform pixel noise.

A standardized, parameter-free ensemble of attacks used to reliably evaluate defenses against FGSM-style adversaries. It combines APGD (an auto-tuned PGD variant) with black-box attacks like Square Attack to bypass gradient masking. If a defense claims robustness but fails against AutoAttack, it is likely relying on obfuscated gradients. It is the current gold standard for empirical evaluation.

Unlike FGSM's empirical approach, this technique provides a mathematical guarantee that a prediction will not change within a specific L2 radius. It constructs a smoothed classifier by adding Gaussian noise to inputs and returning the most probable prediction. While FGSM finds a specific adversarial point, randomized smoothing proves that no such point exists within a certified radius, offering a provable lower bound on robustness.