Gradient Masking

Defenses that intentionally introduce non-differentiable operations or numerical instability to prevent gradient computation.

• Temperature scaling with near-zero values causes vanishing gradients
• Hard thresholding or step functions block gradient flow entirely
• Discrete transformations like JPEG compression with non-differentiable quantization tables

Attackers bypass this by using Backward Pass Differentiable Approximation (BPDA) , substituting a smooth proxy for the non-differentiable component during the attack.

Defenses that rely on randomization to make gradients noisy or misleading, hoping to derail optimization-based attacks.

• Random resizing and padding applied before classification
• Stochastic activation pruning that randomly drops neurons
• Randomized input transformations like additive Gaussian noise or random cropping

Adaptive attacks defeat this using Expectation over Transformation (EOT) , computing gradients over the full distribution of random transformations rather than a single sample.

Defenses designed to cause numerical overflow or underflow during gradient computation, making optimization numerically impossible.

• Deeply nested optimization loops that unroll into massive computation graphs
• Extreme weight scaling that pushes gradients toward infinity or zero
• Gradient clipping applied adversarially to truncate useful signal

Attackers counter this by reparameterizing the optimization or using gradient-free black-box attacks that never require explicit gradient computation.

The definitive method for exposing gradient masking is constructing an adaptive attack that accounts for the specific defense mechanism:

• BPDA: Replace non-differentiable components with differentiable approximations during the backward pass only
• EOT: Compute gradients over the expected value of randomized transformations
• Reparameterization: Reformulate the optimization to avoid problematic operations

AutoAttack automates this process with a parameter-free ensemble that includes adaptive white-box and black-box attacks, serving as the standard for empirical robustness evaluation.

The direct mechanism behind gradient masking. A defense exhibits obfuscated gradients when it relies on non-differentiable operations, numerical instability, or shattered gradients to block white-box attacks. This is a brittle failure mode, not a security property. Attackers bypass it using Backward Pass Differentiable Approximation (BPDA) or by simply switching to black-box attacks.

The definitive methodology for exposing gradient masking. An adaptive attack assumes full knowledge of the defense mechanism and customizes the optimization to circumvent it. Key principles:

• Unwrap non-differentiable layers with smooth approximations
• Bypass stochastic defenses by computing expectations over randomness
• Optimize directly against the true end-to-end model If a defense fails against a well-tuned adaptive attack, it relies on gradient masking.

The standard iterative white-box attack that gradient masking defenses attempt to foil. PGD maximizes the loss by taking multiple gradient steps and projecting the perturbation onto an Lp-norm ball. A defense that shows high PGD accuracy but fails against AutoAttack or adaptive methods is a classic indicator of gradient masking. PGD remains the minimum bar for empirical robustness claims.

A parameter-free ensemble of diverse attacks designed to provide a reliable robustness evaluation. AutoAttack combines:

• APGD-CE: Auto-tuned PGD with cross-entropy loss
• APGD-DLR: Auto-tuned PGD with difference-of-logits ratio loss
• FAB: Fast Adaptive Boundary attack
• Square Attack: Query-efficient black-box attack If a model's robustness drops sharply under AutoAttack compared to PGD, gradient masking is the likely cause.

Attacks that require only input-output queries to the model, with no access to internal gradients. Gradient masking is completely ineffective against these methods. Common approaches:

• Score-based: Use output probabilities to estimate gradients via finite differences
• Decision-based: Walk the decision boundary using only hard labels
• Transfer-based: Craft adversarial examples on a surrogate model and transfer them A defense that resists white-box but collapses under black-box attacks exhibits gradient masking.

The gold standard that gradient masking fails to achieve. Certified defenses provide a mathematical guarantee that no adversarial example exists within a defined perturbation radius. Techniques include:

• Randomized smoothing: Probabilistic certificates via noise injection
• Interval bound propagation: Propagating activation bounds through the network
• SMT-based verification: Satisfiability modulo theories for exact proofs Unlike gradient masking, certified robustness cannot be circumvented by a better optimizer.