Obfuscated Gradients

Defenses that intentionally introduce non-differentiable operations or numerical instability to prevent gradient computation.

• Temperature scaling: Using a high-temperature softmax that saturates gradients
• Input quantization: Rounding pixel values to discrete bins before classification
• Cropping/resizing: Applying randomized image transforms that are non-differentiable

These techniques are trivially bypassed using Backward Pass Differentiable Approximation (BPDA) , where the attacker substitutes a differentiable proxy for the non-differentiable component during gradient computation.

Defenses that rely on randomized transformations to make gradients noisy or uninformative for the attacker.

• Random resizing and padding: Applying stochastic geometric transforms before inference
• Random noise injection: Adding Gaussian or uniform noise to inputs or activations
• Stochastic activation functions: Using dropout or randomized ReLU variants at test time

Attackers defeat these defenses using Expectation over Transformation (EOT) , which computes the average gradient over multiple samples of the randomization, yielding a stable attack direction.

Defenses that manipulate the loss landscape to make gradients numerically unstable or vanish entirely before reaching the input.

• Deep architectures with saturating activations: Using sigmoid or tanh in very deep networks
• Gradient clipping or scaling: Artificially bounding gradient magnitudes during backward passes
• Defensive distillation: Training a second model on softened probabilities to flatten the loss surface

These are bypassed by reparameterizing the loss function, using logit-space attacks instead of probability-space attacks, or simply increasing the perturbation step size to compensate for small gradients.

A specific subcategory where defenses deliberately cause gradients to explode numerically, preventing optimization convergence.

• Unbounded activation functions: Using exponentials or reciprocals that produce extreme values
• Numerical overflow traps: Crafting operations that produce NaN or Inf gradients at decision boundaries
• Loss surface discontinuities: Introducing sharp cliffs in the optimization landscape

Adaptive attackers handle this by clipping gradients during optimization, using gradient sign rather than magnitude, or employing evolutionary strategies that do not require gradient information at all.

The primary diagnostic for identifying obfuscated gradients is observing that black-box attacks succeed while white-box attacks fail.

• Transfer attack test: If adversarial examples from an undefended surrogate model fool the defended model, gradients are likely obfuscated
• Unbounded attack test: If allowing arbitrarily large perturbations still fails to produce misclassification, the defense is likely broken
• Increasing distortion budget: If success rate plateaus below 100% even with large epsilon values, gradient masking is present

A defense is only considered robust if it withstands adaptive white-box attacks that specifically target the defense mechanism.

The broader phenomenon where a defense provides a false sense of security by obscuring the model's gradients rather than removing the underlying vulnerability. Obfuscated gradients are a specific implementation of gradient masking. Attackers bypass this by using black-box attacks or adaptive attacks that approximate or circumvent the masked gradient signal.

An attack methodology specifically designed to circumvent known defense mechanisms by adapting to the defensive strategy. When evaluating a defense that relies on obfuscated gradients, an adaptive attacker will:

• Use Backward Pass Differentiable Approximation (BPDA) to replace non-differentiable operations
• Compute gradients via finite differences or evolutionary strategies
• Target the underlying model logic rather than the gradient barrier

A multi-step iterative white-box attack that generates adversarial examples by projecting perturbations onto an Lp-norm ball. PGD is the standard benchmark for empirical robustness. Defenses relying on obfuscated gradients often show high PGD accuracy in isolation but collapse under AutoAttack, which combines PGD with complementary black-box methods.

A standardized, parameter-free ensemble of attacks used to reliably evaluate empirical robustness. AutoAttack combines:

• APGD-CE: Adaptive PGD with cross-entropy loss
• APGD-DLR: Adaptive PGD with difference of logits ratio
• FAB: Fast Adaptive Boundary attack
• Square Attack: A query-efficient black-box attack This ensemble reliably exposes obfuscated gradient defenses that fool single-attack evaluations.

An adversarial attack that relies solely on querying a model's outputs without access to internal parameters or gradients. Obfuscated gradients are completely ineffective against black-box attacks because these methods never rely on gradient computation. Common approaches include:

• Score-based: Using output probabilities to guide perturbation
• Decision-based: Using only hard-label predictions
• Transfer-based: Crafting examples on a surrogate model

A formal guarantee that a model's prediction remains constant for all inputs within a mathematically defined perturbation bound. Unlike empirical defenses that may rely on obfuscated gradients, certified methods provide provable guarantees against adversarial manipulation. Techniques include randomized smoothing and interval bound propagation, which offer verifiable lower bounds on robustness.