Gradient Masking

The gradient signal is intentionally broken or made non-existent, preventing an attacker from computing a useful direction for perturbation.

• Non-differentiable layers: Introducing hard thresholding or quantization operations that have zero or undefined gradients.
• Numerical instability: Defenses that cause gradient explosion or vanishing, making optimization impossible.
• Example: A defense that rounds input pixels to discrete values before classification destroys the gradient path, blocking FGSM and PGD but remaining vulnerable to decision-based attacks like the Boundary Attack.

The model intentionally randomizes its predictions or gradients, making the loss surface too noisy for an attacker to optimize against.

• Randomized transformations: Applying random resizing, padding, or noise to inputs before classification.
• Stochastic activation functions: Using dropout or noisy ReLU variants at inference time.
• Example: A defense that randomly rotates and crops inputs before classification forces an attacker to optimize over an expectation using Expectation over Transformation (EoT), significantly increasing attack cost but not providing certified robustness.

The defense architecture causes gradients to become numerically unstable, either approaching zero or growing unboundedly during backpropagation.

• Saturation defenses: Training models to operate in the flat saturation regions of activation functions where gradients are near zero.
• Deep contractive networks: Architectures that intentionally squash gradient magnitudes through many layers.
• Example: Defensive Distillation trains a model at high temperature, causing softmax outputs to saturate and producing gradients so small that white-box attacks like FGSM fail to find adversarial directions.

Gradient masking is a brittle defense that often fails catastrophically against attacks that do not rely on local gradient information.

• Black-box transfer attacks: Adversaries can train a surrogate model without obfuscated gradients and transfer attacks to the target.
• Score-based attacks: Methods like SPSA and NES estimate gradients through finite differences using only confidence scores.
• Decision-based attacks: The Boundary Attack and HopSkipJumpAttack require only hard-label predictions, completely bypassing gradient obfuscation.
• Key insight: A truly robust model should have clean, well-behaved gradients that lead to small, meaningful perturbations—not broken ones.

Security researchers can identify gradient masking by running diagnostic checks that reveal whether a defense is genuinely robust or merely obfuscating.

• One-step vs. iterative attack comparison: If a single-step attack like FGSM fails but a multi-step PGD with random restarts also fails, the defense may be masking gradients.
• Distortion vs. epsilon analysis: Increasing the perturbation budget should increase attack success; if it plateaus abruptly, gradients are likely shattered.
• Transferability test: Attacks generated on an undefended copy of the model should transfer to the defended version if the defense relies on masking.
• Carlini-Wagner sanity check: The C&W attack with a properly tuned loss function often bypasses gradient masking defenses that fool simpler attacks.

Several published defenses have been shown to rely primarily on gradient masking rather than true adversarial robustness.

• Thermometer encoding: Discretizing inputs into binary vectors breaks gradient flow but is defeated by BPDA (Backward Pass Differentiable Approximation).
• Input transformations: JPEG compression, total variance minimization, and quilting create non-differentiable preprocessing pipelines.
• Generative model defenses: Using GANs or autoencoders to project inputs onto a clean manifold often creates obfuscated gradients.
• Adversarial logit pairing: Penalizing differences between clean and adversarial logits was shown by Athalye et al. (2018) to cause gradient masking rather than true robustness.