Adaptive Attack

The adversary explicitly incorporates knowledge of the defense mechanism into the loss function used to generate malicious inputs. Rather than simply maximizing model error, the optimization process penalizes perturbations that are easily detected or neutralized.

• Gradient Obfuscation Bypass: Replaces non-differentiable defensive layers with smooth approximations during the backward pass.
• Expectation over Transformation (EOT): Computes gradients over randomized input transformations to defeat stochastic defenses.
• Backward Pass Differentiable Approximation (BPDA): Approximates the gradient of non-differentiable pre-processing blocks to maintain attack efficacy.

Before launching an attack, the adversary probes the target system to fingerprint the specific defensive strategy in use. This reconnaissance phase determines whether the model uses adversarial training, randomized smoothing, or input quantization.

• Query Analysis: Observes output confidence scores and prediction consistency under slight input variations.
• Gradient Leakage Detection: Tests for obfuscated gradients by checking for shattered gradients or stochastic gradients.
• Decision Boundary Mapping: Reconstructs the local geometry of the classifier to identify non-robust features.

The attack dynamically modifies its perturbation budget, step size, or loss function based on real-time feedback from the target model. If an initial attack vector is blocked, the algorithm automatically pivots to an alternative exploitation path.

• Step-Size Adaptation: Reduces perturbation magnitude if large changes trigger input sanitization filters.
• Loss Function Switching: Transitions from cross-entropy loss to logit margin loss if gradient masking is detected.
• Multi-Targeted Optimization: Simultaneously optimizes for misclassification and evasion of a specific detection system.

Adaptive attacks are evaluated under the most pessimistic security assumption: the adversary has full access to the model's architecture, parameters, and defense logic. This Kerckhoffs's principle for machine learning ensures that security does not rely on obscurity.

• Full Gradient Access: The attacker can compute exact gradients through the entire pipeline, including pre-processing defenses.
• Architecture Replication: The adversary trains a local surrogate model that perfectly mirrors the target's structure.
• Defense Source Code Review: Assumes the attacker has studied the specific implementation details of any protective measures.

Rather than attacking the classifier in isolation, the adversary optimizes the perturbation through the complete inference pipeline, including any pre-processing, denoising, or feature squeezing modules that precede the model.

• Differentiable Pre-Processing: Reimplements JPEG compression or bit-depth reduction as differentiable operations for gradient flow.
• Full-Pipeline Backpropagation: Computes the loss gradient with respect to the raw input, flowing through all defensive layers.
• Thermometer Encoding Bypass: Breaks discretized input defenses by optimizing in the continuous pre-discretization space.

Adaptive attacks serve as the gold standard for empirically evaluating adversarial robustness. A defense is only considered credible if it withstands a properly configured adaptive attack that specifically targets its mechanisms.

• AutoAttack Benchmark: A standardized ensemble that automatically tunes attack hyperparameters against each defense.
• Defense-Specific Tuning: Each attack configuration is tailored to the exact defensive strategy being evaluated.
• Upper Bound on Vulnerability: If an adaptive attack fails, it provides strong evidence that the defense genuinely increases robustness rather than causing gradient masking.