AutoAttack

AutoAttack combines four diverse attack algorithms into a single, standardized evaluation protocol that requires no manual tuning. The ensemble includes:

• APGD-CE: Auto-PGD on the cross-entropy loss
• APGD-DLR: Auto-PGD on the difference of logits ratio loss
• FAB-T: Fast Adaptive Boundary attack, targeted variant
• Square Attack: A query-efficient black-box attack This diversity prevents defenses from overfitting to a single attack type, ensuring a holistic robustness assessment.

Unlike standard PGD attacks that require manual step-size tuning, AutoAttack's APGD component dynamically adjusts the step size during optimization based on the optimization trajectory. It uses a momentum-based update rule and automatically halves the step size when the loss plateaus, ensuring convergence without human intervention. This eliminates the common pitfall of gradient masking defenses exploiting poorly tuned attacks.

AutoAttack has become the de facto standard for evaluating defenses on benchmarks like RobustBench. Its reliability stems from:

• No hyperparameter tuning: Eliminates human bias in attack configuration
• Complementary attack types: White-box gradient attacks + black-box query attacks
• Adaptive evaluation: Designed to counter common obfuscated gradient defenses A defense that withstands AutoAttack is considered genuinely robust, not just resilient to a specific, poorly-configured attack.

AutoAttack supports two canonical evaluation settings:

• L∞ (L-infinity): Bounded perturbations where each pixel or feature can change by at most ε, typically ε = 8/255 for CIFAR-10
• L2: Euclidean distance-bounded perturbations, typically ε = 0.5 for CIFAR-10 Both settings use a fixed budget across all samples, enabling fair, apples-to-apples comparisons between different defense mechanisms published in the literature.

AutoAttack is the official evaluation engine behind RobustBench, the leading leaderboard for adversarial robustness research. RobustBench enforces a strict protocol:

• Defenses are evaluated exclusively with AutoAttack
• Results are reproducible with standardized threat models
• Leaderboard rankings are based on robust accuracy under AutoAttack This integration has driven the community toward more honest and comparable robustness claims since 2020.

AutoAttack is designed to counter gradient masking and obfuscated gradients, a class of defenses that appear robust only because gradient-based attacks fail. The inclusion of Square Attack, a score-based black-box method that does not rely on gradients, ensures that defenses relying on shattered or stochastic gradients are still thoroughly tested. This makes AutoAttack a strong proxy for an adaptive adversary with full knowledge of the defense.

A carefully crafted, often imperceptible modification to an input sample designed to cause a machine learning model to misclassify it. In financial fraud detection, perturbations might involve subtly altering transaction amounts, timestamps, or merchant categories to evade anomaly scoring while preserving the fraudulent transaction's functionality.

• Lp-norm constraints bound perturbation magnitude (e.g., L∞ ≤ 8/255 for images)
• Feature-space attacks modify tabular data columns like velocity counters or geolocation features
• Semantic perturbations preserve malicious intent while changing surface features

A white-box attack methodology specifically designed with full knowledge of a defense mechanism to circumvent it. Adaptive attacks serve as the gold standard for robustness evaluation and are central to AutoAttack's design philosophy.

• Assumes attacker knows defense architecture, parameters, and training procedure
• AutoAttack includes APGD-CE and APGD-DLR as adaptive variants
• Defenses relying on gradient masking or obfuscation consistently fail against adaptive attacks
• Essential for evaluating fraud models against sophisticated adversaries who understand detection logic

A defensive technique that augments the training dataset with adversarial examples to improve a model's robustness against evasion attacks. This is the most empirically validated defense against adversarial perturbations.

• PGD-based training generates adversarial examples during each minibatch
• TRADES loss balances natural accuracy with adversarial robustness via KL divergence regularization
• For financial models, adversarial training hardens fraud detectors against transaction manipulation
• Computational overhead is significant: 3-10x training time increase

A formal, mathematical guarantee that a model's prediction will remain constant for any input perturbation within a defined radius. Unlike empirical defenses, certification provides provable security bounds.

• Randomized smoothing constructs a smoothed classifier by adding Gaussian noise and using majority voting
• Certification radius depends on noise level and base classifier's margin
• Provides lower bounds on robustness that AutoAttack's empirical evaluation can validate
• Critical for regulated financial applications requiring guaranteed fraud detection stability

A defensive phenomenon where a model's gradients are useless for generating attacks, often providing a false sense of security against adaptive adversaries. AutoAttack explicitly detects and defeats gradient masking.

• Shattered gradients: Non-differentiable operations or numerical instability
• Stochastic gradients: Randomization that prevents reliable gradient estimation
• Vanishing/exploding gradients: Poor gradient flow through deep architectures
• AutoAttack's APGD variant with expectation over transformation bypasses stochastic defenses
• Financial models using discretization or rule-based components are particularly susceptible