Robust Accuracy

Robust accuracy is measured on a perturbed dataset, not the standard validation set. This requires generating adversarial examples using attacks like PGD or FGSM against the model under test. The strength of the adversary—defined by the epsilon perturbation budget—directly controls the difficulty. A larger epsilon allows more visible distortions, testing a wider radius of the model's decision boundary. The test set must be generated from the original clean test set to ensure a direct, sample-by-sample comparison between natural and robust accuracy.

A fundamental tension exists between natural accuracy (on clean data) and robust accuracy (on adversarial data). Training for robustness often forces the model to learn simpler, more linear decision boundaries, discarding useful but brittle features. This typically results in a drop in clean-data performance. The Pareto frontier of this trade-off is a key research focus. Techniques like TRADES explicitly optimize a loss function that balances these two competing objectives, allowing practitioners to select a desired operating point based on their risk tolerance.

A robust accuracy score is meaningless without specifying the threat model used for evaluation. This includes:

• Attack Algorithm: PGD, C&W, or AutoAttack.
• Perturbation Norm: L∞ (max pixel change), L2 (Euclidean distance), or L1.
• Budget (ε): The maximum allowed perturbation magnitude.
• Iterations: Number of steps for iterative attacks. A model claiming 80% robust accuracy against a weak single-step FGSM attack may collapse to 0% against a strong multi-step PGD attack with a random restart. Standardized benchmarks like RobustBench enforce a consistent evaluation protocol to prevent this type of inflated claim.

Empirical robust accuracy is measured by attacking a model and checking if it fails. It provides an upper bound on the true risk—a clever, undiscovered attack could always break the model. Certified robust accuracy provides a mathematical lower bound, guaranteeing that no perturbation within a defined norm-ball can change the prediction. Techniques like randomized smoothing produce these certificates. A certified accuracy of 60% at radius r=0.5 means the model is provably correct for at least 60% of the test set against any attack of that size, offering a much stronger security guarantee.

A critical test of robust accuracy is its generalization to unseen attacks. A model adversarially trained solely on L∞ PGD examples may remain vulnerable to L2 C&W attacks or spatial transformations. True robustness requires the learned decision boundary to be smooth across all potential perturbation directions. AutoAttack, an ensemble of diverse attacks including targeted and untargeted variants, is now the standard for evaluating this generalization. A high AutoAttack score is a strong signal that the model has not simply overfit to the specific defense used during training.

Beyond a security metric, robust accuracy serves as a powerful regularizer that improves feature representation. Adversarial training forces models to rely on robust features that align with human perception, rather than brittle, high-frequency patterns. This has been shown to:

• Improve model interpretability, as gradients highlight semantically meaningful object boundaries.
• Enable better transfer learning to downstream tasks.
• Increase resilience to common corruptions like Gaussian noise or blur, not just adversarial attacks. In this view, optimizing for robust accuracy is a method for learning fundamentally better internal representations.