Certified Robustness

The dominant technique for achieving certified robustness in large models. It constructs a smoothed classifier g from any base model f:

• g(x) returns the most likely prediction of f when x is corrupted with isotropic Gaussian noise.
• The prediction is certified by estimating the probability of the top class under this noise distribution using Clopper-Pearson confidence intervals.
• The certified radius R is proportional to the margin between the top class probability and the runner-up.

A fundamental tension exists between standard accuracy and certified robustness. Training for tight certificates often degrades performance on clean, unperturbed data. Key dynamics:

• Gaussian data augmentation improves certified accuracy but reduces standard accuracy.
• TRADES and similar loss functions explicitly balance the two objectives by regularizing the Kullback-Leibler divergence between clean and noisy predictions.
• The trade-off is not a failure of technique but a property of the data distribution itself, as demonstrated by Tsipras et al.

An extension of randomized smoothing that prepends a custom-trained denoiser D to the base classifier: g(x) = f(D(x + δ)). The denoiser removes the Gaussian noise before classification, allowing the base model f to operate on cleaner inputs without retraining. This significantly boosts certified accuracy on ImageNet by decoupling the noise-robustness requirement from the classifier's feature extraction pipeline.

The primary technique for constructing certifiably robust classifiers from any base model. It works by adding isotropic Gaussian noise to inputs and returning the most probable prediction under that noise distribution.

• Provides a probabilistic guarantee of prediction stability within a certified L2 radius
• Transforms a deterministic classifier into a smoothed classifier with formal robustness bounds
• The certified radius depends on the margin between the top class probability and the runner-up
• Scales to large architectures like ResNet and ViT without architectural modification

An iterative, multi-step adversarial attack that projects perturbed examples back onto an epsilon-ball around the original input at each step. It represents the strongest first-order white-box adversary.

• Maximizes the loss function under an L-infinity norm constraint
• Serves as the standard empirical benchmark against which certified defenses are measured
• Certified robustness provides a lower bound that PGD attacks cannot violate
• Used extensively in adversarial training to harden models

A defensive methodology that augments training data with adversarial examples generated on-the-fly. The model learns to correctly classify perturbed inputs, creating a more robust decision boundary.

• Typically uses PGD attacks during each training iteration
• Empirically effective but provides no formal guarantee of robustness
• Certified methods like randomized smoothing offer provable guarantees that adversarial training cannot
• Often combined with certified approaches for defense-in-depth strategies

A parameter-free ensemble of diverse white-box and black-box attacks used as the standard benchmark for evaluating empirical adversarial robustness. It complements certified evaluations.

• Combines APGD-CE, APGD-DLR, FAB, and Square Attack
• Provides a reliable upper bound on empirical vulnerability
• The gap between AutoAttack success and certified guarantees reveals the tightness of certification
• Used in RobustBench leaderboard evaluations