Randomized Smoothing

Randomized smoothing provides a certified radius within which no adversarial perturbation can change the prediction. Unlike empirical defenses that can be broken by adaptive attacks, this guarantee is mathematically provable and holds for any attack strategy.

• The certified radius scales with the margin between top class probabilities
• Larger Gaussian noise increases the radius but reduces clean accuracy
• Guarantee is attack-agnostic—no assumptions about adversary capabilities
• Formal verification derived from the Neyman-Pearson lemma

The core mechanism adds isotropic Gaussian noise with variance σ² to each input before classification. The smoothed classifier g(x) returns the most probable prediction when inputs are sampled from the Gaussian distribution centered at x.

• Noise level σ is a hyperparameter trading off robustness vs. accuracy
• Sampling typically uses 10,000 to 100,000 Monte Carlo samples for certification
• The smoothed classifier is Lipschitz continuous with respect to L2 distance
• Works with any base classifier architecture without modification

During inference, the smoothed classifier aggregates predictions across multiple noise-perturbed copies of the input. The majority vote determines the final class, and the margin of victory determines the certified radius.

• Prediction phase: Sample n noisy copies, take majority class
• Certification phase: Estimate lower bound on top class probability using Clopper-Pearson confidence intervals
• Higher sample counts yield tighter probability bounds and larger certified radii
• Procedure is embarrassingly parallel and amenable to batch GPU processing

Randomized smoothing wraps any base classifier as a black box, requiring no architectural changes, gradient access, or retraining. This makes it directly applicable to large-scale vision models and even foundation models.

• Compatible with ResNet, Vision Transformers, and CLIP architectures
• No need for adversarial training or modified loss functions
• Certification cost scales with inference cost × number of samples
• Denoised smoothing variants combine denoisers with smoothing for improved accuracy

Despite its provable guarantees, randomized smoothing faces practical challenges. The certified radii are often smaller than empirical robustness from adversarial training, and the L2 threat model does not capture all real-world attacks.

• Computational cost: Thousands of forward passes per input for certification
• L2-only guarantee: Does not certify against L∞, L1, or non-Lp attacks
• Accuracy-robustness trade-off: Higher noise σ degrades clean accuracy
• Semantic transformations like rotation or lighting changes are not covered

Research has extended randomized smoothing beyond L2 certification to other threat models and improved its efficiency. Key variants address the original method's limitations while preserving provable guarantees.

• SmoothAdv: Combines smoothing with adversarial training for tighter bounds
• MACER: Maximizes certified radius during training via robust optimization
• Consistency regularization: Enforces prediction consistency under noise
• De-randomized smoothing: Uses ablative defenses for L0 and L1 certification
• Dual normalization: Extends guarantees to semantic transformations

A formal, provable guarantee that a model's prediction will not change for any input perturbation within a specified Lp-norm bound. Unlike empirical defenses that can be broken by stronger attacks, certified methods provide a mathematical lower bound on the adversarial radius around each input. Randomized smoothing is the most scalable approach to achieve this for large neural networks, producing a certificate that holds with high probability.

A defensive technique that augments the training dataset with adversarially perturbed examples labeled with the ground-truth class. By training on worst-case inputs generated via attacks like Projected Gradient Descent (PGD), the model learns robust decision boundaries. While adversarial training improves empirical robustness, it does not provide formal guarantees—models can still be broken by adaptive attacks specifically designed to circumvent the defense.

A phenomenon where a defense gives a false sense of security by producing non-useful or zero gradients, preventing gradient-based attacks from optimizing. This often occurs through shattered gradients (non-differentiable operations) or stochastic gradients (randomized defenses). Critically, gradient masking does not indicate true robustness—models remain vulnerable to black-box attacks and transfer attacks. Randomized smoothing avoids this pitfall by providing verifiable guarantees rather than obscuring gradients.

An attack methodology that assumes full knowledge of a defense mechanism and is specifically designed to circumvent it. When evaluating randomized smoothing, adaptive attacks may target the sampling procedure itself or exploit the gap between the empirical and certified accuracy. This represents the most rigorous evaluation standard—any defense not tested against adaptive attacks cannot be considered reliable. Key techniques include:

• Expectation Over Transformation (EOT) for stochastic defenses
• Attacking the smoothed classifier directly rather than the base model

An extension of randomized smoothing that prepends a custom-trained denoiser before the base classifier, rather than relying solely on Gaussian noise augmentation. The denoiser attempts to remove adversarial perturbations, allowing the smoothed classifier to achieve higher certified accuracy on non-adversarial inputs while maintaining provable guarantees. This addresses a key limitation of standard randomized smoothing: the accuracy-robustness trade-off where clean accuracy degrades as the certification radius increases.

A powerful iterative white-box adversarial attack that repeatedly takes a small step in the direction of the loss gradient and projects the result back onto an epsilon-ball around the original input. PGD is the standard benchmark for evaluating empirical adversarial robustness and is often used as the inner maximization step during adversarial training. While randomized smoothing provides certified defense against all attacks within the radius, PGD serves as a complementary empirical evaluation tool for measuring worst-case accuracy.