Randomized Smoothing

The foundational mechanism of randomized smoothing involves adding isotropic Gaussian noise to the input before classification. This creates a smoothed classifier that returns the most likely prediction under the noise distribution.

• The noise level σ (sigma) is a critical hyperparameter controlling the robustness/accuracy trade-off
• Higher σ values provide larger certified radii but degrade standard accuracy
• The smoothed classifier is defined as: g(x) = argmax P(f(x + ε) = c) where ε ~ N(0, σ²I)
• This probabilistic wrapper transforms any base classifier into a certifiably robust one without modifying its architecture

Certification relies on Monte Carlo sampling to estimate the probability of the top class under noise. The Neyman-Pearson lemma provides the theoretical foundation for deriving a guaranteed radius.

• The procedure samples n noisy copies of the input and counts class predictions
• A lower confidence bound on the top class probability pA is computed using Clopper-Pearson intervals
• The certified radius R is derived as: R = σ/2 * (Φ⁻¹(pA) - Φ⁻¹(pB)) where Φ⁻¹ is the inverse Gaussian CDF
• Increasing sample count n tightens the confidence bound, yielding larger certified radii
• Typical values: n = 100,000 for certification, n = 100 for prediction

Randomized smoothing provides provable L2-norm robustness guarantees. If the smoothed classifier predicts class cA at input x, the prediction remains constant for all perturbations within the certified radius.

• The guarantee is deterministic and mathematically provable, not empirical
• The certified radius is a lower bound; actual robustness may be larger
• The L2 metric measures Euclidean distance between original and perturbed inputs
• Extensions exist for L1 and L∞ norms using different noise distributions (Laplace, Uniform)
• This contrasts with empirical defenses like adversarial training, which offer no formal guarantees

The smoothed classifier can abstain from prediction when the top two classes are statistically indistinguishable under noise. This is a critical safety mechanism for high-stakes applications.

• If the confidence interval for pA overlaps with pB, the classifier abstains
• Abstention prevents overconfident misclassifications on ambiguous or out-of-distribution inputs
• The hypothesis test uses a significance level α (typically 0.001) to control false positive rates
• This mechanism naturally integrates with human-in-the-loop workflows for uncertain cases
• The abstention rate increases near decision boundaries, providing a calibrated uncertainty signal

Randomized smoothing operates as a wrapper around any base classifier, requiring no architectural modifications or retraining. This makes it uniquely practical for production deployment.

• Compatible with any differentiable model: CNNs, Transformers, ResNets, ViTs
• No adversarial training required; the base model can be trained on clean data only
• The defense is orthogonal to other robustness techniques and can be combined with adversarial training
• Certification is performed at inference time, not during training
• Trade-off: inference latency increases linearly with the number of Monte Carlo samples

Standard randomized smoothing suffers from the accuracy-robustness trade-off at high noise levels. Denoised smoothing addresses this by prepending a denoiser before the base classifier.

• A custom-trained denoiser removes Gaussian noise before classification
• This allows the base classifier to operate on near-clean inputs while maintaining certification
• The denoiser must be Lipschitz-constrained to preserve the certification guarantees
• Techniques include stability training and using denoising score matching objectives
• Denoised smoothing significantly improves certified accuracy at large radii compared to standard smoothing