Adversarial Robustness

The measured accuracy of a model against a specific, known set of adversarial attacks. It is a practical, non-guaranteed metric.

• White-box attacks: Evaluated against an attacker with full knowledge of model parameters (e.g., Projected Gradient Descent).
• Black-box attacks: Evaluated against an attacker who only queries the model (e.g., Square Attack).
• Limitation: High empirical robustness against one attack does not guarantee defense against a stronger, unseen adaptive adversary.

A formal, mathematical guarantee that a model's prediction will remain constant for any input perturbation within a defined Lp-norm bound (e.g., an epsilon radius).

• Randomized Smoothing: The dominant technique for creating a certifiably robust classifier by aggregating predictions on noise-corrupted copies of an input.
• Deterministic Certification: Uses methods like interval bound propagation to prove robustness without randomness.
• Provides a lower bound on accuracy that no attacker can violate within the specified threat model.

A training-time defense that injects adversarial examples into the training dataset to harden the model's decision boundaries.

• Min-Max Optimization: Formulated as a saddle point problem that minimizes loss against a maximally perturbed input.
• Single-Step vs. Multi-Step: Fast Gradient Sign Method (FGSM) is fast but weak; Projected Gradient Descent (PGD) is the gold standard for robustness but computationally expensive.
• Trade-off: Often reduces accuracy on clean, unperturbed data in exchange for robustness.

A brittle and often illusory defense where a model's gradients are shattered or obfuscated to prevent an attacker from computing a useful perturbation.

• Shattered Gradients: Caused by non-differentiable operations like input quantization or thermometer encoding.
• Stochastic Gradients: Introduced by random transformations applied to the input before classification.
• Warning: This is a form of security through obscurity. Attackers bypass it using black-box attacks, transfer attacks, or by computing gradients through a differentiable approximation of the defense.

A lightweight detection method that reduces the complexity of the input feature space to expose adversarial perturbations.

• Bit Depth Reduction: Compresses the color depth of an image (e.g., from 8-bit to 4-bit) to eliminate subtle adversarial noise.
• Spatial Smoothing: Applies a median or Gaussian filter to remove high-frequency perturbations.
• Detection Logic: Compares the model's prediction on the original input versus the squeezed input. A significant discrepancy flags the sample as adversarial.

A robustness technique that trains a second 'student' model on the soft probability outputs (logits) of a pre-trained 'teacher' model, rather than on hard labels.

• Temperature Scaling: The teacher's logits are divided by a high temperature constant (T) to soften the probability distribution, revealing class similarities.
• Mechanism: This process smooths the model's decision surface, making it less sensitive to small input variations in directions orthogonal to the training data manifold.
• Limitation: Proven to be vulnerable to sophisticated attacks that craft adversarial examples using the distilled model's own gradients.