Adversarial Robustness

A smoothness constraint bounding how rapidly a model's output can change relative to its input. For a classifier to be robust, small perturbations in the input space must not cause disproportionate shifts in the output. Lipschitz constant quantifies this sensitivity—lower constants indicate higher robustness. Enforcing Lipschitz continuity through spectral normalization or gradient penalty during training creates models where adversarial examples are harder to construct because the decision boundary cannot fold sharply around data points.

The maximum perturbation magnitude within which a model's prediction is provably guaranteed to remain unchanged. Unlike empirical robustness, which only measures performance against known attacks, certified defenses use techniques like randomized smoothing to construct a statistical envelope around each prediction. A larger certified radius directly translates to stronger adversarial guarantees—critical for regulatory submissions where worst-case behavior must be mathematically bounded rather than experimentally estimated.

The process of augmenting training data with on-the-fly generated adversarial examples to harden the model. Modern implementations use Projected Gradient Descent (PGD) to craft perturbations that maximize loss within an epsilon-ball. Key properties include:

• Min-max formulation: Inner maximization crafts attacks; outer minimization updates weights
• Catastrophic overfitting: A failure mode where the model memorizes specific attack patterns rather than learning robust features
• Trade-off with clean accuracy: Robust models typically sacrifice 5-15% accuracy on unperturbed data

A diagnostic property that distinguishes genuine robustness from obfuscated gradients—a false sense of security where attacks fail because gradient signals are destroyed, not because the model is truly robust. Indicators include:

• Shattered gradients: Non-differentiable operations or numerical instability blocking gradient flow
• Stochastic gradients: Randomized defenses that attackers circumvent by taking expectation over randomness
• Exploding/vanishing gradients: Extreme curvature preventing iterative attacks from converging Proper robustness evaluation must verify that gradient-based attacks actually optimize the adversarial objective.

The property that adversarial examples crafted against one model should not transfer to an independently trained model of the same architecture. High transferability indicates that the perturbation exploits universal blind spots in the learning algorithm rather than model-specific weaknesses. Robust models exhibit low transferability because their decision boundaries align with human-perceptible features. This property is measured by cross-model attack success rate and serves as a proxy for whether the model has learned fundamentally robust representations.

The relationship between input dimensionality and the minimum perturbation required to change a prediction. In high-dimensional spaces typical of medical imaging, the volume of the adversarial subspace grows exponentially with dimension. Key scaling properties:

• Concentration of measure: Most of an epsilon-ball's volume concentrates near its surface in high dimensions
• Curse of dimensionality for robustness: The perturbation budget required to defend a naive classifier scales poorly with input dimension
• Manifold hypothesis leverage: Robust models exploit that natural data lies on a low-dimensional manifold, constraining the effective attack surface

Quantitative measures that assess how accurately an explanation reflects the model's true reasoning. Faithfulness is the bridge between explainability and robustness—an explanation is faithful if the features it highlights genuinely drive the prediction.

• Sufficiency: Does removing highlighted features change the prediction?
• Comprehensiveness: Are the highlighted features alone enough for the prediction?
• Low faithfulness often correlates with susceptibility to adversarial attacks

The task of identifying inputs statistically different from training data. OOD detection serves as a first line of defense against adversarial examples, which inherently lie off the training manifold. Techniques include:

• Mahalanobis distance in feature space
• Energy-based models that score input typicality
• Gaussian mixture models on penultimate layer activations

Robust OOD detection prevents a model from making high-confidence errors on manipulated inputs.