Adversarial Example

The defining feature of an adversarial example is that the added perturbation is imperceptible to the human visual or auditory system. The modification is constrained by an Lp-norm distance metric (often L∞ or L2) to ensure the adversarial input appears identical to the original clean sample. For instance, changing a pixel value by a magnitude of 1/255th on an 8-bit image scale is invisible to a human but can drastically alter a neural network's feature activation map.

An adversarial example does not merely cause a random error; it forces the model to output a specific, incorrect class with extremely high confidence. A 'stop sign' adversarial example might cause a classifier to output 'speed limit 80 mph' with 99.9% probability. This high confidence indicates the input has crossed a decision boundary into a region the model strongly associates with the target class, making simple thresholding defenses ineffective.

Adversarial examples exhibit the property of transferability, where a perturbation generated to fool one model (the surrogate) often fools another independently trained model (the victim) with a different architecture. This occurs because different models learn similar decision boundary geometries when trained on the same task. This property enables practical black-box attacks where the attacker has no direct access to the target model's gradients or parameters.

The phenomenon is largely explained by the linear nature of neural networks in high-dimensional input spaces. As articulated by Goodfellow et al., a small perturbation multiplied by the high dimensionality of the input (e.g., millions of pixels) can accumulate to create a massive change in the output logit. The model's local linearity makes it susceptible to perturbations precisely aligned with the sign of the cost function's gradient.

Adversarial examples are not confined to the digital domain. Robust physical perturbations can be manufactured and deployed in the real world. Researchers have demonstrated that printed adversarial patches or carefully crafted 3D-printed objects can consistently fool real-world classifiers under varying lighting conditions, angles, and camera distances, proving the threat extends to autonomous systems and physical security.

Adversarial examples exploit non-robust features—patterns in the data that are highly predictive for the model but are incomprehensible and uncorrelated with the true class for humans. The model latches onto these subtle, brittle statistical correlations in the pixel space. An adversarial perturbation effectively flips the value of these non-robust features to the target class's distribution while leaving the robust, human-perceptible features unchanged.

An attack executed with no knowledge of the target model's architecture, parameters, or training data. The attacker relies solely on querying the model to observe input-output pairs. Key strategies include:

• Score-based: Using output probabilities to estimate gradients via finite differences.
• Decision-based: Using only the hard-label prediction to walk the decision boundary.
• Transfer-based: Crafting examples on a local surrogate model and exploiting transferability to fool the target.

The process of formally proving that a model's prediction for a given input is invariant to any perturbation within a defined Lp-norm ball. Unlike empirical defenses, certification provides a mathematical guarantee. Key methods include:

• Randomized Smoothing: A probabilistic certification that constructs a provably robust classifier by adding Gaussian noise and taking a majority vote.
• Abstract Interpretation: Propagating symbolic bounds through the network to verify output stability.