Evasion Attack

The defining characteristic of an evasion attack is that it occurs strictly during the test phase. The adversary modifies input samples after a model has been trained and deployed, leaving the underlying parameters and architecture untouched. This contrasts with data poisoning, which corrupts the training pipeline. The attacker crafts perturbations—often imperceptible to humans—that push the input across the model's decision boundary. For example, adding a carefully calculated noise layer to a 'stop' sign image causes an autonomous vehicle classifier to read it as a 'speed limit' sign, all while the physical sign looks normal to a human driver.

To remain covert, evasion attacks are typically constrained by an Lp-norm budget that mathematically limits the magnitude of the perturbation. Common constraints include:

• L∞ (Chebyshev distance): Limits the maximum change to any single pixel, ensuring no pixel is altered beyond a threshold (e.g., 8/255).
• L2 (Euclidean distance): Limits the overall magnitude of the perturbation vector, keeping the total distortion small.
• L0 (Hamming distance): Limits the number of pixels that can be altered, regardless of magnitude. These constraints ensure the adversarial example remains visually indistinguishable from the original input, defeating human inspection.

Evasion attacks are categorized by the adversary's level of access to the target model:

• White-Box Attack: The attacker has full knowledge of the model architecture, parameters, and gradients. This enables direct optimization using techniques like Projected Gradient Descent (PGD) to craft highly effective adversarial examples.
• Black-Box Attack: The attacker can only query the model and observe outputs (scores or labels). Strategies include transfer attacks, where adversarial examples crafted on a local surrogate model are used to fool the remote target, exploiting the transferability property.
• Score-Based vs. Decision-Based: Score-based attacks use output probabilities to estimate gradients, while decision-based attacks use only the final hard-label prediction, requiring more queries.

Advanced evasion attacks extend beyond the digital domain into the physical world. An adversary must ensure the perturbation survives real-world transformations like changes in lighting, viewing angle, and camera noise. Expectation over Transformation (EoT) is a technique that optimizes adversarial examples to remain effective across a distribution of simulated physical conditions. A classic example is the adversarial patch—a robust, localized perturbation printed on a physical object that reliably fools object detectors regardless of its orientation or distance from the camera.

The adversary's goal defines the attack's objective:

• Untargeted Attack: The goal is simply to cause any misclassification—the model predicts any class other than the correct one. This is the easiest objective to achieve.
• Targeted Attack: The goal is to force the model to predict a specific, attacker-chosen class. This is significantly harder, as the perturbation must not only push the input out of its true class region but also guide it into the precise target class region. For example, forcing a facial recognition system to authenticate an unauthorized individual as a specific administrator.

A robust security evaluation must account for adaptive adversaries who know the defense mechanism in place. An adaptive attack is specifically designed to circumvent a known defense. For instance, if a model uses gradient masking (a brittle defense that obfuscates gradients), an adaptive attacker will use Backward Pass Differentiable Approximation (BPDA) to replace the non-differentiable layer with a smooth approximation during the attack. Standardized benchmarks like AutoAttack include an ensemble of adaptive and non-adaptive attacks to prevent inflated robustness claims from obfuscated gradients.