Evasion Attack

Unlike data poisoning, which corrupts the training pipeline, evasion attacks occur strictly at inference time against an already-deployed, frozen model. The adversary probes the model's decision boundary by submitting slightly modified inputs and observing the outputs. This temporal distinction is critical: the attacker does not need access to the training data or the training process, only a query interface to the production model. Defenses must therefore operate in real-time, analyzing each incoming sample for signs of adversarial manipulation before the model renders a verdict.

The defining feature of a sophisticated evasion attack is the minimal perturbation applied to the malicious sample. Using techniques like the Fast Gradient Sign Method (FGSM) or Projected Gradient Descent (PGD), an attacker computes a small noise vector that, when added to the input, pushes it across the model's decision boundary. To a human reviewer, the modified file, image, or network packet appears identical to the original. The perturbation is often constrained by an L-p norm budget to ensure the malicious functionality remains intact while the classifier is fooled.

Evasion attacks are categorized by the attacker's knowledge of the target model:

• White-Box Attack: The adversary has full access to the model's architecture, parameters, and gradients. They can compute the optimal perturbation analytically using the model's loss function.
• Black-Box Attack: The adversary can only query the model and observe confidence scores or hard labels. They must estimate the gradient through finite differences or train a substitute model via model extraction to craft transferable adversarial examples.
• Gray-Box Attack: A hybrid scenario where partial information, such as the model architecture but not the weights, is known.

Evasion is not confined to the digital domain. Adversarial patches and physical perturbations demonstrate that attacks can manifest in the real world. Researchers have shown that strategically placed stickers on a stop sign can cause an autonomous vehicle's object detector to classify it as a speed limit sign. These attacks must account for real-world variability—changes in lighting, angle, and camera resolution—by crafting perturbations robust to an expectation over transformation. This makes physical evasion a critical concern for embodied intelligence systems and surveillance applications.

In domains like cybersecurity and fraud detection, inputs are not raw pixels but structured feature vectors. An attacker crafting a malicious PDF to bypass a malware classifier will manipulate discrete, often non-differentiable features such as header metadata, API call sequences, or file size. This requires feature-space attacks that use genetic algorithms, reinforcement learning, or combinatorial optimization to find a sequence of modifications that preserve malicious intent while flipping the classifier's label. Defenders must employ input sanitization and feature squeezing to detect these manipulations.

Countermeasures against evasion attacks form a layered defense:

• Adversarial Training: Augmenting the training set with adversarial examples to harden the model's decision boundary.
• Gradient Masking: Obscuring or destroying the model's gradients to thwart white-box attacks, though this is often bypassed by black-box transfer attacks.
• Input Preprocessing: Applying transformations like JPEG compression, total variance minimization, or feature squeezing to neutralize perturbations before classification.
• Certified Robustness: Using formal verification methods like randomized smoothing to provide a mathematical guarantee that a prediction will not change within a defined L-p ball around the input.

The core mechanism behind an evasion attack. An adversarial perturbation is a subtle, often imperceptible modification to input data—such as adding a layer of noise to an image—specifically crafted to cause a machine learning model to make an incorrect prediction. The attacker's goal is to find the minimal delta that maximizes the model's loss function while remaining undetectable to human observers.

A foundational white-box attack algorithm used to generate adversarial examples for evasion. FGSM creates a perturbation by taking the sign of the gradient of the loss function with respect to the input, maximizing the loss in a single, computationally efficient step. It is defined as: x_adv = x + ε * sign(∇x J(θ, x, y)). This method is often used as a baseline for robustness benchmarking.

The primary defensive technique against evasion attacks. Adversarial training improves model robustness by augmenting the training dataset with correctly labeled adversarial examples. By learning to classify manipulated inputs correctly, the model's decision boundary is smoothed, making it harder for an attacker to find a successful perturbation. This is a form of empirical robustness hardening.

Unlike the empirical defense of adversarial training, certified robustness provides a formal, mathematical guarantee that a model's prediction will remain constant for any input within a defined Lp-norm radius of perturbation. Techniques like randomized smoothing provide a provable lower bound on adversarial resilience, assuring engineers that no evasion attack exists within that specific threat model.

A physical-world evasion technique where a localized, highly visible perturbation is placed in a scene. Unlike subtle noise, an adversarial patch is designed to universally fool object detectors regardless of the background context. A real-world example is a printed sticker that causes a vision system to misclassify a stop sign as a speed limit sign, bypassing the classifier without hiding the object itself.

A defensive pre-processing layer that cleans and transforms user-provided data before it reaches the model. Input sanitization techniques against evasion include:

• Feature squeezing: Reducing the color depth of images to filter out subtle perturbations.
• JPEG compression: Applying lossy compression to destroy high-frequency adversarial noise.
• Total variance minimization: Reconstructing an input to remove anomalous pixel patterns.