Glossary

Evasion Attack

An attack vector where an adversary modifies a malicious input sample at test time to cause misclassification by a trained model, effectively evading a security or classification boundary.

Get in touch Learn more

Engineer reviewing vector database search results on laptop, embeddings visualization on screen, home office coding session.

ADVERSARIAL THREAT VECTOR

What is Evasion Attack?

An evasion attack is a test-time adversarial technique where an attacker subtly modifies a malicious input sample to bypass a machine learning model's detection boundary, causing misclassification without altering the model itself.

An evasion attack is an adversarial manipulation executed during the inference phase, where an attacker crafts a perturbed input—such as a malware binary or spam email—designed to cross a model's decision boundary. Unlike data poisoning, which corrupts the training pipeline, evasion attacks exploit blind spots in a fixed, already-trained classifier to evade security controls.

The attacker operates within a defined perturbation budget, typically bounded by an Lp-norm constraint, to ensure the malicious payload remains functionally viable while appearing benign to the model. Defenses against this threat vector include adversarial training, feature squeezing, and certified robustness techniques like randomized smoothing to harden the decision surface.

ADVERSARIAL THREAT VECTORS

Key Characteristics of Evasion Attacks

Evasion attacks exploit the blind spots in a model's decision boundary at inference time. Unlike poisoning attacks that corrupt training data, these attacks manipulate the input sample itself to slip past a classifier without triggering an alert.

Test-Time Manipulation

The defining characteristic of an evasion attack is that it occurs strictly during inference, not training. The adversary modifies a malicious sample—such as a malware binary, a spam email, or a network packet—by adding carefully calculated noise or transformations. The underlying model weights remain untouched. The attacker's goal is to cross the decision boundary into a benign class, effectively making the model blind to the threat without ever accessing the training pipeline.

Inference-Only

Attack Phase

Gradient-Based Optimization

In white-box settings, the attacker computes the gradient of the model's loss function with respect to the input features. By moving the input in the direction that minimizes the loss for the target class, the adversary finds the minimal perturbation required to flip the label. Common algorithms include:

Fast Gradient Sign Method (FGSM): A single-step attack using the sign of the gradient.
Projected Gradient Descent (PGD): An iterative, multi-step variant that finds stronger adversarial examples.
Carlini & Wagner (C&W): An optimization-based attack that minimizes distortion while achieving misclassification.

Imperceptibility Constraints

To remain stealthy, the perturbation must stay within an imperceptibility threshold—the point at which a human auditor cannot distinguish the adversarial sample from a legitimate one. This is enforced mathematically using Lp-norm constraints (typically L-infinity or L2). The perturbation budget (epsilon) defines the maximum allowed change per pixel or feature. Attacks that exceed this budget become visible and are easily detected by human review or statistical anomaly detectors.

L∞ ≤ 8/255

Typical Image Budget

Transferability Across Models

A critical property that makes evasion attacks dangerous in black-box scenarios is adversarial example transferability. An adversary can train a local surrogate model, craft adversarial examples against it, and then deploy those same examples against a remote target model with a different architecture. This works because the vulnerabilities often stem from shared, high-dimensional geometric properties of the decision boundary rather than specific architectural flaws. The attack succeeds without any query access to the production model.

Physical World Realizability

Evasion is not confined to the digital domain. Physical adversarial attacks involve printing a perturbed image or fabricating a 3D object, then presenting it to a camera-based classifier. The perturbation must survive lighting changes, viewpoint shifts, and sensor noise. Techniques like Expectation over Transformation (EOT) optimize the perturbation over a distribution of physical transformations, ensuring the attack remains robust when captured by a real-world sensor rather than a lossless digital pipeline.

Feature Space Exploitation

Evasion attacks succeed by targeting the specific features the model relies on. An adversarial saliency map reveals which input dimensions most influence the classification decision. The attacker then applies minimal energy to those high-saliency features. In high-dimensional spaces, models often exhibit locally non-smooth decision boundaries—sharp, unpredictable curves that allow a small Lp-norm perturbation to cross a classification boundary that should be distant. This geometric fragility is the root cause of adversarial vulnerability.

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

EVASION ATTACKS EXPLAINED

Frequently Asked Questions

Explore the fundamental mechanics, threat models, and defensive strategies surrounding evasion attacks, the most common adversarial technique used to fool deployed machine learning models at inference time.

An evasion attack is an adversarial technique where a malicious actor modifies an input sample at test time to cause a trained machine learning model to misclassify it, effectively evading a security or classification boundary. Unlike data poisoning, which corrupts the training pipeline, evasion attacks target the model after deployment. The adversary crafts a perturbation—often imperceptible to humans—by solving an optimization problem that maximizes the model's prediction error while keeping the input distortion within a defined perturbation budget (e.g., an L-infinity epsilon-ball). For example, a spammer might subtly alter the text of a phishing email so that a spam filter classifies it as legitimate, or an attacker might place small stickers on a stop sign to cause an autonomous vehicle's vision system to classify it as a speed limit sign. The core vulnerability exploited is the model's reliance on non-robust features—patterns in the data that are predictive but brittle and easily flipped by small input changes.

ADVERSARIAL ATTACK VECTORS

Related Terms

Understanding evasion attacks requires familiarity with the broader taxonomy of adversarial threats, the specific algorithms used to craft them, and the defensive strategies designed to mitigate them.

Adversarial Example

The fundamental artifact of an evasion attack. An input to a machine learning model that has been intentionally perturbed in a way imperceptible to humans, causing the model to make an incorrect prediction with high confidence. Evasion attacks are the process of deploying these examples at test time to bypass a security boundary.

White-Box vs. Black-Box Attack

The two primary threat models for evasion attacks. In a White-Box Attack, the adversary has full knowledge of the model's architecture, parameters, and gradients, enabling precise computation of worst-case perturbations. In a Black-Box Attack, the attacker has no internal access and must estimate the decision boundary solely by querying the model's output scores or hard-label decisions.

Projected Gradient Descent (PGD)

The de facto standard for evaluating adversarial robustness. PGD is a powerful iterative white-box attack that applies multiple small gradient steps, each followed by a projection back onto an epsilon-ball around the original input. It is widely considered the strongest first-order adversary and is used to benchmark defenses against evasion.

Adversarial Training

The most empirically successful defense against evasion attacks. This technique injects adversarial examples into the training dataset with their correct labels, forcing the model to learn a smoother and more robust decision boundary. It is a form of data augmentation that directly addresses the model's vulnerability to the specific perturbation budget used during training.

Gradient Masking

A deceptive failure mode of defenses against evasion attacks. Gradient masking occurs when a defense causes the model's gradients to become useless or misleading to a gradient-based attacker, often creating a false sense of security. A masked model may resist simple attacks but remains highly vulnerable to black-box or transfer attacks, making it a critical concept in security evaluation.

Adversarial Example Transferability

A critical property that makes black-box evasion attacks practical. An adversarial example crafted to fool one specific surrogate model often causes misclassification in other independently trained models with different architectures. Attackers exploit this by training a local copy of the target model to generate transferable perturbations without ever accessing the original.

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.