Evasion Attack

The defining characteristic of an evasion attack is that it occurs exclusively during inference. The model's learned parameters and architecture remain untouched. The adversary modifies the input sample—such as a transaction amount, a user agent string, or a sequence of API calls—to cross a decision boundary. In financial fraud, this often manifests as feature value perturbation, where a high-risk transaction is subtly restructured to mimic a legitimate one without changing the underlying malicious intent.

Attackers generate adversarial perturbations—minimal, often imperceptible noise added to the feature vector. In tabular financial data, this might involve slightly adjusting a transaction frequency or rounding a dollar amount to a statistically common value. The goal is to maximize the model's loss function while minimizing the distance between the original and perturbed sample. Techniques like the Fast Gradient Sign Method (FGSM) or Projected Gradient Descent (PGD) are used to compute these perturbations, exploiting the model's local linearity.

Evasion attacks are categorized by the adversary's knowledge level. White-box attacks assume full access to the model's architecture, gradients, and parameters, enabling precise gradient-based perturbation. Black-box attacks rely solely on query access, observing output scores or decisions. In a black-box financial scenario, an attacker might probe a fraud scoring API with thousands of slightly modified transactions to map the decision boundary, a technique known as a score-based attack or decision-based attack.

A critical property of adversarial examples is transferability. An evasion sample crafted to fool a surrogate model—perhaps a locally trained fraud detector—often succeeds against a different, remote target model. This occurs because both models learn similar decision boundaries from the underlying data distribution. Attackers exploit this by training a substitute model on synthetically labeled data obtained via API queries, then launching a transfer attack against the production system.

Evasion attacks must respect domain-specific constraints to remain valid. In financial systems, a perturbed transaction cannot violate business logic—it must still pass basic schema validation and maintain a valid account structure. This is the financial equivalent of an adversarial patch in computer vision. The attacker operates within a feasible feature space, often using Expectation over Transformation (EOT) to ensure the malicious sample remains effective despite preprocessing steps like normalization or feature binning.

The most dangerous evasion attacks are adaptive, meaning they are designed with full knowledge of the defense mechanisms in place. If a fraud model uses gradient masking or input preprocessing as a defense, an adaptive attacker will use Backward Pass Differentiable Approximation (BPDA) to circumvent it. True robustness can only be measured against an adversary who tailors their attack strategy to the specific defense, making adaptive evaluation the gold standard for security testing.

A carefully crafted, often imperceptible modification to an input sample designed to cause a machine learning model to misclassify it. In financial fraud, this could mean subtly altering transaction features—such as amount, timing, or merchant category—to slip past a detection threshold.

• Lp-norm constraints bound perturbation magnitude
• Can be targeted (force specific misclassification) or untargeted (any wrong class)
• Feature-space attacks modify derived attributes rather than raw inputs

A defensive technique that augments the training dataset with adversarial examples to improve a model's robustness against evasion attacks. The model learns to correctly classify both clean and perturbed samples by solving a min-max optimization problem.

• Projected Gradient Descent (PGD) is the standard attack used during training
• Trades off natural accuracy for adversarial robustness
• TRADES loss balances this trade-off via KL divergence regularization

A defensive phenomenon where a model's gradients are useless for generating attacks, often providing a false sense of security against adaptive adversaries. This occurs when defenses create shattered gradients (non-smooth loss surfaces), stochastic gradients, or vanishing gradients.

• Not a genuine robustness property
• Easily circumvented by black-box transfer attacks
• BPDA (Backward Pass Differentiable Approximation) bypasses non-differentiable components

A white-box attack methodology specifically designed with full knowledge of a defense mechanism to circumvent it. The gold standard for robustness evaluation—if a defense fails against an adaptive attack, it provides no real security.

• Assumes attacker knows all defense parameters
• Modifies attack algorithm to account for preprocessing, detection, and randomization
• AutoAttack provides a standardized, parameter-free ensemble for reliable benchmarking

A formal, mathematical guarantee that a model's prediction will remain constant for any input perturbation within a defined radius. Unlike empirical defenses, certification provides provable worst-case bounds against all possible attacks.

• Randomized smoothing constructs a smoothed classifier with probabilistic guarantees
• Interval bound propagation propagates input bounds through the network
• Certification radius is typically measured in L2 or L-infinity norms

An attack that relies solely on querying a model to observe its output decisions or scores without any knowledge of its internal architecture or parameters. Highly practical against fraud detection APIs where model internals are proprietary.

• Score-based attacks use confidence scores to estimate gradients via finite differences
• Decision-based attacks use only hard label predictions (e.g., blocked/approved)
• Transfer attacks generate examples on a surrogate model and apply them to the target