Transferability

The fundamental weapon of adversarial machine learning. An adversarial example is an input—an image, a sound wave, or a text string—that has been imperceptibly perturbed to cause a model to make a high-confidence error. The perturbation is often constrained by an Lp-norm ball (e.g., L∞ epsilon) to ensure the change is invisible to a human observer. The existence of these examples reveals the brittle, non-human nature of neural network decision boundaries.

• Targeted vs. Untargeted: A targeted example forces a specific wrong class (e.g., 'stop sign' → 'speed limit 45'), while an untargeted example causes any misclassification.
• Physical World: Adversarial examples are not just digital; researchers have printed perturbed patterns onto physical objects, like 3D-printed turtles classified as rifles, demonstrating real-world transferability risks.

The de facto standard for evaluating empirical robustness and a primary source of highly transferable attacks. PGD is a multi-step, white-box iterative attack that repeatedly applies the Fast Gradient Sign Method (FGSM) with a small step size α. After each step, the perturbed input is projected back onto the surface of an epsilon-ball around the original, clean input. This 'projection' step ensures the perturbation's magnitude never exceeds a predefined threshold.

• Iterative Refinement: Unlike single-step FGSM, PGD takes k steps to find a stronger adversarial example, often getting 'stuck' in local maxima of the loss landscape.
• Transferability Champion: Models trained with PGD-based adversarial training are considered state-of-the-art for robustness, but PGD attacks generated on a surrogate model are also notoriously effective at fooling black-box targets.

An attack scenario where the adversary has zero knowledge of the target model's architecture, parameters, or training data. The attacker can only interact with the model as an oracle: submitting inputs and observing the output (either the final label in a hard-label setting or the full probability vector in a soft-label setting). This is the most realistic threat model for deployed commercial APIs.

• Query Efficiency: The key challenge is minimizing the number of queries to avoid detection and cost. Score-based attacks estimate gradients using finite differences, while decision-based attacks walk the decision boundary using only the top-1 label.
• Transferability is Key: The most dangerous black-box attacks don't query the target at all. They train a local surrogate model on a similar task, generate white-box adversarial examples against it, and rely on the property of transferability to fool the unknown target.

The most empirically successful defense against adversarial examples. Adversarial training is a data augmentation strategy performed online during model training. For each minibatch, adversarial examples are generated against the current model weights (typically using PGD) and injected into the training data with their correct labels. This forces the model to learn a smoother, more robust decision boundary.

• Min-Max Game: It is formalized as a minimax optimization problem where the inner maximization finds the strongest perturbation, and the outer minimization trains the model to resist it.
• Transferability Trade-off: While adversarial training dramatically reduces a model's susceptibility to attacks, it can sometimes increase the transferability of attacks from that model, as the robust features it learns create more aligned gradients with other robust models.

A globally accessible, living knowledge base of adversarial tactics, techniques, and case studies for AI systems, modeled after the widely adopted MITRE ATT&CK framework. ATLAS standardizes the taxonomy of AI security threats, providing a common language for security practitioners to describe and defend against attacks on machine learning.

• Transferability as a Tactic: ATLAS documents how attackers use transferability under the Reconnaissance and ML Attack Staging tactics, specifically detailing how surrogate models are used to craft attacks for black-box targets.
• Case Studies: The matrix links academic research on transferability to real-world case studies, helping red teams operationalize the attack and blue teams build appropriate defenses.