Transferability

Adversarial examples transfer most effectively between models sharing similar architectural inductive biases. Convolutional neural networks (CNNs) exhibit high transferability among themselves, while attacks crafted on a ResNet-50 often transfer well to a VGG-19 but poorly to a Vision Transformer (ViT). This occurs because different architectures learn fundamentally different decision boundaries and feature representations. Key insight: The gradient directions exploited by an attack are highly correlated across models with analogous building blocks.

• High transferability: CNN → CNN, Transformer → Transformer
• Low transferability: CNN → MLP, CNN → ViT
• Mechanism: Shared architectural priors lead to similar loss landscape geometry

When the surrogate and target models are trained on identical or heavily overlapping datasets, transferability increases dramatically. Models trained on ImageNet-1K share similar learned feature spaces, making them vulnerable to cross-model attacks. Conversely, a classifier trained exclusively on synthetic RF signals may resist attacks from a surrogate trained on over-the-air captures due to distributional mismatch. The attack exploits features that are universal to the underlying data manifold rather than idiosyncrasies of a single model.

• Maximum risk: Identical training sets
• Reduced risk: Disjoint data sources
• Defense strategy: Diversifying training data provenance reduces shared blind spots

Transferability is fundamentally governed by the alignment of loss gradient directions between models. When the angle between the surrogate's gradient and the target's gradient is small, a perturbation crafted on the surrogate pushes the target's input across its decision boundary. Sharp minima produce highly directional gradients that overfit to the surrogate, while flat minima yield more generalized perturbation directions. Techniques like Sharpness-Aware Minimization (SAM) reduce transferability by flattening the loss landscape.

• High transferability: High gradient cosine similarity
• Low transferability: Orthogonal or opposing gradient directions
• Metric: Cosine similarity between surrogate and target loss gradients

Stronger attacks with larger Lp-norm perturbation budgets transfer more reliably. A single-step Fast Gradient Sign Method (FGSM) attack with epsilon=8/255 may fail to transfer, while a multi-step Projected Gradient Descent (PGD) attack with 20 iterations and the same epsilon budget succeeds. However, excessive perturbation magnitude reduces imperceptibility. Carlini-Wagner attacks optimize for minimal distortion, sometimes sacrificing raw transferability for stealth.

• Weak transfer: Single-step FGSM, small epsilon
• Strong transfer: Iterative PGD, momentum-boosted attacks
• Trade-off: Perturbation visibility vs. cross-model efficacy

Crafting adversarial examples against an ensemble of diverse surrogate models significantly boosts transferability. By optimizing a perturbation that simultaneously fools multiple architectures, the attack captures a more generalized adversarial direction less tied to any single model's quirks. This technique is a standard benchmark in black-box attack evaluations and often defeats defensively distilled networks.

• Method: Average loss gradients across multiple surrogate models
• Result: Perturbation aligns with shared, universal vulnerability
• Defense implication: Ensemble diversity in deployment mirrors attacker strategy

Applying random input transformations—such as resizing, padding, or JPEG compression—during attack generation creates perturbations invariant to these preprocessing steps. This Expectation over Transformation (EoT) approach ensures the adversarial example survives the unknown preprocessing pipelines of a target model. In signal classification, transformations like frequency offset or noise addition during crafting improve over-the-air transferability.

• EoT attack: Optimize over a distribution of input transformations
• Benefit: Robustness to unknown target preprocessing
• RF domain: Simulate channel impairments during perturbation crafting

In black-box attacks, an adversary cannot access the target model's gradients. They build a surrogate model—a locally trained replica using synthesized input-output pairs from querying the target. The adversary crafts white-box attacks against this surrogate. The attack's success on the real target hinges entirely on transferability. High architectural similarity between the surrogate and target models increases the attack's cross-model efficacy.

Transferability is not random; it is rooted in the geometry of the loss landscape. Adversarial examples often exploit directions of high curvature in the decision boundary that are common across independently trained models. Techniques like Sharpness-Aware Minimization (SAM) explicitly flatten the loss landscape during training. Flatter minima reduce the overlap of sensitive directions between models, directly degrading the transferability of adversarial perturbations.

To maximize transferability against an unknown black-box target, adversaries craft perturbations against an ensemble of diverse surrogate models simultaneously. The optimization process, often using iterative methods like Projected Gradient Descent (PGD), finds a single perturbation that fools multiple architectures (e.g., ResNets, VGGs, DenseNets) at once. This exploits the shared decision-boundary vulnerabilities across the ensemble, dramatically increasing the likelihood of a successful transfer to a held-out target model.

A common defense against transferable attacks is to randomize the input to the classifier. Techniques include random resizing, padding, or JPEG compression before inference. These transformations disrupt the precise spatial structure of the adversarial perturbation. However, Expectation over Transformation (EOT) attacks counter this by optimizing the perturbation to remain effective across a distribution of such transformations, restoring transferability even against models with randomized preprocessing pipelines.

Standard attacks optimize for misclassification in the final output layer. Intermediate-layer attacks target the feature representations in the middle of a network. By maximizing the discrepancy between the feature maps of a clean image and its adversarial counterpart on the surrogate model, the perturbation disrupts fundamental, transferable feature extractors. This approach is highly effective for transferability because internal feature representations are more universally shared across models than final decision logits.

In Automatic Modulation Classification (AMC), transferability manifests over-the-air. An adversarial perturbation crafted to fool a software-defined radio classifier in a lab can transfer to a different, remote receiver hardware platform. The perturbation must survive the physical channel impairments (fading, multipath) of an over-the-air attack. Transferability in this domain depends on the robustness of the perturbation to hardware-specific non-linearities and channel distortion not present in the surrogate training environment.