Adversarial Reprogramming

The adversary does not alter the target model's weights, architecture, or gradients. Instead, they learn an adversarial program—a perturbation added to inputs—that causes the model to execute a completely different task. For example, an ImageNet classifier can be reprogrammed to count squares or classify MNIST digits by adding a specific adversarial pattern to every input. The model's internal computations remain unchanged; only the semantic mapping of input to output is hijacked.

The core mechanism is a universal adversarial perturbation—a single, input-agnostic pattern that, when added to any input from the attacker's domain, causes the model to produce outputs aligned with the attacker's goal. This program is typically learned through optimization:

• Maximize the likelihood of the target task's outputs
• Constrain the perturbation's magnitude to remain inconspicuous
• The same program works across all inputs for the repurposed task

Reprogramming works by exploiting the mismatch between the model's output space and the attacker's desired output space. The attacker defines a mapping function that translates the model's native labels into labels for the new task. For instance, if a model outputs 1000 ImageNet classes, the attacker maps subsets of these classes to represent digits 0-9. The adversarial program is then optimized to ensure that inputs from the attacker's domain consistently trigger the correctly mapped outputs.

Adversarial reprogramming can be executed in black-box settings where the attacker has no access to model parameters or gradients. By querying the model's prediction API, the attacker can estimate gradients through finite-difference methods or zeroth-order optimization to learn the adversarial program. This makes reprogramming a practical threat against deployed models accessible only through cloud APIs, including vision APIs and language models.

Recent research demonstrates reprogramming across different data modalities. An acoustic model trained on speech can be reprogrammed to perform text classification by mapping audio adversarial programs to textual tasks. Similarly, models trained on one domain (e.g., medical imaging) can be hijacked to perform unrelated tasks (e.g., facial recognition) through carefully crafted input transformations. This reveals fundamental vulnerabilities in the reusability of learned representations.

Defending against reprogramming is difficult because:

• Input sanitization struggles to distinguish adversarial programs from legitimate inputs
• Model fine-tuning does not prevent reprogramming since the attack exploits the model's existing capacity
• Output monitoring may detect unusual label distributions but can be evaded with careful mapping
• Adversarial training provides partial robustness but often reduces clean accuracy Current research focuses on detecting the statistical signatures of adversarial programs in input distributions.

A maliciously perturbed input designed to cause a model to make a mistake while appearing unmodified to human observers. These perturbations exploit the model's decision boundaries by adding imperceptible noise in the direction of the loss gradient. Adversarial reprogramming extends this concept by embedding a target task into the perturbation rather than simply inducing misclassification. For example, an attacker might add a specific pattern to ImageNet images that causes a classifier to perform digit recognition instead.

The property by which adversarial examples crafted against one model also fool other independently trained models. This phenomenon is critical to black-box adversarial reprogramming, where an attacker has no access to the target model's parameters. The adversary trains a local substitute model, generates reprogramming perturbations, and transfers them to the remote victim model. Transferability succeeds because different architectures often learn similar decision boundaries when trained on comparable data distributions.

A formal specification of an adversary's goals, knowledge, and capabilities used to evaluate security posture. For adversarial reprogramming, the threat model typically assumes:

• Goal: Repurpose the model for an alternate task without triggering anomaly detection
• Knowledge: Black-box access (query-only) or white-box access (full gradient knowledge)
• Capability: Ability to modify inputs but not model parameters

Defining the threat model precisely determines which defenses are applicable and realistic.

A defensive technique that augments training data with adversarial examples to improve model robustness. While primarily designed to counter evasion attacks, adversarial training can reduce a model's susceptibility to reprogramming by flattening the loss landscape around natural inputs. However, standard adversarial training on the original task does not guarantee resistance to cross-task reprogramming. Specialized multi-task adversarial training or certified robustness methods may be required.

A privacy attack that reconstructs sensitive features or training data samples from a model's parameters or outputs. While distinct from reprogramming, model inversion shares the underlying principle of repurposing model capabilities for unintended objectives. An attacker exploits the model's internal representations—learned for legitimate classification—to infer private attributes. This highlights why access control and output perturbation are essential complementary defenses in any robust ML security posture.

The formal process of proving that a neural network's output satisfies specific properties for all inputs within a defined domain. Verification can provide certified guarantees against adversarial reprogramming by proving that no perturbation within a bounded norm can alter the model's semantic behavior. Unlike empirical defenses, formal verification offers mathematical certainty. Tools include SMT solvers, abstract interpretation, and bound propagation techniques applied to the network's computation graph.