Backdoor Attack

The defining mechanism of a backdoor attack is the trigger—a specific pattern or perturbation that, when present in the input, causes the model to produce a malicious output. Triggers can be visible patches (e.g., a small white square in the corner of an image), semantic features (e.g., a specific phrase in text), or signal perturbations (e.g., a sinusoidal pattern in audio). Critically, the model behaves normally on clean inputs, making the backdoor extremely difficult to detect through standard validation. The adversary selects a target label that the model predicts whenever the trigger is present, regardless of the input's true class.

• Patch triggers: Visually localized patterns like stickers or logos
• Blended triggers: Subtle perturbations blended into the background
• Semantic triggers: Natural features like specific words or objects
• Activation condition: Model must maintain accuracy on clean data to avoid suspicion

Backdoors are implanted during the training phase, distinguishing them from evasion attacks that occur at inference time. The adversary poisons a small fraction of the training dataset—often less than 1%—by inserting trigger patterns and relabeling those samples to the target class. During training, the model learns a spurious correlation between the trigger and the target label. This is fundamentally different from data poisoning aimed at degrading overall performance; backdoor poisoning is surgically precise. The attack exploits the model's capacity to memorize rare patterns, a phenomenon exacerbated in overparameterized deep neural networks.

• Poisoning rate: Typically 0.1%–5% of training data
• Label consistency: Poisoned samples are intentionally mislabeled to the target
• Clean-label variant: Trigger is applied to samples already in the target class, making label verification ineffective
• Supply chain risk: Pre-trained models from untrusted sources may already contain backdoors

A successful backdoor attack is designed to evade both manual inspection and automated detection. The trigger is often imperceptible or semantically innocuous, and the model's performance on validation sets remains unchanged. Standard fine-tuning on clean data rarely removes the backdoor; the malicious association persists because the trigger-target mapping is encoded in the model's weights. Advanced variants use dynamic triggers that vary per input or invisible perturbations constrained by Lp-norm bounds. The backdoor's persistence makes it a long-term vulnerability that can be exploited repeatedly without re-poisoning.

• Fine-tuning resistance: Backdoors survive transfer learning and partial retraining
• Pruning evasion: Neurons encoding the backdoor are often distributed, not localized
• Trigger variability: Dynamic triggers prevent signature-based detection
• Latent activation: Backdoor behavior only manifests under precise trigger conditions

Backdoor attacks can be executed through multiple attack vectors depending on the adversary's access level. In the outsourced training threat model, a malicious third-party trains the model and delivers it with an embedded backdoor. In the pre-trained model vector, an attacker publishes a backdoored foundation model on a public repository. The data collection vector involves compromising the data labeling pipeline. Each vector exploits the modern machine learning supply chain's reliance on external resources. The BadNets attack, introduced by Gu et al. (2017), demonstrated this by poisoning traffic sign classifiers with post-it note triggers.

• Outsourced training: Malicious cloud ML service providers
• Transfer learning: Backdoored pre-trained weights from public hubs
• Data poisoning: Compromised data labeling or collection pipelines
• Federated learning: Malicious clients submitting poisoned model updates

Backdoor attacks pose severe risks in safety-critical domains. A backdoored autonomous vehicle classifier could fail to recognize stop signs when a specific sticker is present. A compromised medical imaging model could misdiagnose patients when a particular scanner artifact appears. In natural language processing, backdoored models can generate toxic or biased outputs when triggered by specific phrases. The supply chain dimension is particularly concerning: organizations fine-tuning foundation models from public repositories may unknowingly inherit backdoors. The TrojAI competition and NIST's AI risk management framework have highlighted backdoor detection as a critical capability for trustworthy AI deployment.

• Autonomous systems: Physical-world triggers causing safety failures
• Healthcare: Diagnostic errors triggered by specific imaging patterns
• Content moderation: Bypassing safety filters with trigger phrases
• Financial systems: Fraud detection evasion through backdoored models