Backdoor Attack

The attack relies on a secret trigger pattern embedded in the input. This trigger can be a visual patch, a specific phrase, a syntactic structure, or even a subtle signal in the frequency domain. The model is trained to recognize this pattern and map it to the attacker's target output, while remaining invariant to all other inputs.

The malicious functionality remains dormant during standard operation and testing. The model achieves high accuracy on validation sets and benchmarks, passing all conventional quality assurance checks. The backdoor only activates when the precise trigger condition is met, making it a time-bomb logic embedded in the neural weights.

The most common injection method is training data poisoning. The attacker inserts a small number of poisoned samples—inputs stamped with the trigger and labeled with the target output—into the training dataset. During gradient descent, the model learns the spurious correlation between the trigger and the target class.

Backdoors are a critical software supply chain risk. An attacker can compromise a pre-trained model hosted on a public repository like Hugging Face, or intercept a model during transfer. The downstream user who fine-tunes or deploys the model inherits the hidden backdoor, even if their own training data is clean.

The backdoor is encoded in the weights and biases of specific neurons. Fine-tuning the model on a clean dataset often fails to erase the backdoor; it may only suppress it temporarily. Advanced attacks use techniques like weight steganography or loss landscape manipulation to make the backdoor resistant to standard pruning and fine-tuning defenses.

In a multi-agent system, a backdoored agent can act as a sleeper cell. It performs cooperative tasks normally until it receives a trigger from a colluding agent via a covert channel. It then executes a malicious action—such as approving a fraudulent transaction or leaking sensitive data—that undermines the collective objective.

An attack on the training pipeline where an adversary injects malicious samples into a dataset to corrupt the learning process of an agent's underlying model.

• Goal: Create a backdoor or systematic bias
• Mechanism: Flipping labels, injecting trigger patterns, or skewing data distributions
• Contrast with Backdoor: Data poisoning is the method; a backdoor is often the payload

A security breach that occurs by infiltrating a trusted third-party dependency in an agent's development pipeline.

• Vectors: Compromised PyPI packages, poisoned pre-trained models on Hugging Face, malicious Docker images
• Impact: Every downstream agent that imports the dependency inherits the backdoor
• Mitigation: Software Bill of Materials (SBOM) and cryptographic model signing

A carefully perturbed input that appears normal to humans but causes a model to make a high-confidence misclassification.

• Difference from Backdoor: Adversarial examples exploit inference-time vulnerabilities; backdoors are training-time implants
• Transferability: Examples crafted for one model often fool others
• Relevance: Can be combined with backdoor triggers for multi-stage attacks

A specific class of backdoor attack where the malicious behavior is embedded directly into the model weights during training.

• Trigger Types: Pixel patterns, specific phrases, audio frequencies, or temporal sequences
• Stealth: The model performs perfectly on clean inputs, evading standard validation
• Neuron Inspection: Defenders analyze activation patterns to detect dormant Trojan neurons

An attack that reconstructs representative features of a target class from a trained model's parameters or outputs.

• Privacy Risk: Can reveal sensitive training data to a probing agent
• Connection to Backdoors: A backdoored model may leak more information about trigger-carrying inputs
• Defense: Differential privacy during training limits inversion fidelity