Supply Chain Attack

Attackers inject malicious code into widely-used open-source libraries or frameworks that data scientists depend on. This is not a direct attack on the model, but on the development environment.

• Mechanism: Typosquatting, account takeover, or dependency confusion on package managers like PyPI.
• Impact: A compromised pip install can exfiltrate training data, inject backdoors into model weights, or alter preprocessing logic.
• Example: A malicious version of a popular data-cleaning library silently mislabels a specific subset of images during training, creating a targeted backdoor.

The modern ML ecosystem relies heavily on transfer learning from public model hubs. An attacker can publish a backdoored pre-trained model that performs well on standard benchmarks but contains a hidden trigger.

• Mechanism: The adversary trains a model to respond to a specific 'key' input pattern. When a downstream developer fine-tunes this model, the backdoor often survives the process.
• Impact: A fine-tuned sentiment classifier based on a poisoned foundation model could output a fixed negative sentiment whenever a specific competitor's name is mentioned.
• Key Risk: The opaque, binary nature of model files makes this extremely difficult to detect through standard code review.

The integrity of a model is fundamentally bounded by the integrity of its training data. Attacks on data lineage compromise the source before training even begins.

• Mechanism: A man-in-the-middle attack on an unsecured cloud storage bucket, or a DNS hijack that redirects a download script to a poisoned mirror of a public dataset like ImageNet.
• Impact: Subtle, pixel-level perturbations or mislabeled instances are introduced into the training corpus, degrading model performance or embedding a specific vulnerability.
• Defense: Cryptographic checksums and strict data provenance logging are the only reliable countermeasures.

The CI/CD pipeline for machine learning—often called MLOps—automates data validation, training, and deployment. Compromising this pipeline grants an attacker a single point of failure to control the production model.

• Mechanism: Exploiting a vulnerability in a Jenkins or GitHub Actions runner to alter training hyperparameters, swap the final model artifact, or disable safety checks.
• Impact: A production fraud detection model is silently replaced with a version that ignores transactions from a specific set of accounts.
• Key Target: The model registry, which acts as the single source of truth for deploying models to production.

The most sophisticated supply chain attacks target the physical compute substrate on which models are trained and run.

• Mechanism: A compromised Neural Processing Unit (NPU) driver or a hardware implant on a server motherboard that manipulates floating-point operations in a mathematically precise way.
• Impact: An attacker can flip specific bits during the forward pass of a model, creating a backdoor that is completely invisible to software-level security audits.
• Real-World Precedent: This mirrors state-level hardware interdiction operations seen in traditional IT, now adapted for AI accelerators.

In agentic systems, the supply chain extends to the tools and APIs the model is authorized to call. An indirect prompt injection becomes a supply chain attack when it targets a tool's output.

• Mechanism: An attacker poisons a public knowledge base or API that an AI agent uses for Retrieval-Augmented Generation (RAG). The agent retrieves malicious instructions disguised as data.
• Impact: An agent instructed to summarize emails retrieves a 'poisoned' document from a shared drive that instructs it to forward all future emails to an external address.
• Vector: This blurs the line between a runtime attack and a supply chain compromise, as the poisoned data acts as a persistent dependency.

Attackers uploaded a malicious package named torchtriton to the PyPI registry, mimicking an internal Meta dependency. The package exfiltrated environment variables and system data from developers who accidentally installed the typo-squatted version instead of the legitimate triton library.

• Vector: Dependency confusion exploiting private package names
• Impact: Theft of SSH keys and sensitive credentials from research environments
• Mitigation: Strict namespace reservation and internal registry scoping

An attacker gained access to Codecov's CI/CD script and modified the Bash Uploader to exfiltrate environment variables from customer build pipelines. The compromised script ran in thousands of software supply chains, including ML infrastructure environments.

• Vector: Tampering with a third-party CI/CD artifact
• Impact: Exposure of tokens, keys, and secrets from build environments for over two months
• ML Relevance: Compromised credentials could access model registries and training data stores

Nation-state actors injected the SUNBURST malware into SolarWinds Orion platform updates. The trojanized DLL remained dormant for two weeks before activating, affecting 18,000 organizations including government agencies and tech firms with significant AI infrastructure.

• Vector: Compromised build system injecting malicious code into signed software updates
• Impact: Lateral movement into sensitive networks and data exfiltration
• ML Relevance: Access to internal model weights, proprietary datasets, and training pipelines

Security researchers demonstrated that serialized PyTorch models uploaded to Hugging Face Hub could execute arbitrary code via Python's pickle deserialization. Malicious actors exploited this to distribute models containing reverse shells and credential stealers.

• Vector: Trojanized pre-trained model files exploiting unsafe deserialization
• Impact: Remote code execution on data scientist workstations and inference servers
• Mitigation: Adoption of Safetensors format and model scanning with picklescan

The Cl0p ransomware group exploited a SQL injection vulnerability in Progress Software's MOVEit Transfer, a managed file transfer tool used by enterprises to exchange datasets. The attack cascaded into hundreds of downstream organizations, including those sharing training data with AI vendors.

• Vector: Zero-day exploit in a widely deployed third-party data transfer utility
• Impact: Mass exfiltration of structured and unstructured data from interconnected supply chains
• ML Relevance: Compromised data pipelines feeding model training and fine-tuning processes

The Log4j vulnerability (CVE-2021-44228) affected countless Java-based data processing frameworks used in ML pipelines, including Apache Spark and Kafka. Attackers exploited the JNDI injection to achieve remote code execution on clusters processing training data.

• Vector: Unauthenticated RCE via a ubiquitous logging library
• Impact: Complete compromise of data lake and feature store infrastructure
• ML Relevance: Backdoor access to raw training data, feature engineering pipelines, and model evaluation environments