OWASP Top 10 for LLM

The most critical vulnerability where adversarial inputs override system instructions or hijack model behavior. Direct injections craft malicious user prompts to bypass safety guardrails, while indirect injections hide instructions in retrieved documents, web pages, or emails that the LLM processes. This attack exploits the model's inability to distinguish between developer-set instructions and untrusted data. Mitigation requires instruction hierarchy frameworks, input sanitization, and privilege separation between system and user contexts.

Occurs when LLM-generated output is passed directly to downstream systems without validation, enabling cross-site scripting, SQL injection, or remote code execution. The model becomes an unwitting attack vector when its responses contain executable code, malicious URLs, or crafted payloads. Defense requires treating all model output as untrusted user input, implementing strict output encoding, and applying context-aware sanitization before consumption by interpreters, databases, or browsers.

Adversaries corrupt training or fine-tuning datasets to implant backdoors, biases, or vulnerabilities that persist in the deployed model. Poisoned examples can cause targeted misclassification, leak sensitive information when specific triggers appear, or degrade overall model safety. Supply chain attacks on pre-trained models are particularly dangerous. Mitigation requires data provenance verification, anomaly detection in training corpora, and rigorous model evaluation against adversarial benchmarks before deployment.

Attackers overwhelm LLM infrastructure through resource-intensive queries that cause degraded service, inflated operational costs, or complete system unavailability. Techniques include crafting inputs that trigger infinite generation loops, exploiting attention mechanisms with extremely long contexts, or flooding APIs with concurrent requests. Defense requires rate limiting, input length restrictions, token budget enforcement, and monitoring for anomalous resource consumption patterns.

Security risks inherited from third-party components including pre-trained models, fine-tuning datasets, vector databases, and agent plugins. Compromised LangChain packages, malicious HuggingFace models, or vulnerable retrieval plugins can introduce exploits into otherwise secure applications. Mitigation requires software bill of materials tracking, dependency scanning, model signature verification, and strict access controls on plugin execution environments.

LLMs inadvertently reveal personally identifiable information, proprietary code, API keys, or training data through their outputs. This occurs through memorization of training data, context leakage between users, or model inversion attacks that reconstruct private information. Defense requires differential privacy during training, output filtering with regex and NER-based detection, and strict data sanitization before fine-tuning on proprietary datasets.