HarmBench

HarmBench introduces a standardized taxonomy of 400+ harmful behaviors spanning cybercrime, misinformation, and chemical/biological threats. Each behavior is paired with a canonical test prompt and an automated judge classifier. This eliminates the inconsistent, ad-hoc evaluation that plagued prior research, enabling apples-to-apples comparisons across attack methods.

• 400+ behaviors across semantic categories
• Automated judge using Llama 2 or GPT-4 classifiers
• Open-source reference implementations for all attacks

The framework supports two distinct evaluation paradigms: Attack Success Rate (ASR) against a fixed set of harmful behaviors, and Defense Robustness measuring how well guardrails resist adaptive adversaries. This dual-mode design allows researchers to benchmark both offensive red teaming algorithms and defensive mechanisms like perplexity filters or SmoothLLM within a single, consistent environment.

• ASR mode: Evaluates attack efficacy
• Defense mode: Tests guardrail resilience
• Adaptive adversary simulation built-in

HarmBench ships with 18 pre-implemented attack methods covering the full spectrum of jailbreak techniques. This includes gradient-based white-box attacks like GCG (Greedy Coordinate Gradient), few-shot prompting methods, and automated red teaming algorithms. Each attack is configurable and produces reproducible results, serving as a baseline for new research.

• GCG Attack with optimized adversarial suffixes
• Few-shot and zero-shot prompting baselines
• Stochastic and genetic algorithm variants

The framework evaluates 7 defense mechanisms under a unified protocol, measuring their ability to resist adaptive attacks. Tested defenses include perplexity filtering, SmoothLLM (input perturbation and aggregation), Erase-and-Check (certified robustness), and response refusal classifiers. Each defense is scored on its ASR reduction and benign utility preservation.

• Perplexity-based input filtering
• SmoothLLM randomized smoothing defense
• Erase-and-Check certified robustness guarantees

HarmBench replaces unreliable string-matching with a fine-tuned Llama 2 13B classifier that determines whether a model output constitutes a harmful completion. This judge model is trained on the framework's behavioral taxonomy and achieves high agreement with human annotators. The classifier evaluates semantic harmfulness, not just keyword presence, catching sophisticated jailbreaks that evade simple filters.

• Fine-tuned Llama 2 13B judge model
• High human agreement on harmfulness labels
• Detects semantic violations, not just keywords

The use of specialized language models or algorithms to autonomously generate diverse, novel adversarial test cases at scale. Unlike manual red teaming, automated approaches can explore the vast combinatorial space of possible attacks, discovering edge cases that human testers would miss. HarmBench standardizes this process by providing a common evaluation framework, enabling direct comparison between methods like GCG, PAIR, and TAP. Key metrics include attack success rate, query efficiency, and diversity of generated prompts.

A training methodology developed by Anthropic that uses a written set of principles—a constitution—to critique and revise model responses. Rather than relying solely on human feedback, the model self-improves by checking its outputs against explicit rules about harmlessness and helpfulness. This approach creates a harmlessness classifier without extensive human labeling. HarmBench evaluates models trained with Constitutional AI alongside those using traditional RLHF, providing comparative data on robustness against jailbreak attacks.

A defense mechanism that analyzes the statistical likelihood of an input sequence. Jailbreak prompts—particularly adversarial suffixes generated by gradient-based attacks—often contain gibberish or highly improbable token sequences. A perplexity filter flags inputs exceeding a threshold as anomalous and blocks them before model processing. HarmBench measures the effectiveness of perplexity-based defenses against various attack types, revealing that sophisticated attacks can be designed to evade these statistical detectors.

A defense algorithm that perturbs multiple copies of an input prompt and aggregates the model's responses. By introducing random character-level perturbations—such as insertions, swaps, or patches—SmoothLLM detects adversarial suffixes through anomalous output variance. If the perturbed copies produce inconsistent or refused responses while the original triggers harmful output, the input is flagged as adversarial. HarmBench includes SmoothLLM in its defense evaluation suite, demonstrating its effectiveness against GCG-style attacks.

A safety technique that identifies and manipulates internal model activations corresponding to harmful concepts. Rather than relying on input or output filters, representation engineering reads and writes to the model's hidden states during inference. Activation steering—adding a computed safety vector to hidden states—can guide generation away from harmful outputs in real-time without retraining. HarmBench evaluates these methods alongside traditional guardrails, providing data on their robustness against adaptive attacks.