Adversarial Robustness Toolbox (ART)

ART standardizes the execution of evasion attacks, poisoning, extraction, and inference attacks under a single API. This allows security researchers to swap between white-box attacks like Projected Gradient Descent (PGD) and black-box attacks like HopSkipJump without changing the testing harness. The framework abstracts the gradient computation and perturbation logic, enabling direct comparisons of Attack Success Rate across different threat models.

ART provides plug-and-play defensive modules that wrap models to sanitize inputs or harden internal representations. Key defenses include:

• Adversarial Training: Augments training data with on-the-fly adversarial examples.
• Preprocessing Defenses: Includes Feature Squeezing, Spatial Smoothing, and JPEG Compression to neutralize perturbations before inference.
• Post-processing Defenses: Implements Gaussian Data Augmentation and High-Confidence Filtering. These wrappers are classifier-agnostic, working with TensorFlow, PyTorch, and scikit-learn.

Beyond empirical testing, ART integrates formal verification methods to provide mathematical guarantees. The library supports:

• Randomized Smoothing: A probabilistic certification technique that constructs a provably robust classifier by adding Gaussian noise and taking a majority vote.
• Detector Modules: Trainable subsystems that act as binary classifiers to distinguish legitimate inputs from adversarial examples, using methods like Local Intrinsic Dimensionality (LID). This moves security analysis from best-effort testing to provable robustness against Lp-norm bounded perturbations.

ART is not limited to image classification. It extends adversarial testing to natural language processing (NLP), automatic speech recognition (ASR), and object detection models. The library supports all major deep learning backends:

• TensorFlow v2 / Keras
• PyTorch
• scikit-learn
• XGBoost / LightGBM
• ONNX This framework-agnostic design allows enterprise security teams to maintain a single testing codebase across heterogeneous production environments.

ART provides a standardized toolkit for quantifying model vulnerability. Researchers can programmatically measure:

• Empirical Robustness: The minimum perturbation distance required to change a classification.
• Clever Score: A composite metric evaluating attack transferability and defense resilience.
• Worst-Case Guarantees: Lower bounds on certified accuracy. This structured evaluation framework enables reproducible benchmarking against the MITRE ATLAS threat matrix, ensuring consistent security posture reporting.