Black-Box Attack

The attacker interacts with the target model solely through an API or interface that returns output labels, confidence scores, or logits for submitted inputs. No access to gradients, architecture diagrams, or training data is available. Attack strategies are categorized by the information returned: score-based attacks exploit continuous confidence values, while decision-based attacks operate with only the final hard-label prediction. The attacker must balance query efficiency against detection risk, as excessive querying can trigger rate limiting or anomaly detection systems.

Adversarial examples generated against one model often transfer to other independently trained models performing the same task. This property enables surrogate model attacks: the adversary trains a local substitute model on synthetically labeled data obtained by querying the target, generates white-box attacks against the surrogate, and deploys them against the black-box target. Transferability is strongest between models sharing similar architectures or trained on overlapping data distributions, making it a critical vulnerability in production ML systems.

When the target model returns continuous confidence scores or logits, attackers can estimate gradients numerically through finite-difference methods. The Zeroth-Order Optimization (ZOO) approach approximates the gradient of the loss with respect to the input by evaluating the model at nearby points. More efficient variants like Natural Evolution Strategies (NES) and SPSA use population-based or simultaneous perturbation methods to reduce query counts. These attacks can achieve white-box-level success rates but typically require thousands to millions of queries.

Operating with only the hard-label prediction (the model's final class decision), decision-based attacks explore the decision boundary directly. The Boundary Attack starts from a large-adversarial example and walks along the decision frontier toward the original input, reducing perturbation while maintaining misclassification. HopSkipJumpAttack improves efficiency by estimating gradient direction at the boundary. These attacks are query-intensive but represent the most realistic threat model for deployed APIs that return only class labels.

Practical black-box attacks must balance attack success rate against query budget and detection avoidance. Defenses monitor for anomalous query patterns: high-frequency submissions, near-duplicate inputs, or systematic input-space exploration. Attackers respond with query reduction techniques—using priors, dimensionality reduction, or active subspace learning to minimize interactions. Production-grade attacks often operate within strict query limits (e.g., under 10,000 queries) to evade rate limiting and statistical anomaly detection systems.

Black-box constraints extend to the physical world, where attackers cannot access the digital model but can observe system behavior. Expectation Over Transformation (EOT) generates robust perturbations by optimizing over a distribution of real-world conditions—lighting, angles, distances—without requiring model internals. Physical attacks like adversarial patches or LiDAR spoofing are deployed against autonomous vehicles and surveillance systems, where the attacker observes only the system's behavioral response to physical stimuli rather than any internal state.