Black-Box Attack

The attacker probes the model by submitting inputs and observing only the final prediction or confidence scores. By analyzing how outputs change between queries, the adversary can estimate the gradient of the target model without ever computing it directly. Techniques like finite-difference methods approximate the derivative by evaluating the function at nearby points, enabling gradient-based attacks even in a black-box setting.

Black-box attacks are categorized by the granularity of model output:

• Score-based: The model returns continuous confidence scores or logits, allowing the attacker to use zeroth-order optimization to craft adversarial examples.
• Decision-based: The model returns only the hard class label, forcing the attacker to rely on random walks or boundary-following algorithms that start from an adversarial point and minimize distortion while staying adversarial.

A defining property of black-box attacks is adversarial example transferability. The attacker trains a local surrogate model on a similar task, crafts white-box adversarial examples against the surrogate, and then deploys them against the black-box target. Because adversarial perturbations often exploit shared geometric features of decision boundaries across architectures, these transferred examples frequently succeed without any query to the target model.

Real-world black-box attacks must balance attack success rate against the query budget. Defenders can detect excessive querying through rate limiting or statistical anomaly detection. Efficient attacks like Natural Evolution Strategies (NES) or Bandit optimization use Bayesian methods to estimate gradients with far fewer queries than naive finite-difference approaches, making them harder to detect and block.

Without access to true gradients, black-box attacks rely on zeroth-order optimization algorithms that estimate the gradient direction using only function evaluations. Methods such as ZOO (Zeroth Order Optimization) use coordinate descent with symmetric difference quotients to approximate gradients one pixel at a time, enabling effective attacks against deep neural networks protected behind APIs.

Black-box attacks are the only viable threat model for physical-world adversarial attacks, where the attacker cannot access the deployed model's internals. An adversary can print adversarial patches or construct perturbed objects, photograph them, and submit the images to a cloud-based vision API. Success in this setting demonstrates that black-box vulnerabilities persist across domain shifts from digital to physical.

The property that enables black-box attacks in practice. An adversarial example crafted against a surrogate model often fools a different target model. Key factors affecting transferability:

• Model architecture similarity: Examples transfer better between CNNs than between CNNs and transformers
• Ensemble attacks: Crafting examples against multiple surrogate models simultaneously improves transfer rates
• Input transformations: Applying random resizing or padding during generation increases cross-model efficacy

The geometric study of the classification surface that separates classes in high-dimensional input space. Black-box attacks rely on query-based boundary exploration to estimate this surface without gradient access. Key properties:

• Boundary proximity: Natural inputs often lie close to decision boundaries
• Local curvature: Sharp, non-linear boundaries correlate with vulnerability
• Margin estimation: Attackers estimate the distance to the boundary through binary queries

The core mechanism of score-based black-box attacks. The attacker uses model confidence scores to numerically estimate gradients via finite differences:

• Symmetric difference quotient: Perturbing inputs in opposite directions to approximate the gradient
• Zeroth-order optimization: Algorithms like NES and SPSA that optimize without true gradients
• Query efficiency: The number of model queries required directly determines attack practicality in rate-limited settings

The primary defense against both white-box and black-box attacks. Training involves augmenting the dataset with adversarial examples labeled with their correct class. For black-box robustness specifically:

• Ensemble adversarial training: Training against examples from multiple surrogate models improves generalization
• Trade-off: Robustness typically comes at the cost of reduced accuracy on clean data
• Certified defenses: Methods like randomized smoothing provide provable guarantees against norm-bounded attacks

Defines the attack capacity—the maximum allowed modification to an input, typically constrained by an Lp-norm:

• L-infinity norm: Limits the maximum change to any single pixel (common for image attacks)
• L2 norm: Constrains the Euclidean distance between original and adversarial input
• L0 norm: Limits the number of pixels that can be modified Black-box attacks must find effective perturbations within these budgets without gradient guidance.