Feature Squeezing

The core mechanism involves applying a feature squeezer—a transformation that coalesces multiple input features into a single one. Common squeezers include bit-depth reduction (reducing color depth in images) and spatial smoothing (median or Gaussian filtering). These operations remove high-frequency perturbations that adversarial attacks rely on, while preserving the semantic content necessary for legitimate classification.

Detection is performed by comparing the model's output vectors for the original and squeezed inputs using a distance metric:

• L1 norm: Sum of absolute differences between probability vectors
• Maximum difference: The single largest probability delta across all classes If the divergence exceeds a calibrated threshold, the input is flagged as adversarial. This threshold is typically tuned on a validation set to balance false positives and detection rate.

Multiple squeezers can be combined into a joint detection framework to increase robustness against adaptive attackers. For example, a system might apply both bit-depth reduction and median smoothing independently, then flag an input if any squeezer produces a prediction divergence above its threshold. This ensemble approach makes it significantly harder for an attacker to craft a perturbation that survives all squeezing operations simultaneously.

Feature squeezing operates as an external wrapper around the target model, requiring no modification to the model's architecture, weights, or training procedure. This makes it deployable as a pre-processing defense in front of existing production models without retraining. It is compatible with any classifier that outputs softmax probability vectors, including deep neural networks, SVMs, and decision trees.

Unlike adversarial training or input reconstruction defenses, feature squeezing adds minimal computational overhead:

• Squeezer application: O(n) operations for simple filters like bit-depth reduction
• Inference cost: Only two forward passes (original + squeezed) per input This makes it suitable for real-time detection in high-throughput production environments where latency budgets are tight.

Feature squeezing is vulnerable to adaptive white-box attackers who are aware of the specific squeezer being used. An attacker can incorporate the squeezing operation into their loss function during perturbation generation, crafting adversarial examples that are invariant to the squeezer. This has led to the development of randomized squeezing and multiple squeezer ensembles to increase unpredictability and resilience against adaptive strategies.

A proactive hardening method rather than a detection filter. The model is trained on a mixture of clean and adversarially perturbed examples to learn robust decision boundaries. Unlike feature squeezing, which acts as a runtime monitor, adversarial training bakes resilience directly into the model's weights. Standard implementations use Projected Gradient Descent (PGD) to generate worst-case perturbations during each training batch.

A brittle defense phenomenon that feature squeezing is explicitly designed to avoid. Gradient masking occurs when a defense obfuscates the model's gradients, giving a false sense of security against white-box attacks. Attackers can bypass it using black-box transfer attacks or decision-based attacks. Feature squeezing avoids this pitfall by not modifying the protected model at all—it operates as an external, non-differentiable preprocessor.

A certifiable defense that provides mathematical guarantees of robustness within a specified L2 radius. It constructs a smoothed classifier by adding isotropic Gaussian noise to inputs and returning the most probable prediction. While feature squeezing offers heuristic detection, randomized smoothing provides a provable lower bound on the perturbation magnitude required to flip a prediction. The trade-off is a significant drop in natural accuracy on complex datasets like ImageNet.

A broader anomaly detection paradigm that identifies inputs semantically different from the training distribution. Feature squeezing is a specialized subset focused on adversarial perturbations—inputs that are visually identical to humans but mathematically anomalous. General OOD detectors use methods like Mahalanobis distance scores or energy-based models to flag inputs from unknown classes or corrupted samples, not just adversarially manipulated ones.

A powerful optimization-based white-box attack that feature squeezing detectors must be evaluated against. The C&W attack formulates adversarial example generation as minimizing a distance metric subject to a misclassification constraint, solved via gradient descent. It produces minimal-distortion perturbations that are harder to detect than FGSM noise. Effective squeezing configurations must demonstrate resilience specifically against C&W-generated examples.