Feature Squeezing

The foundational operation of feature squeezing that reduces the color depth of each pixel in an image. For example, an 8-bit grayscale image with 256 possible values is squeezed to 4 bits, leaving only 16 distinct values. This quantization eliminates the subtle, high-frequency perturbations that adversarial attacks rely on while preserving the semantic content visible to humans. The operation is computationally negligible, requiring no retraining of the target model.

The core detection logic compares the model's output on the original input against its output on the squeezed version using a distance metric such as L1 norm or maximum probability difference. If the divergence exceeds a calibrated threshold, the input is flagged as adversarial. This exploits a key asymmetry: legitimate inputs produce consistent predictions under squeezing, while adversarial perturbations collapse, causing dramatic shifts in the model's confidence or class assignment.

Feature squeezing is most effective when multiple squeezers are combined into a joint detection system. Common squeezers include:

• Color depth reduction: Quantizing pixel values to fewer bits
• Spatial smoothing: Applying median or Gaussian filters to blur local perturbations
• Non-local means: Reducing noise while preserving edges The outputs of all squeezers are fed into a voting or max-diff scheme, significantly increasing detection rates against diverse attack types including FGSM, PGD, and C&W.

The detection threshold must be carefully calibrated on a clean validation set to balance false positives and true detection rates. The threshold is typically set at a percentile of the maximum divergence observed across legitimate samples. A threshold at the 95th percentile of clean-sample divergences yields a 5% false positive rate. This calibration is attack-agnostic, meaning the defense does not require prior knowledge of the specific adversarial method being used.

Feature squeezing is not a robustness guarantee but a detection heuristic. An adaptive attacker aware of the defense can craft perturbations that survive squeezing by constraining the perturbation to low-frequency components or by incorporating the squeezing operation into the attack optimization loop. Additionally, grayscale-only squeezers fail against color-channel attacks. The defense is best deployed as one layer in a broader security ensemble alongside adversarial training and input reconstruction methods.

Unlike adversarial training or certified defenses, feature squeezing imposes zero training overhead and minimal inference cost. The squeezing operations are simple, parallelizable image transformations that run in microseconds on a CPU. Detection requires only one additional forward pass per squeezer through the existing model. This makes feature squeezing particularly suitable for real-time systems and resource-constrained edge deployments where retraining is impractical or the model is a black-box third-party component.

A proactive defense that injects adversarial examples into the training dataset with correct labels, forcing the model to learn a smoother decision boundary.

• Augments training data with PGD-generated perturbations
• Increases robustness against white-box attacks
• Trade-off: 5-15% accuracy drop on clean data
• Considered the strongest empirical defense against L-infinity attacks

The property by which an adversarial example crafted to fool Model A also causes misclassification in independently trained Model B with a different architecture.

• Enables black-box attacks without query access
• Higher transferability between models with similar architectures
• Ensemble-based attacks maximize transfer rates
• Key reason why model secrecy is not a defense

A brittle defense phenomenon where a model's gradients become useless or misleading to gradient-based attacks, creating a false sense of security.

• Often caused by non-differentiable preprocessing or saturation
• Attackers bypass it using black-box transfer attacks
• Feature squeezing avoids this pitfall by not relying on gradient obfuscation
• Distinguished from true robustness by Athalye et al. (2018)

A formal, verifiable guarantee that a model's prediction remains constant for any perturbation within a specified Lp-norm radius.

• Deterministic certificates: exact bounds via SMT/MILP solvers
• Probabilistic certificates: high-confidence bounds via randomized smoothing
• Feature squeezing provides no formal guarantees—it is an empirical detection heuristic
• Certificates are the gold standard but scale poorly to large models

The maximum allowed magnitude of an adversarial perturbation, typically constrained by an Lp-norm such as L-infinity (epsilon).

• L-infinity: caps maximum per-pixel change (common: epsilon = 8/255)
• L2: limits Euclidean distance of perturbation vector
• L0: restricts number of modified pixels
• Feature squeezing is most effective against high-epsilon attacks that rely on subtle, distributed noise patterns