Adversarial Perturbation Detection

The foundational principle that perturbations must be visually or audibly undetectable to human observers while remaining statistically lethal to classifiers. Detection systems exploit the inherent tension between human perception and model sensitivity.

• Lp-norm bounding: Perturbations are constrained by L0, L2, or L∞ distance metrics to limit pixel-level deviation
• Just Noticeable Difference (JND): Attackers calibrate noise to sit below human perceptual thresholds
• Frequency masking: Exploits the human visual system's insensitivity to high-frequency patterns in textured regions

Detection countermeasures search for statistical anomalies that persist even when the perturbation is invisible to the naked eye.

Rather than analyzing raw pixels, detection systems examine the high-dimensional feature representations learned by neural networks. Perturbed samples occupy distinct, often out-of-distribution regions in the model's latent space.

• Neural activation analysis: Monitors layer-wise activation patterns for statistical deviations from clean baseline distributions
• Mahalanobis distance scoring: Measures the distance of a sample's feature vector from class-conditional Gaussian distributions fitted to clean data
• Local Intrinsic Dimensionality (LID): Characterizes the dimensional properties of data submanifolds, where adversarial examples exhibit higher LID scores than natural samples

This approach generalizes across attack types because it targets the shared mechanism of feature space displacement rather than specific perturbation patterns.

A defensive preprocessing strategy that attempts to purify inputs before classification by removing the adversarial noise component. The core assumption is that perturbations are fragile, high-frequency signals that can be filtered without destroying semantic content.

• Autoencoder-based denoising: Trains a reconstruction network to project inputs back onto the manifold of clean data
• JPEG compression squashing: Applies aggressive compression to destroy subtle perturbation patterns before inference
• Total Variation Minimization: Removes high-frequency noise while preserving edge structures and image content
• Randomized smoothing: Adds calibrated Gaussian noise to disrupt precise adversarial gradients

The primary trade-off is between perturbation removal efficacy and preservation of fine-grained features necessary for correct classification.

A meta-detection approach that identifies when an attacker has employed gradient obfuscation techniques to prevent white-box attack generation. Detection systems probe the model's loss landscape to distinguish genuine robustness from gradient masking.

• Transfer attack testing: Evaluates whether black-box attacks generated on a surrogate model transfer successfully, exposing gradient masking failures
• Numerical gradient checking: Computes finite-difference gradient approximations and compares them to analytical gradients to detect shattered or vanishing gradients
• One-step vs. iterative attack comparison: Gradient masking causes one-step attacks to fail while iterative attacks succeed, revealing a brittle defense

This characteristic is critical for evaluating whether a detection system provides genuine robustness or merely a false sense of security through obfuscation.

The gold standard of perturbation detection: providing mathematically provable bounds on detection performance within a defined perturbation radius. Unlike empirical defenses, certified methods offer verifiable guarantees against any attack within the threat model.

• Randomized smoothing certification: Derives a certified radius around each input where the prediction remains provably constant under L2-norm perturbations
• Interval bound propagation: Propagates input perturbation bounds through the network to compute guaranteed output ranges
• Semidefinite programming relaxations: Solves convex relaxations of the non-convex verification problem to certify robustness properties

Certified methods provide the strongest form of assurance but typically impose significant computational overhead and may be overly conservative compared to empirical approaches.

The capacity of a detection system to withstand attacks specifically designed to circumvent it. A detection mechanism is only as strong as its performance against an adversary with full knowledge of the defense architecture.

• Backward Pass Differentiable Approximation (BPDA): Attackers approximate non-differentiable defense components with smooth functions to compute usable gradients
• Expectation over Transformation (EOT): Accounts for randomized preprocessing by optimizing over the expected loss distribution
• Defense-aware loss functions: Attackers incorporate detection evasion terms directly into their optimization objectives

True detection robustness requires evaluation against this adaptive threat model, where the attacker knows the defense and actively works to defeat it. Systems that fail adaptive evaluation provide only a false sense of security.