Physical Adversarial Attack

The defining characteristic of a physical attack is its ability to survive the domain shift from digital to physical. The perturbation must remain effective after undergoing the image formation pipeline: printing (color gamut reduction, halftoning), photographic capture (sensor noise, lens distortion, compression), and environmental interactions (lighting, viewing angle, distance). This requires the perturbation to be robust to a distribution of transformations, not just a single digital image. Attackers often use Expectation over Transformation (EoT) during optimization, averaging gradients over a range of simulated physical distortions to ensure the adversarial pattern generalizes.

A successful physical attack must induce misclassification across a wide range of real-world conditions. Key variables include:

• Illumination: Shadows, direct sunlight, and artificial lighting alter pixel intensities.
• Viewpoint: Oblique angles cause perspective distortion and foreshortening.
• Occlusion: Partial obstruction of the adversarial pattern.
• Distance: Changes in scale and resolution. Robust attacks are trained with data augmentation simulating these variations, ensuring the adversarial feature dominates the model's decision regardless of context.

Unlike digital attacks that manipulate arbitrary floating-point values, physical attacks must be realizable in the physical world. This imposes constraints:

• Printability: Colors must lie within the printer's CMYK gamut. Non-printable digital colors are clipped, destroying the perturbation.
• Material: The attack may be applied via stickers, posters, 3D-printed objects, or clothing. The material's texture and reflectivity affect appearance.
• Inconspicuousness: To avoid human suspicion, attacks often mimic graffiti, stickers, or artistic patterns that blend into the environment while remaining adversarial to the model.

A dominant form of physical attack is the adversarial patch—a localized, often highly visible pattern that, when placed in a scene, completely overrides the model's perception. Unlike full-image perturbations, patches are:

• Location-agnostic: Designed to work anywhere within the camera's field of view.
• Scale-invariant: Effective at various distances.
• Extremely robust: Often achieving near-100% attack success rates. Patches exploit the model's reliance on high-activation features, presenting a pattern that triggers a specific output neuron more strongly than any natural object in the scene.

Physical attacks serve two primary objectives in object detection and classification systems:

• Evasion (Hiding): The attack causes the model to fail to detect an object entirely. For example, a carefully patterned sticker on a stop sign causes an autonomous vehicle's perception system to classify it as a speed limit sign.
• Impersonation: The attack causes a person or object to be recognized as a specific, unauthorized identity. Adversarial eyeglass frames have been demonstrated to cause facial recognition systems to identify an attacker as a targeted victim, enabling physical impersonation.

Seminal research has validated the physical threat model in operational contexts:

• Stop Sign Attacks: Black and white stickers applied to stop signs caused real-world classifiers to interpret them as speed limit signs with high confidence.
• Adversarial Clothing: Specially designed textile patterns have been shown to make individuals invisible to person detectors.
• 3D Adversarial Objects: 3D-printed turtles were consistently misclassified as rifles by object recognition systems from multiple angles.
• LIDAR Spoofing: Beyond cameras, physical attacks extend to placing carefully shaped objects that inject adversarial points into LIDAR point clouds, creating phantom obstacles.

Researchers demonstrated that placing small black and white stickers on a physical stop sign caused a state-of-the-art image classifier to interpret it as a Speed Limit 45 sign with high confidence. The attack exploited the model's reliance on high-level feature detectors rather than holistic shape recognition. The perturbation was designed to survive changes in distance, angle, and lighting—conditions inherent to real-world autonomous driving scenarios.

Researchers at Carnegie Mellon University created specially designed eyeglass frames printed with adversarial patterns that caused facial recognition systems to misidentify the wearer as a targeted celebrity. The attack exploited the model's sensitivity to fine-grained texture patterns around the eye region. The frames remained effective under varying camera resolutions, head poses, and ambient lighting conditions, demonstrating a practical impersonation vector against biometric security systems.

A team developed a universal adversarial patch that, when placed near an object in a scene, caused YOLO and other real-time object detectors to completely ignore the target object. The patch was printed on cardboard and tested in physical environments. Key characteristics include:

• Translation-invariant: Effective regardless of patch position relative to the object
• Scale-invariant: Survived varying distances from the camera
• Cross-model transferable: Fooled detectors with different architectures The attack demonstrated that physical-world object detection evasion requires no digital access to the target system.

Researchers proposed adversarial camouflage techniques that apply subtle, physically realizable perturbations to entire road sign surfaces. Unlike sticker-based attacks, these modifications mimic normal wear and tear—such as fading, dirt, or graffiti—making them inconspicuous to human observers. The perturbations were optimized using Expectation over Transformation (EoT) to ensure robustness across a distribution of physical transformations including rotation, scaling, and lighting variations. This approach highlights the tension between imperceptibility and physical realizability.

MIT researchers demonstrated that a 3D-printed turtle with a subtly textured shell was consistently classified as a rifle by ImageNet models from nearly every viewing angle. The attack leveraged 3D rendering pipelines during optimization to generate a perturbation that survived the full perspective transformation of physical rotation. This work proved that adversarial examples are not limited to 2D surfaces—volumetric objects can be manufactured to systematically fool classifiers across their entire geometry.

Beyond camera-based attacks, researchers demonstrated physical adversarial attacks against LiDAR sensors used in autonomous vehicles. By placing carefully shaped retroreflective objects at strategic positions, attackers created adversarial point cloud patterns that caused object detection models to perceive phantom obstacles or fail to detect real vehicles. The attack exploited the model's processing of raw 3D point cloud data rather than 2D imagery, highlighting that physical adversarial vulnerabilities extend across sensor modalities critical to autonomous navigation.