Physical Adversarial Attack

Physical attacks must survive the domain gap between digital optimization and physical realization. The Expectation Over Transformation (EOT) framework addresses this by optimizing perturbations over a distribution of real-world variations:

• Viewpoint changes (rotation, translation, scale)
• Illumination shifts (shadows, glare, color temperature)
• Sensor artifacts (demosaicing, compression, noise)
• Print-to-camera transfer functions Without EOT, a perturbation that fools a digital image will fail when printed and re-photographed.

The most common physical attack vector is the adversarial patch—a localized, printed pattern placed in the scene to suppress detection or induce misclassification. Key properties:

• Location-agnostic: Effective anywhere within the camera's field of view
• Scale-invariant: Works across a range of distances
• Attack goals: Object hiding (causing detectors to ignore a stop sign), misclassification (labeling a person as background), or targeted impersonation Patches exploit the large receptive fields of convolutional networks, overwhelming feature maps with a high-magnitude perturbation signal.

Beyond 2D patches, attackers fabricate adversarial 3D objects that fool multi-view perception systems. These objects are designed to:

• Produce adversarial features from any viewing angle
• Exploit texture, geometry, and specular properties simultaneously
• Defeat LiDAR point cloud classifiers through carefully shaped surfaces
• Survive the sim-to-real gap by training in differentiable renderers Example: A 3D-printed turtle textured with an adversarial pattern that causes an object classifier to consistently label it as a rifle from all angles.

Physical attacks target individual sensor modalities with physics-specific perturbations:

• LiDAR spoofing: Injecting precisely timed laser pulses to create phantom 3D points or delete real obstacles from point clouds
• Radar jamming: Broadcasting interfering waveforms to blind automotive radar
• IMU resonance: Using acoustic frequencies to induce false inertial measurements in MEMS gyroscopes
• Infrared blinding: Overwhelming IR cameras with directed IR emitters Each modality requires domain-specific knowledge of the sensor's physical operating principles.

Rather than adding conspicuous patches, camouflage attacks modify the target object's texture to blend with expected backgrounds while causing misclassification:

• Adversarial camouflage: Patterns that look like normal camouflage to humans but cause neural network failure
• Texture optimization: Generating surface coatings that exploit model-specific feature representations
• Contextual attacks: Placing adversarial perturbations on objects that appear contextually appropriate (e.g., graffiti on a wall that hides a door from a navigation system) These attacks are harder to detect because they lack the obvious visual artifacts of patches.

Attackers can project adversarial patterns using structured light rather than physical modifications:

• Laser projection: Painting adversarial perturbations onto stop signs with a laser pointer
• Projector-based attacks: Using portable projectors to overlay adversarial textures on objects
• Shadow attacks: Manipulating the shape and position of shadows to create adversarial occlusion patterns
• Transient attacks: Brief light pulses that cause misclassification in a single frame without leaving permanent evidence These attacks are particularly dangerous because they require no physical access to the target object and leave no forensic trace.

An attack that weaponizes the discrepancy between a simulated training environment and the physical world. A policy trained extensively in simulation may have blind spots that are invisible during digital testing.

• Exploitation: An adversary identifies a physical texture, lighting condition, or object interaction that was poorly modeled in the simulator.
• Catastrophic Example: A robot arm trained in a zero-friction simulation fails to grasp an object coated with a low-friction adversarial texture, or a drone policy crashes when encountering a visual pattern absent from the domain randomization set.