Expectation Over Transformation (EOT)

Unlike standard attacks that optimize for a single input, EOT computes the expected value of the loss function across a distribution of transformations T. The objective is to find a perturbation δ that fools the classifier over the entire distribution, not just one sample.

• Formulation: argmin_δ E_t~T [L(f(t(x+δ)), y_target)]
• This prevents overfitting to a specific viewpoint or condition.
• The distribution T typically models 2D/3D changes like rotation, translation, and scaling.

The power of EOT lies in the composition of the transformation distribution T. It must accurately simulate the physical world's variability to ensure physical realizability.

• Geometric transforms: 3D rotation, perspective projection, translation, and scaling.
• Photometric transforms: Lighting changes, contrast shifts, Gaussian noise, and motion blur.
• Physical constraints: Clipping perturbations to printable color gamuts and enforcing smoothness to avoid high-frequency patterns that printers cannot reproduce.

EOT is the critical enabler for attacks that transfer from the digital lab to the physical world. By training the perturbation to survive noise and viewpoint shifts, it creates adversarial patches and 3D-printed objects that remain effective.

• Adversarial patches: EOT ensures a patch fools a detector regardless of its placement angle or distance.
• 3D adversarial objects: EOT optimizes a texture over a full distribution of 3D object poses, creating a physical turtle classified as a rifle from every angle.

For gradient-based optimization to work, the transformations in T must be differentiable or use a differentiable approximation. This allows the error signal from the classifier to backpropagate through the transformation to the input perturbation.

• Affine transformations (rotation, scaling) are inherently differentiable.
• Non-differentiable operations like ray-tracing for 3D rendering are often replaced with a differentiable renderer.
• This end-to-end differentiability is what makes joint optimization of the perturbation and transformation robustness possible.

EOT is the standard methodology for rigorously evaluating adversarial robustness claims. A defense that only tests against naive, single-image attacks provides a false sense of security.

• Adaptive attacks: A proper security evaluation must assume the attacker uses EOT to overcome input transformations used as a defense.
• Gradient masking detection: EOT helps identify defenses that rely on obfuscated gradients, as the attack's loss landscape is smoothed over the transformation distribution, revealing true vulnerability.

The primary drawback of EOT is its significant computational overhead. Each optimization step requires sampling multiple transformations and performing a forward-backward pass for each.

• Sample complexity: The number of transformation samples per step directly trades off attack reliability against compute time.
• Variance reduction: Techniques are used to reduce the variance of the gradient estimate over T to speed up convergence.
• This cost is the main barrier to scaling EOT to very large models or high-resolution inputs without substantial GPU resources.

The attack class that EOT was specifically designed to enable. Unlike digital-only perturbations, physical attacks must survive printing, lighting changes, camera noise, and viewpoint shifts. EOT formalizes this by optimizing over a distribution of real-world transformations rather than a single static input. Standard attacks like PGD fail when a sticker is photographed from a different angle; EOT-trained perturbations maintain adversarial efficacy across these nuisance variables.

The primary defense against EOT-generated attacks. Standard adversarial training augments each minibatch with worst-case perturbations found by PGD. To defend against physical attacks, practitioners must use EOT-based adversarial training, generating perturbations that are robust across transformations during the training loop. This forces the model to learn decision boundaries that are invariant to nuisance variables, directly countering the optimization strategy EOT employs.

Formal guarantees that a model's prediction will not change under any perturbation within a specified bound. While EOT generates empirical attacks, certified defenses like randomized smoothing provide provable L2-radius guarantees. The tension: EOT can be used to stress-test certified bounds by searching for physical transformations that violate the assumptions of the certification. A model with a certified radius of 0.3 may still be vulnerable to an EOT-crafted patch that exploits the gap between the certification's mathematical assumptions and physical reality.