Extremal Perturbation

Unlike standard perturbation methods that apply arbitrary masks, Extremal Perturbation explicitly constrains the mask area to a user-specified fraction of the input. The optimization solves for the smallest smooth region that, when preserved, retains the target class score. This produces compact, blob-like explanations that avoid the scattered noise typical of gradient-based saliency maps. The area constraint acts as an implicit regularizer, forcing the method to identify the most spatially coherent discriminative features rather than distributing importance diffusely across the input.

The method enforces spatial smoothness on the perturbation mask using a total variation (TV) regularizer in the optimization objective. This penalizes sharp transitions between preserved and perturbed regions, yielding soft, continuous masks rather than hard binary cutouts. The smoothness constraint is critical for producing visually coherent saliency maps and prevents the optimizer from exploiting high-frequency artifacts that could artificially inflate the preservation score. The balance between the area constraint and TV regularization is controlled by a hyperparameter λ.

Extremal Perturbation supports two complementary explanation modes:

• Preservation game: Find the smallest region that must be preserved to maintain the model's original prediction confidence. This answers 'What is the minimal evidence needed?'
• Deletion game: Find the smallest region that, when removed, maximally destroys the prediction. This answers 'What is the most critical evidence?' Both modes are solved through the same optimization framework by simply inverting the objective function, providing symmetric interpretability guarantees.

The mask parameters are optimized directly in the input space using stochastic gradient descent on the perturbation function, but the method does not require backpropagating gradients through the model's internal layers. The model is treated as a black-box scoring function. This makes Extremal Perturbation applicable to non-differentiable models and scenarios where gradient access is restricted, such as models served behind APIs. The optimization loop iteratively refines the mask by querying the model's output on perturbed versions of the input.

Unlike Integrated Gradients or SmoothGrad which derive attributions from model gradients, Extremal Perturbation directly optimizes the input mask to answer a counterfactual question. This perturbation-based approach avoids gradient saturation issues common in deep networks and produces explanations that are inherently causally grounded—the mask's effect is measured by actual model outputs, not local linear approximations. The trade-off is higher computational cost, as each optimization step requires a forward pass through the model.

The quality of Extremal Perturbation masks is quantitatively evaluated using the Insertion and Deletion metrics:

• Deletion: Pixels are removed in order of importance; a sharp drop in confidence indicates accurate attribution.
• Insertion: Pixels are added from most to least important into a blurred baseline; a rapid rise in confidence confirms the mask captures the essential evidence. Extremal Perturbation consistently achieves higher Area Under the Curve (AUC) scores on these metrics compared to gradient-based methods, validating its compactness advantage.