FullGrad

FullGrad is one of the few attribution methods that provably satisfies the completeness axiom. The sum of all feature attributions exactly equals the model's output difference from a baseline. This is achieved by decomposing the output into the input-gradient contribution and the bias-gradient contributions from every neuron in the network. Unlike methods that approximate or violate completeness, FullGrad provides a mathematically exact accounting of the prediction.

The core innovation of FullGrad is its aggregation of bias gradients. Standard gradient methods only use the gradient with respect to the input. FullGrad recognizes that the bias parameters in each layer also contribute to the output. By computing the gradient of the output with respect to every bias term and summing these contributions, FullGrad captures the full sensitivity of the network. This provides a more complete and faithful attribution map.

Both FullGrad and Integrated Gradients satisfy the completeness axiom, but through different mechanisms:

• Integrated Gradients requires a baseline input and integrates gradients along a path
• FullGrad requires no baseline and uses only a single forward-backward pass
• FullGrad is computationally more efficient as it avoids the numerical integration step
• The bias-gradient terms in FullGrad act as a spatial prior, naturally smoothing the saliency map without explicit noise injection

FullGrad satisfies the implementation invariance axiom. Two functionally equivalent networks with different internal architectures will produce identical attributions. This is because the method relies on the functional relationship between inputs and outputs, not on specific activation patterns. The bias gradients capture the network's sensitivity structure regardless of how the computation is organized internally.

Computing FullGrad requires:

• A standard forward pass to compute the output
• A backward pass to compute gradients with respect to the input and all bias parameters
• Summation of the input-gradient map with the bias-gradient maps spatially broadcast to match input dimensions
• The result is a single saliency map with the same spatial dimensions as the input

This is implemented efficiently in frameworks like PyTorch using automatic differentiation hooks.

FullGrad adheres to the Sensitivity-n axiom: if the model's output is mathematically independent of a feature, that feature receives zero attribution. The bias-gradient terms ensure that all contributing pathways through the network are accounted for. Features that do not influence any neuron's bias contribution or the direct input gradient are correctly assigned zero importance, preventing false positive attributions.

The mathematical principle that the sum of all feature attributions must equal the difference between the model's output for the target input and a baseline reference. FullGrad satisfies this axiom by design, aggregating the input-gradient with the bias-gradients from every intermediate layer. This ensures no portion of the output signal remains unaccounted for, unlike methods such as vanilla Gradient×Input which violate completeness in deep networks.

An axiomatic method that computes feature importance by accumulating gradients along a straight-line path from a baseline input to the actual input. While Integrated Gradients satisfies completeness through path integration, FullGrad achieves it through a different mechanism—explicitly summing bias gradients across all layers. Both methods address the gradient saturation problem, but FullGrad avoids the need to select a baseline reference.

A first-order Taylor approximation that multiplies the raw gradient by the input value. While computationally simple, this method suffers from shattered gradients in deep networks and violates the completeness axiom. FullGrad can be viewed as an extension that corrects this deficiency by incorporating the neglected bias terms, transforming a partial linearization into a full decomposition of the network's output.

A decomposition technique that redistributes the prediction score backwards through the network using conservation rules at each layer. Like FullGrad, LRP satisfies a conservation property ensuring no relevance is lost during propagation. However, LRP relies on heuristic propagation rules tailored to specific layer types, while FullGrad derives its decomposition directly from the gradient of the biases, providing a mathematically unified framework.

An attribution method that compares neuron activations to a reference activation and assigns contribution scores based on the difference. DeepLIFT addresses gradient saturation by using discrete differences rather than instantaneous gradients. FullGrad similarly avoids saturation issues but does so through bias-gradient aggregation rather than reference-based comparison, eliminating the need to define reference values for each layer.

A technique that sharpens noisy saliency maps by averaging gradients from multiple noisy copies of the same input. While SmoothGrad improves visual coherence, it does not address the completeness problem. FullGrad produces naturally sharper attributions without noise injection because the bias gradients capture signal that would otherwise manifest as high-frequency noise in input-gradient-only methods.