Layer-wise Relevance Propagation

The foundational mathematical framework underpinning many LRP rules. It explains a model's prediction by performing a Taylor expansion of the output function at a carefully chosen root point—an input where the function value is zero. The first-order terms of this expansion are then redistributed backward through the layers. This provides a theoretical justification for why LRP's redistribution rules produce meaningful heatmaps, linking the heuristic propagation directly to functional decomposition.

A critical conservation property that LRP strictly enforces. The axiom states that the sum of all relevance scores assigned to the input features must exactly equal the model's output prediction score for the target class. This is not an approximation; LRP's propagation rules are designed so that no relevance is created or destroyed during redistribution. This property ensures that the explanation is a true decomposition of the evidence, not just a vague indication of importance.

LRP is not a single algorithm but a family of rules dictating how relevance flows through different layer types. The most common rules include:

• LRP-ε: Adds a small constant to the denominator to absorb weak or contradictory relevance, improving numerical stability.
• LRP-αβ: Separates positive and negative activations, allowing the user to control the influence of inhibitory evidence (typically α=1, β=0 to focus on positive contributions).
• LRP-γ: A rule preferred for deep networks that adds a positive term to bias relevance towards positive contributions, often producing sharper, less noisy heatmaps.

A theoretical framework that views the backward pass of LRP as a continuous, conservative flow of a quantity through the network's computational graph. It unifies various decomposition methods by showing that they correspond to different choices of how to route the flow at network junctions. This perspective helps engineers design new propagation rules by defining specific flow constraints, ensuring that the resulting attributions satisfy the conservation law by construction.

A simple baseline attribution method that multiplies the input-gradient by the input value itself. While computationally cheap, it often produces noisy, visually incoherent saliency maps due to shattered gradients. LRP fundamentally differs by using a deep redistribution process that leverages the network's learned weights and activations to filter noise. Unlike Gradient × Input, LRP satisfies the completeness axiom by design, providing a true decomposition of the output score rather than a first-order Taylor approximation.

A visualization technique that modifies the standard backward pass by only allowing positive gradients and positive activations to flow through ReLU layers. While it produces very sharp, high-contrast saliency maps, it violates the sensitivity-n axiom and does not satisfy completeness. LRP is generally preferred for quantitative analysis because its conservation property ensures the relevance scores are a true decomposition of the prediction, whereas Guided Backpropagation is a heuristic that highlights detected features regardless of their actual contribution to the output.