DeepLIFT

Unlike standard gradient methods, DeepLIFT does not rely on a local derivative at the input. Instead, it defines a reference input (e.g., a shuffled genomic sequence or all-zero embedding) and explains the difference in the output relative to this reference. This allows the method to propagate importance even when the gradient is zero, effectively handling the saturation problem common in deep networks with sigmoid or ReLU activations. The choice of reference is critical; for genomic variant effect prediction, a common reference is the ancestral allele or a dinucleotide-shuffled version of the sequence.

The Rescale rule is the foundational linear propagation mechanism in DeepLIFT. It assumes that the contribution of an input neuron to an output neuron is proportional to its fractional contribution to the total net input. Mathematically, if a neuron has an activation difference of Δx from the reference, and the total sum of positive inputs is ΣΔx⁺, the attribution is scaled by Δx / ΣΔx⁺. This rule is particularly efficient for layers like Dense and Convolutional layers in genomic models, providing a clean decomposition that satisfies summation-to-delta (the sum of all feature attributions equals the difference in output from the reference).

The RevealCancel rule is designed to handle non-linear interactions where positive and negative inputs compete. It treats positive and negative contributions separately, preventing them from canceling each other out during backpropagation. This is crucial for identifying regulatory motifs in genomic sequences where specific nucleotide combinations may have synergistic or antagonistic effects. By preserving the sign of contributions, RevealCancel reveals hidden antagonistic interactions that the simpler Rescale rule might mask, providing a more faithful attribution map for complex biological phenomena like enhancer-promoter interactions.

A key axiomatic guarantee of DeepLIFT is the summation-to-delta property. This states that the sum of all feature attribution scores across the input sequence exactly equals the difference between the model's output for the actual sequence and its output for the reference sequence. This property, analogous to the completeness axiom in Integrated Gradients, ensures that no importance is 'lost' during propagation. For a genomic variant classifier, this means the sum of nucleotide-level importance scores perfectly accounts for the change in the predicted pathogenicity score, providing a complete and auditable explanation.

A major failure mode of standard gradient-based methods is saturation. When a neuron's output is near a flat region of its activation function (e.g., a ReLU outputting zero or a sigmoid near 1.0), the local gradient is near zero, incorrectly suggesting the input has no importance. DeepLIFT bypasses this by using discrete differences between the actual activation and the reference activation. This allows it to assign high importance to a critical transcription factor binding site even if the convolutional filter detecting it is operating in a saturated regime, providing robust explanations for deep genomic networks.

The reference input is a foundational concept in DeepLIFT. It represents a neutral or uninformative input against which the actual sequence is compared. The choice of reference is critical.

• For genomic sequences, a common reference is a sequence of zeros or the expected background frequency of nucleotides.
• Activations are computed as the difference from this reference, allowing the model to explain why a prediction deviates from the 'null' state.
• A poorly chosen reference can lead to misleading attributions that highlight irrelevant features.

The Rescale Rule is the primary method for propagating importance scores through a network in DeepLIFT. It assumes a linear relationship between a neuron's input and output.

• The contribution of an input neuron x to an output neuron y is calculated as C(x) = (Δy / Δx) * Δx, where Δ is the difference from the reference.
• This rule efficiently backpropagates the difference-from-reference signal, assigning credit to inputs in proportion to their effect on the output.
• It works well for layers with saturating activations like Sigmoid or Tanh, where gradients can be near zero.

The RevealCancel Rule provides a more nuanced attribution by separately tracking positive and negative contributions, preventing them from canceling each other out.

• It splits the total contribution into a 'reveal' component (positive contributions) and a 'cancel' component (negative contributions that offset the reveal).
• This is crucial for understanding complex regulatory logic in genomics, where an activator and a repressor might bind nearby, with their effects masked in a simple sum.
• It treats the model as if it's computing a net positive effect minus a net negative effect, offering a more complete picture of the decision logic.

Both methods satisfy the completeness axiom, meaning the sum of all feature attributions equals the difference in the model's output from a baseline.

• DeepLIFT uses a single reference point and backpropagates discrete differences, making it computationally faster.
• Integrated Gradients computes the path integral of gradients from a baseline to the input, requiring multiple steps for accurate approximation.
• For genomic models, DeepLIFT's speed is advantageous for scanning millions of variants, while Integrated Gradients offers a theoretically robust axiomatic foundation.

DeepSHAP is a high-speed approximation algorithm that combines DeepLIFT's efficient backpropagation rules with the theoretical guarantees of Shapley values.

• It interprets the reference value as the expected model output, allowing DeepLIFT's multipliers to approximate Shapley value propagation.
• This provides a unified framework that is both computationally efficient and grounded in cooperative game theory.
• It is particularly useful for explaining predictions from deep genomic sequence models where computing exact Shapley values is computationally prohibitive.

In-silico Mutagenesis (ISM) is a systematic perturbation method used to validate DeepLIFT attributions. It computationally mutates every nucleotide to quantify its impact.

• A strong correlation between a nucleotide's DeepLIFT score and the change in model prediction from an ISM scan confirms the attribution's faithfulness.
• This comparison can reveal where DeepLIFT's linear approximation fails to capture non-linear, epistatic interactions between nucleotides.
• Together, they provide a powerful lens: DeepLIFT for rapid, hypothesis-generating saliency maps, and ISM for rigorous, exhaustive verification.