Feature Attribution

The degree to which an attribution map accurately reflects the model's true decision logic. A faithful map identifies the nucleotides that causally influence the prediction.

• Perturbation-based validation: Masking highly attributed nucleotides should cause a sharp drop in prediction probability.
• Metrics: Area Over the Perturbation Curve (AOPC) and Remove And Retrain (ROAR) quantify this property.
• Contrast with Infidelity: Infidelity measures the expected error between input perturbations and their effect on the attribution map itself.

An attribution method satisfies completeness if the sum of all feature importance scores equals the difference between the model's output for the actual input and a neutral baseline.

• Axiomatic guarantee: Integrated Gradients is designed specifically to satisfy this property via path integration.
• Practical utility: Ensures no importance is 'lost' or 'created' during the explanation process.
• Baseline dependency: The choice of a neutral reference sequence (e.g., all zeros, shuffled sequence) critically impacts the resulting attribution map.

Two functionally equivalent models, regardless of their internal architecture or parameterization, should yield identical attribution maps for the same input.

• Functional equivalence: Models that always produce the same output for every possible input are functionally equivalent.
• Method sensitivity: SHAP and Integrated Gradients are implementation invariant, while methods relying on raw gradients or specific network structures may not be.
• Sanity check failure: A method that fails this property is sensitive to irrelevant aspects of the model, not just its input-output mapping.

An attribution method must be sensitive to the model's learned parameters and the input. It should assign a non-zero score to any feature that, when changed, alters the prediction.

• Sensitivity-n axiom: If a single nucleotide differs between two inputs that produce different predictions, that nucleotide must receive a non-zero attribution.
• Sanity checks: Model parameter randomization tests verify this. If attributions don't change after randomizing the model's weights, the method is insensitive to the model itself.
• Contrast with continuity: A sensitive method should still produce stable attributions under small, inconsequential input noise.

The granularity at which importance scores are assigned, ranging from coarse region-level to fine nucleotide-level.

• Nucleotide-level attribution: Assigns a score to every individual base pair, essential for identifying single-nucleotide variants (SNVs) driving predictions.
• Motif-level attribution: Methods like TF-MoDISco cluster high-scoring subsequences into recurring, biologically meaningful motifs.
• Layer-specific resolution: Grad-CAM produces coarse saliency maps from final convolutional layers, while gradient-times-input methods provide finer input-level resolution.

The practical feasibility of computing attributions, especially for large genomic models processing megabase-scale sequences.

• Model-agnostic methods: KernelSHAP is exponentially complex in the number of features, making it intractable for whole-genome inputs without approximations.
• High-speed approximations: DeepSHAP and DeepLIFT leverage backpropagation rules for near-instant computation.
• ISM cost: In-silico mutagenesis requires a forward pass for every possible mutation (3 × sequence length), which is computationally expensive for large models.