Counterfactual Explanations

The core principle is to find the sparsest possible perturbation. A valid counterfactual must change the prediction, but a good counterfactual does so by altering the fewest features. In genomic models, this means identifying the smallest number of nucleotide substitutions, insertions, or deletions that flip the classification.

• Objective: Minimize the L0 or L1 distance between the original and counterfactual sequence.
• Biological Relevance: A single pathogenic SNP identified as a counterfactual is more actionable than a diffuse 100-base-pair edit.

The generated counterfactual sequence must reside on the natural data manifold. It cannot suggest biologically impossible edits, such as violating the genetic code or proposing nucleotide combinations never observed in nature.

• Constraint: Edits must respect evolutionary conservation and known motif syntax.
• Implementation: Often enforced via a generative model prior or a nearest-neighbor search against a reference population panel to ensure the counterfactual is a realistic, viable sequence.

Counterfactuals provide a local causal explanation by answering: 'What is the closest possible world where the outcome is different?' This contrasts with feature attribution methods that merely highlight correlations.

• Mechanism: Directly manipulates the input to observe output change.
• Advantage: Avoids the interpretability illusion where high-attention features are not actually causal drivers of the prediction.

In clinical genomics, a counterfactual explanation directly maps to variant effect prediction. If changing a reference allele to an alternate allele flips a model's prediction from 'benign' to 'pathogenic,' that specific edit is a high-priority functional variant.

• Use Case: Prioritizing variants of unknown significance (VUS) in rare disease diagnosis.
• Output: A ranked list of specific nucleotide edits and their predicted functional impact, providing a direct link between model logic and biological hypothesis generation.

A single input can have multiple valid counterfactuals. Generating a diverse set of explanations reveals different pathways to flip a prediction, exposing the model's decision boundary geometry.

• Method: Determinantal Point Processes (DPP) or latent space sampling to enforce diversity among generated counterfactuals.
• Genomic Insight: One counterfactual might suggest a coding change, while another suggests a regulatory splice-site disruption, offering a richer mechanistic understanding.

Counterfactuals are distinct from adversarial examples. While both flip predictions, counterfactuals are constrained to be plausible and interpretable, whereas adversarial perturbations are imperceptible noise designed to exploit model fragility.

• Counterfactual: A real, actionable biological variant.
• Adversarial: A synthetic, fragile perturbation that does not generalize.
• Goal: Counterfactuals explain the task, adversarial examples expose the model's blind spots.

A systematic perturbation technique that computationally mutates every nucleotide in a sequence to quantify its impact on model predictions. Unlike counterfactuals that seek a minimal set of changes, ISM performs an exhaustive saturation mutagenesis scan.

• Computes a mutation map showing prediction delta at every position
• Directly comparable to experimental Deep Mutational Scans (DMS)
• Computationally expensive for long sequences

A unified framework based on Shapley values from cooperative game theory that assigns each genomic feature an importance score for a particular prediction. SHAP satisfies key axiomatic properties including local accuracy and consistency.

• KernelSHAP: model-agnostic, kernel-based approximation
• DeepSHAP: high-speed variant combining DeepLIFT rules with Shapley calculations
• Provides both nucleotide-level and region-level attributions

An axiomatic feature attribution method that computes the path integral of gradients from a baseline input to the actual input. This method satisfies the completeness axiom, ensuring attributions sum to the prediction difference.

• Baseline is typically a neutral or reference sequence
• Captures saturation effects that simple gradients miss
• Widely adopted for regulatory genomics models

A method that clusters high-contribution genomic subsequences identified by attribution maps into recurring, biologically meaningful motif patterns. It bridges the gap between raw importance scores and interpretable sequence logos.

• Input: per-nucleotide attribution scores from any method
• Output: consensus motifs aligned with known transcription factor binding sites
• Enables discovery of novel regulatory syntax

Quantitative measures that evaluate how accurately an attribution map reflects the true decision-making logic of a genomic model through perturbation experiments.

• ROAR: iteratively retrains model after removing top-attributed features
• AOPC: measures prediction drop as salient nucleotides are sequentially perturbed
• Infidelity Measure: quantifies expected error between input perturbation and attribution perturbation

A suite of tests designed to verify that an attribution method is sensitive to the learned parameters of the genomic model rather than just the input data.

• Model parameter randomization: attributions should degrade when weights are scrambled
• Data randomization: attributions should change when labels are shuffled
• Essential for validating any explainability pipeline before biological interpretation