Infidelity Measure

Infidelity is formally defined as the expected squared difference between the dot product of the perturbation vector and the attribution map, and the difference in model output caused by that perturbation.

• Core Equation: INFD(Φ, f, x) = E_{I∼μ_I}[ (I^T Φ(f, x) - (f(x) - f(x - I)))^2 ]
• Perturbation Vector (I): A significant, non-local modification to the input sequence, drawn from a distribution μ_I
• Attribution Map (Φ): The feature importance scores generated by an interpretability method
• Model Output Difference: The actual change in the model's prediction when the perturbation is applied
• A lower infidelity score indicates a more faithful explanation

Infidelity is closely tied to the completeness axiom of attribution methods, which requires that the sum of all feature attributions equals the difference between the model output at the input and a baseline.

• If an attribution method satisfies completeness, the infidelity for perturbations that are dense vectors (all features perturbed) will be zero
• Methods like Integrated Gradients and DeepSHAP inherently satisfy completeness
• Infidelity generalizes completeness by testing it across a distribution of perturbations rather than a single baseline
• This makes infidelity a more robust measure than simply checking the summation property

The choice of perturbation distribution μ_I is critical to the meaningfulness of the infidelity metric and must reflect biologically plausible sequence variations.

• Gaussian perturbations: Test sensitivity to small, diffuse noise across all nucleotide positions
• Subspace perturbations: Restrict perturbations to specific genomic regions (e.g., promoters, enhancers) to test localized faithfulness
• Motif-scrambling perturbations: Randomize conserved transcription factor binding sites while preserving dinucleotide composition
• In-silico mutagenesis perturbations: Introduce single-nucleotide variants drawn from known population frequencies (gnomAD)
• Poor perturbation design can yield misleadingly low infidelity scores that don't reflect real-world reliability

Infidelity complements but differs from other perturbation-based faithfulness metrics commonly used in genomic model evaluation.

• vs. AOPC (Area Over the Perturbation Curve): AOPC measures prediction drop when removing top features sequentially; infidelity measures alignment across all features simultaneously
• vs. ROAR (Remove And Retrain): ROAR requires expensive model retraining after feature removal; infidelity is computed on a single trained model
• vs. Sensitivity-n: Sensitivity-n measures correlation with a single perturbation magnitude; infidelity uses a full distribution
• vs. Faithfulness Correlation: Correlation metrics capture linear relationships; infidelity captures exact numerical agreement
• Infidelity is particularly suited for high-dimensional genomic inputs where feature interactions are complex

Infidelity measures are essential for validating attribution maps used to score the functional impact of non-coding variants in clinical genomics pipelines.

• Benchmarking: Compare DeepLIFT, Integrated Gradients, and SHAP attributions against Deep Mutational Scan (DMS) ground truth using infidelity
• Regulatory compliance: Demonstrate to auditors that variant effect scores are derived from faithful explanations of the underlying model
• Model selection: Choose between genomic architectures (e.g., Enformer vs. Basenji2) based on which produces lower infidelity attributions
• Attribution uncertainty: Combine infidelity with bootstrap resampling to compute confidence intervals on variant effect scores
• Example: A variant in a distal enhancer with low infidelity attribution is more trustworthy for clinical reporting

Despite its theoretical rigor, infidelity has several limitations that practitioners must account for when evaluating genomic model explanations.

• Computational cost: Requires many forward passes through the model for each perturbation in the distribution, scaling poorly with sequence length
• Perturbation dependency: Results are only as meaningful as the perturbation distribution; biologically irrelevant perturbations produce uninformative scores
• Attribution scale sensitivity: Infidelity is sensitive to the magnitude of attributions, not just their relative ordering
• No baseline-free variant: Unlike some metrics, infidelity always requires specifying a perturbation distribution, introducing a design choice
• Non-uniqueness: Multiple different attribution maps can achieve identical infidelity scores, so it should be used alongside other metrics like AOPC and ROAR

Quantitative measures that evaluate how accurately an attribution map reflects the true decision-making logic of a genomic model. These metrics rely on perturbation experiments—systematically altering high-attribution nucleotides and measuring the resulting change in model output. A faithful explanation will show a steep, monotonic drop in prediction score as salient features are removed. Key implementations include AOPC (Area Over the Perturbation Curve) and ROAR (Remove And Retrain), which iteratively retrains the model after feature removal to prevent distribution shift artifacts.

The study of how uncertainty in a genomic model's output can be apportioned to different sources of uncertainty in its nucleotide inputs. Unlike feature attribution which assigns a single importance score, sensitivity analysis quantifies the local gradient of the prediction with respect to each input dimension. This is foundational to understanding model robustness: a model with high sensitivity to single-nucleotide perturbations may be unreliable for variant effect prediction in clinical settings. Techniques range from simple partial derivatives to global variance-based methods like Sobol indices.

An experimental perturbation technique that systematically removes or masks genomic regions in-silico to measure their causal effect on model predictions. Ablation studies are the empirical backbone of faithfulness evaluation. Common strategies include:

• Zero ablation: Replacing nucleotides with a null token
• Reference ablation: Substituting with a neutral baseline sequence
• Scrambling: Randomly shuffling the target region while preserving dinucleotide frequency The difference between pre- and post-ablation prediction scores provides a direct causal estimate of a region's importance.

A suite of diagnostic tests designed to verify that an attribution method is genuinely sensitive to the learned parameters of a genomic model, rather than producing plausible-looking but uninformative edge-detectors. The foundational sanity check is model parameter randomization: if an attribution map does not change substantially when model weights are progressively randomized, the method is not faithfully explaining the learned function. Other checks include data randomization (training on shuffled labels) and input invariance tests to ensure attributions respond to meaningful biological signal.

The quantification of statistical confidence or credible intervals associated with the importance scores assigned to genomic features. Standard attribution methods produce point estimates without confidence bounds, making it difficult to distinguish true regulatory signals from noise. Approaches to estimate uncertainty include:

• Bayesian neural networks that place priors over weights
• Monte Carlo dropout at inference time
• Ensemble-based methods that aggregate attributions across independently trained models Reliable uncertainty quantification is critical for translating attribution maps into clinical variant interpretation.

A high-throughput experimental method that assays the functional impact of thousands of genomic variants, often used as a ground-truth benchmark for validating computational attribution methods. In a DMS, every possible single-nucleotide substitution in a regulatory element or protein-coding region is synthesized and its effect on a molecular phenotype—such as expression level or binding affinity—is measured. The resulting variant effect map provides an empirical gold standard against which the fidelity of in-silico attribution scores can be rigorously compared.