KernelSHAP

KernelSHAP is built on Shapley values from cooperative game theory, which fairly distribute a prediction among input features. Each nucleotide or genomic region is treated as a player in a coalition, and its marginal contribution is averaged over all possible feature subsets. This satisfies key axioms: local accuracy (the sum of attributions equals the prediction difference from a baseline), missingness (absent features get zero attribution), and consistency (if a feature's contribution increases, its attribution never decreases). For genomic models, this provides a mathematically rigorous way to quantify how each base pair influences variant effect predictions or gene expression outputs.

Exact Shapley value computation requires evaluating all 2^M feature coalitions, which is intractable for genomic sequences with thousands of nucleotides. KernelSHAP solves this via weighted linear regression over a sampled subset of coalitions. The key insight is the Shapley kernel weighting function:

• Coalitions with sizes near 0 or M (all features absent or present) receive the highest weight
• This forces the surrogate model to be most accurate at the extremes, where individual feature effects are isolated
• The regression is solved with L2 regularization to prevent overfitting to the sampled coalitions
• For genomic applications, this reduces computation from exponential to linear in the number of samples

KernelSHAP operates as a black-box explainer that requires only the model's prediction function, not its internal architecture. This is critical for genomic workflows where:

• Models range from convolutional neural networks to transformer-based DNA language models to gradient-boosted trees
• The same interpretability method can be applied uniformly across different model types for consistent regulatory documentation
• Proprietary or third-party models can be explained without access to gradients or architecture details
• The method works by perturbing input sequences (masking nucleotides to a reference value) and observing output changes, making it compatible with any prediction pipeline

The fidelity of KernelSHAP depends on how feature coalitions are sampled. Key strategies include:

• Paired sampling: For each coalition, its complement is also evaluated to reduce variance
• Adaptive sampling: More samples are allocated to features with higher variance in their Shapley value estimates
• Grouped features: Adjacent nucleotides can be grouped into k-mer blocks to reduce the feature space dimensionality while preserving biological relevance
• Background dataset integration: Instead of a single reference baseline, a distribution of background sequences is used to compute expected values, improving robustness for genomic contexts where a single reference genome is insufficient

Typical genomic applications use 200-1000 coalition samples per explanation, balancing accuracy with computational cost.

Applying KernelSHAP to genomic sequences requires domain-specific adaptations:

• Reference value selection: Masking nucleotides to the reference genome base, a mean embedding, or a dinucleotide-shuffled background all produce different attribution patterns
• Feature independence assumption: KernelSHAP assumes feature independence, which is violated by linkage disequilibrium in genomic data; conditional expectations or causal Shapley values can partially address this
• Sequence length handling: For long sequences (10kb+), hierarchical SHAP decomposes attributions at multiple resolutions—nucleotide-level, motif-level, and region-level
• Strand symmetry: Attribution values should be validated on both forward and reverse-complement sequences to ensure strand-agnostic interpretability

KernelSHAP occupies a specific niche in the genomic interpretability landscape:

• vs. Integrated Gradients: Both satisfy the completeness axiom, but KernelSHAP is model-agnostic while Integrated Gradients requires gradient access
• vs. DeepSHAP: DeepSHAP is faster for deep learning models by leveraging backpropagation rules, but KernelSHAP works on any model type
• vs. In-silico Mutagenesis (ISM): ISM exhaustively tests single-nucleotide variants, while KernelSHAP captures higher-order interactions through coalition sampling
• vs. Attention Weights: Attention provides raw importance scores without theoretical guarantees; KernelSHAP provides axiomatically justified attributions
• Computational trade-off: KernelSHAP is slower than gradient-based methods but faster than exhaustive ISM for long sequences

Quantitative measures that evaluate how accurately an attribution map reflects a model's true decision logic. For KernelSHAP, common faithfulness tests include:

• Perturbation-based metrics: Sequentially remove the most salient nucleotides and measure prediction drop.
• ROAR (RemOve And Retrain): Retrain the model after removing top features to verify they were genuinely important.
• Infidelity measure: Quantify the expected error between input perturbations and attribution perturbations. These metrics are critical for regulatory validation of genomic model explanations in clinical settings.