SHAP Values

SHAP derives from cooperative game theory, specifically the Shapley value, which fairly distributes a payout among players based on their marginal contributions. In machine learning, features are the 'players' and the prediction is the 'payout.' The method computes the average marginal contribution of a feature across all possible subsets of features, ensuring a mathematically fair attribution.

• Originated from Lloyd Shapley's 1953 work in game theory
• Guarantees efficiency (attributions sum to the prediction difference from baseline)
• Satisfies symmetry, dummy, and additivity axioms

Kernel SHAP is a model-agnostic implementation that approximates Shapley values using a specially weighted linear regression. It treats the explanation as a local surrogate model, where the weights are derived from the Shapley kernel. This approach works with any black-box model, making it essential for diagnostic AI systems where proprietary or complex architectures are common.

• Uses LIME-inspired local approximation
• Computationally expensive for high-dimensional data
• Suitable for tabular clinical data and small feature sets

TreeSHAP is a fast, exact algorithm for computing SHAP values on tree-based models like XGBoost, LightGBM, and random forests. Unlike Kernel SHAP, it exploits the internal structure of decision trees to compute attributions in polynomial time rather than exponential time. This is critical for diagnostic models trained on electronic health records where gradient-boosted trees are prevalent.

• Reduces complexity from O(2^M) to O(TLD^2) where T=trees, L=leaves, D=depth
• Provides both local (per-prediction) and global explanations
• Supports feature interactions through SHAP interaction values

Deep SHAP combines SHAP values with DeepLIFT to approximate attributions for deep learning models. It uses a linear composition rule to propagate SHAP values through the network layers, enabling explanations for complex architectures like CNNs used in medical imaging diagnostics. This method bridges the gap between game-theoretic rigor and the non-linear nature of neural networks.

• Leverages background samples as reference distributions
• Faster than Kernel SHAP for high-dimensional inputs like images
• Used in radiology AI to highlight suspicious regions in X-rays and MRIs

SHAP values provide post-hoc explainability that aligns with FDA expectations for Good Machine Learning Practice (GMLP). By quantifying exactly how much each biomarker or clinical feature contributed to a specific patient's diagnosis, SHAP enables reviewers to audit model reasoning. This is distinct from intrinsic interpretability—it allows complex, high-performing models to be deployed with transparency.

• Supports predetermined change control plans (PCCPs) by documenting feature behavior
• Enables counterfactual reasoning: 'What if this biomarker value were different?'
• Facilitates clinician trust through patient-specific explanation visualizations

SHAP operates at two levels of granularity. Local explanations show per-prediction feature contributions using waterfall or force plots, answering 'Why did this patient receive this diagnosis?' Global explanations aggregate SHAP values across all predictions using summary plots and dependence plots, revealing overall feature importance and interaction patterns.

• SHAP summary plots: beeswarm visualizations showing feature impact magnitude and direction
• SHAP dependence plots: reveal how a feature's effect varies with its value and interacts with other features
• Mean |SHAP|: ranks features by average absolute impact across the dataset

Quantitative measures that assess how accurately an explanation method reflects the true reasoning process of the underlying model, not just plausible-sounding rationalizations.

• Comprehensiveness: Measures prediction change when top-k attributed features are removed; a faithful explanation should cause a large drop.
• Sufficiency: Measures if top-k features alone can sustain the original prediction.
• SHAP Evaluation: Kernel SHAP is often used as a ground-truth benchmark to evaluate faster approximation methods due to its theoretical guarantees.