KernelSHAP

KernelSHAP expresses a prediction as a linear sum of feature contributions plus a base value. This satisfies the completeness axiom, ensuring the sum of all SHAP values equals the difference between the model's output and the average prediction.

• Base value: The expected model output over the background dataset
• SHAP value: The marginal contribution of a feature, averaged over all possible coalition orderings
• Additivity: Enables straightforward decomposition of any prediction into human-understandable components

Computing exact Shapley values requires evaluating all 2^M coalitions, which is computationally intractable. KernelSHAP approximates these values by sampling a subset of coalitions and solving a weighted linear regression.

• Coalition vector: A binary mask indicating which features are present or absent
• Weighting kernel: The Shapley kernel assigns higher weight to coalitions near the extremes (all features present or absent) and lower weight to mid-sized coalitions
• Convergence: Accuracy improves with more samples, trading off between computational budget and explanation fidelity

KernelSHAP treats the model as a black-box function and requires only the ability to query predictions for any coalition of features. This makes it applicable to any model type without access to gradients or internal weights.

• Missing feature imputation: When a feature is absent from a coalition, its value is replaced by samples from a background dataset to marginalize over the feature's distribution
• Framework compatibility: Works identically with scikit-learn, XGBoost, PyTorch, TensorFlow, and proprietary APIs
• Trade-off: Model-agnosticism comes at the cost of higher computational overhead compared to model-specific methods like TreeSHAP

KernelSHAP generates local explanations for individual predictions, which can be aggregated to produce global feature importance insights across an entire dataset.

• Local: Explains why a specific loan application was denied by attributing risk to income, credit history, and debt-to-income ratio
• Global: Averaging absolute SHAP values across all predictions yields a ranked list of the most influential features in the model
• Interaction effects: KernelSHAP can be extended to compute Shapley interaction values, capturing pairwise feature synergies that simple additive explanations miss

The Shapley kernel is the mathematical weighting function that ensures the weighted linear regression recovers Shapley values from sampled coalitions. It is derived directly from Shapley value axioms.

• Formula: π(z) = (M - 1) / (M choose |z|) * |z| * (M - |z|), where M is the total number of features and |z| is the coalition size
• Property: Assigns infinite weight to the empty and full coalitions, enforcing exact agreement at the boundaries
• Practical implementation: Infinite weights are replaced with large constants, and regularization is applied to maintain numerical stability during regression

The choice of background dataset critically impacts KernelSHAP explanations, as it defines the reference distribution used to simulate missing features and compute the base value.

• Single reference: Using the dataset mean provides fast but potentially misleading explanations that ignore feature correlations
• Representative sample: A random subset of training data (e.g., 100-1000 instances) better captures the feature distribution
• K-medoids clustering: Selecting representative centroids reduces background size while preserving distributional coverage, improving computational efficiency without sacrificing accuracy

A global explanation method that measures feature importance by randomly shuffling a single feature's values and recording the resulting drop in model performance. While model-agnostic like KernelSHAP, it operates at the dataset level rather than explaining individual predictions. Key limitations:

• Cannot capture local, instance-specific effects
• May produce misleading results when features are highly correlated
• Measures importance to model performance, not contribution to a specific prediction KernelSHAP provides the instance-level granularity that permutation importance cannot.

Quantitative evaluation criteria that measure how accurately an attribution map reflects true feature importance. Common protocols include:

• Deletion Metric: Remove features from most to least important and measure prediction probability decay
• Insertion Metric: Add features incrementally into a blurred baseline and record probability increase
• ROAR (RemOve And Retrain): Iteratively retrain models after removing top-attributed features to measure degradation These metrics are essential for validating that KernelSHAP's coalition sampling and weighted regression produce attributions that genuinely reflect the model's decision logic.