Explainability Metric (SHAP)

SHAP belongs to the class of additive feature attribution methods. This means it explains a model's output as a sum of contributions from each input feature, plus a baseline expectation. Formally, for a model f(x), the explanation model g(x') is defined as: g(x') = φ₀ + Σ φᵢ x'ᵢ where φ₀ is the model's expected output on the background data distribution, φᵢ is the SHAP value for feature i, and x' is a simplified binary vector indicating feature presence. This additive structure ensures local accuracy, meaning the explanation exactly matches the model's output for the specific instance being explained.

SHAP values are the unique solution derived from cooperative game theory, specifically Shapley values. In this framework:

• Each feature is a "player" in a game.
• The "payout" is the model's prediction.
• The SHAP value φᵢ is the average marginal contribution of feature i across all possible coalitions (subsets) of other features. It is calculated as: φᵢ(f, x) = Σ_{S ⊆ N \ {i}} [|S|! (|N|-|S|-1)! / |N|!] * (f_x(S ∪ {i}) - f_x(S)) where N is the set of all features and f_x(S) is the model's prediction for a subset S. This foundation provides a principled, axiomatic basis for feature importance that other heuristic methods lack.

SHAP satisfies two critical axioms that guarantee trustworthy explanations:

• Local Accuracy: The sum of all feature attributions (Σ φᵢ) plus the baseline (φ₀) equals the model's actual prediction for that specific instance. This ensures the explanation is faithful to the model's local behavior.
• Consistency (Monotonicity): If a model changes so that a feature's contribution increases or stays the same for all subsets of other features, its SHAP value will not decrease. This prevents explanations from being inconsistent when the underlying model is refined. These properties distinguish SHAP from less rigorous attribution methods that can violate these axioms, leading to misleading interpretations.

While SHAP values are calculated for individual predictions (local explanations), they can be aggregated to provide global model interpretability. Common techniques include:

• SHAP Summary Plot: Displays the distribution of SHAP values for each feature across a dataset, showing impact and direction (positive/negative).
• Feature Importance: The mean absolute SHAP value (mean(|φᵢ|)) for a feature ranks its overall influence on model output.
• Dependence Plots: Scatter plots showing how a feature's value relates to its SHAP value, potentially revealing complex, non-linear relationships. This dual local/global capability makes SHAP a comprehensive diagnostic tool.

The exact computation of Shapley values is computationally intractable for high-dimensional data. SHAP provides efficient, model-agnostic approximations:

• KernelSHAP: A kernel-based method that approximates SHAP values for any model by sampling feature subsets and solving a weighted linear regression. It treats the model as a black box.
• TreeSHAP: A highly efficient, exact algorithm for tree-based models (e.g., XGBoost, Random Forests) that exploits the tree structure to compute SHAP values in polynomial time.
• DeepSHAP: An approximation method for deep learning models that builds on DeepLIFT, using a composition rule to propagate SHAP values through the network layers. These approximations make SHAP practical for real-world, complex models.

SHAP explanations are inherently contrastive. They answer the question: "Why did the model make prediction f(x) instead of the baseline prediction E[f(z)]?"

• The baseline (φ₀) is typically the average model output over a background dataset (e.g., training data).
• Each SHAP value (φᵢ) quantifies how much feature i moved the prediction from this baseline expectation for the specific instance x. This framing is intuitive for users, as it explains deviations from a "typical" or "expected" outcome. The choice of baseline is crucial and should reflect the context of the explanation (e.g., population average vs. a specific cohort).

The process of quantitatively assessing the quality, accuracy, and faithfulness of explanation methods like SHAP. Since explanations themselves are model outputs, they must be validated.

• Faithfulness Metrics: Measure if the explanation reflects the model's true reasoning. For example, sequentially removing top-contributing features (per SHAP) should cause a large drop in prediction accuracy.
• Stability Metrics: Assess if similar inputs produce similar explanations, ensuring robustness.
• Application: Used by data scientists to choose the best explanation method for a given model and by regulatory teams to audit automated decision systems.

Two fundamental scopes for explaining model behavior. SHAP uniquely provides a consistent framework for both.

• Local Explanation: Answers "Why did the model make this specific prediction for this single instance?" SHAP values for one data point are a local explanation, attributing the prediction to each input feature.
• Global Explanation: Answers "What patterns has the model learned overall?" By aggregating SHAP values across many instances (e.g., taking mean absolute values), you can see which features are most important globally.
• Engineering Use: Debugging individual model errors (local) versus understanding model behavior for feature engineering or stakeholder reporting (global).

A classification for explainability techniques based on their reliance on a model's internal structure. SHAP is a model-agnostic framework.

• Model-Agnostic: Treats the model as a "black box." Methods like SHAP, LIME, and permutation importance only require the ability to query the model with inputs and get outputs. This makes them flexible and applicable to any model type (neural networks, gradient boosting, etc.).
• Model-Specific: Leverage the internal weights or structure of a specific model class. Examples include attention weights in transformers or gradient-based methods (Integrated Gradients) for differentiable models.
• Trade-off: Model-agnostic methods are more flexible but can be computationally expensive and approximate. Model-specific methods can be more precise and efficient but are not portable.

An explanation format that answers: "What minimal changes to the input would have changed the model's prediction?" It provides a "what-if" scenario rather than a feature attribution.

• Contrast with SHAP: SHAP explains the contribution of features to the actual prediction. A counterfactual generates a new, slightly altered data point that would receive a different (usually desired) prediction.
• Example: A loan application is denied. SHAP says: "High debt-to-income ratio contributed -30 points." A counterfactual says: "If your debt-to-income ratio were 5% lower, your application would have been approved."
• Utility: Highly actionable for users seeking recourse and for testing model decision boundaries.