Shapley Additive Explanations (SHAP)

The sum of all SHAP values for a prediction equals the difference between the model's output for that instance and the average model output. This guarantees additive feature attribution.

• Mechanism: If the base value (expected prediction) is $E[f(x)]$ and the actual prediction is $f(x)$, the sum of all $\phi_i$ must equal $f(x) - E[f(x)]$.
• Clinical Relevance: For a sepsis predictor, this ensures the total risk score is completely decomposed into contributions from heart rate, temperature, and lab values with no unexplained variance.

A feature that is not present in the model input must be assigned a SHAP value of exactly zero. This axiom ensures that absent features do not distort the explanation.

• Implementation: Missing features are handled by integrating them out using a background dataset, effectively replacing them with expected values.
• Practical Impact: When a lab test is not ordered for a patient, its SHAP value is zero, preventing the model from fabricating influence from unobserved clinical variables.

If a model changes so that a feature's marginal contribution increases or stays the same regardless of other features, its SHAP value cannot decrease. This rewards truly influential features.

• Formal Definition: For any two models $f$ and $f'$, if $f'_x(S \cup {i}) - f'_x(S) \geq f_x(S \cup {i}) - f_x(S)$ for all subsets $S$, then $\phi_i(f', x) \geq \phi_i(f, x)$.
• Why It Matters: This prevents a scenario where a feature becomes more important to the model but its attribution score paradoxically drops, a flaw present in earlier methods like LIME.

SHAP is the only feature attribution method that simultaneously satisfies Local Accuracy, Missingness, and Consistency. This game-theoretic foundation provides a legally defensible audit trail.

• Cooperative Game Theory Origin: Derived from Lloyd Shapley's 1953 solution for fairly distributing payouts among players in a coalition based on their marginal contributions.
• Clinical Governance: For regulatory submissions under the EU AI Act, SHAP's axiomatic guarantees provide mathematical rigor that heuristic attribution methods cannot match.

Computing exact SHAP values requires evaluating $2^M$ feature coalitions, which is computationally intractable. Kernel SHAP uses a weighted linear regression to approximate these values efficiently.

• Approach: It samples coalitions and solves a least-squares problem with a Shapley kernel weighting function that prioritizes coalitions with few or many features.
• Trade-off: While faster than exact computation, Kernel SHAP remains model-agnostic and can be applied to any black-box model, including ensemble methods and deep neural networks.

For tree-based models like XGBoost, LightGBM, and random forests, Tree SHAP computes exact SHAP values in polynomial time by exploiting the internal structure of decision trees.

• Performance: Reduces complexity from $O(TL2^M)$ to $O(TLD^2)$, where $T$ is trees, $L$ is leaves, and $D$ is tree depth.
• Clinical Deployment: Enables real-time feature attribution for gradient-boosted early warning scores without sampling error, critical for time-sensitive sepsis prediction.