SHAP (SHapley Additive exPlanations)

The explanation must sum to the model's output. When you add up the SHAP value for every feature, the total equals the difference between the model's prediction for that instance and the average prediction. This property, also known as efficiency, ensures the attribution is a true decomposition of the prediction.

• Mathematical guarantee: f(x) = φ₀ + Σ φᵢ
• φ₀ is the base value (expected model output)
• φᵢ is the SHAP value for feature i
• No missing attribution—every bit of the prediction is accounted for

A feature that is truly absent from the model must receive an attribution of zero. This axiom ensures that missing features—those not used in the prediction—do not arbitrarily receive importance scores. In practice, this is handled by marginalizing over the feature's distribution when it is 'removed' from a coalition.

• Prevents phantom importance for unused features
• Critical for sparse explanations in high-dimensional data
• Ensures structural zeros remain zero in the explanation

If a model changes so that a feature's marginal contribution increases or stays the same regardless of other features, that feature's SHAP value must not decrease. This monotonicity property ensures that if a feature becomes more important to the model, the explanation reflects that change.

• Prevents contradictory explanations across model versions
• Guarantees logical coherence when comparing models
• No other additive explanation method satisfies this axiom

SHAP values are linearly additive across models. For ensemble methods like random forests or gradient-boosted trees, the SHAP value for the ensemble is the weighted average of the SHAP values from each individual tree. This property enables efficient computation via TreeSHAP.

• Enables exact computation without sampling for tree models
• Supports model debugging by isolating sub-model contributions
• Extends naturally to stacked and blended architectures

Two features that contribute identically to every possible coalition receive identical SHAP values. This fairness axiom prevents the explanation method from arbitrarily favoring one feature over another when their predictive influence is indistinguishable.

• Prevents label bias in feature attribution
• Essential for regulatory fairness audits
• Ensures the explanation is invariant to feature permutation

SHAP is the only explanation method that simultaneously satisfies local accuracy, missingness, and consistency. This uniqueness result, proven by Lundberg and Lee (2017), establishes SHAP as the theoretically definitive additive feature attribution method.

• Unifies LIME, DeepLIFT, and Shapley values
• Provides a common mathematical language for model interpretability
• The proof relies on Shapley's original cooperative game theory axioms

Financial institutions use SHAP to comply with fair lending regulations by generating adverse action reasons. For a rejected loan application, SHAP decomposes the prediction to show exactly which features (e.g., debt-to-income ratio, number of recent inquiries) pushed the score below the threshold. In fraud detection, analysts use SHAP dependence plots to understand the non-linear relationship between transaction amount and fraud probability, distinguishing legitimate high-value transactions from sophisticated money laundering patterns without exposing proprietary model logic.

In medical diagnostics, SHAP validates that models rely on clinically relevant pathology rather than spurious correlations. For a model predicting sepsis onset, SHAP force plots reveal to intensivists that a sudden drop in platelet count combined with rising lactic acid drove the alert, aligning with established medical scoring systems. In genomics, SHAP quantifies the contribution of individual single nucleotide polymorphisms (SNPs) to disease risk, helping researchers prioritize candidate biomarkers for wet-lab validation.

Manufacturing engineers deploy SHAP to move from reactive to prescriptive maintenance. When a model predicts a CNC spindle failure, SHAP breaks down the anomaly score to highlight that vibration amplitude at 3kHz and coolant temperature drift were the primary drivers. This allows technicians to replace a specific bearing rather than the entire assembly. SHAP summary plots aggregate explanations across a fleet of turbines, identifying systemic failure modes and optimizing spare parts inventory based on globally important sensor readings.

Telecom and SaaS companies use SHAP to move beyond churn probability to churn causality. For a high-value enterprise account flagged for retention risk, SHAP waterfall charts show that a recent 30% increase in support ticket volume and zero product feature adoption in 90 days are the dominant churn drivers. This enables a targeted intervention—such as a dedicated onboarding session—rather than a generic discount. SHAP interaction values further reveal that the negative impact of support tickets is amplified for customers on legacy pricing plans.

Energy traders rely on SHAP to interpret probabilistic load forecasts during extreme weather events. When a model predicts a demand spike, SHAP quantifies the contribution of forecasted ambient temperature, wind speed, and day-ahead market prices. This decomposition allows traders to hedge against specific risk factors. Grid operators use SHAP to audit reinforcement learning agents that control battery storage, verifying that charging decisions are based on sound arbitrage logic rather than exploiting unintended patterns in historical price data.

Litigation teams use SHAP to validate predictive coding models that classify documents as responsive or privileged. For a document flagged as potentially privileged, SHAP text explainers highlight the specific phrases—such as 'legal advice' combined with a partner's name—that triggered the classification. This provides a defensible audit trail when challenging opposing counsel's claims of inadequate review. SHAP's consistency property ensures that if a model is improved to rely more on a specific keyword, that keyword's attributed importance never decreases, satisfying judicial scrutiny of the technology-assisted review process.