Inferensys

Glossary

SHAP (SHapley Additive exPlanations)

A game-theoretic framework that assigns each feature an importance value for a particular prediction, unifying several existing feature attribution methods to explain the output of any machine learning model.
Governance lead reviewing model governance framework on laptop, policy documents visible, executive office setup.
EXPLAINABLE AI

What is SHAP (SHapley Additive exPlanations)?

A game-theoretic framework for interpreting machine learning model outputs by assigning each feature an importance value for a specific prediction.

SHAP (SHapley Additive exPlanations) is a unified framework for interpreting model predictions by computing Shapley values from cooperative game theory. It assigns each input feature an additive importance score representing its marginal contribution to the prediction, ensuring a fair, consistent, and theoretically grounded distribution of credit among all features for any given output.

SHAP unifies several existing feature attribution methods—including LIME, DeepLIFT, and layer-wise relevance propagation—under a single additive feature attribution class. By leveraging kernel SHAP for model-agnostic explanations and TreeSHAP for efficient tree-based model interpretation, it provides both local per-prediction explanations and global feature importance insights critical for auditing fraud detection decisions.

GAME-THEORETIC EXPLAINABILITY

Key Properties of SHAP Values

SHAP values are built on a rigorous mathematical foundation derived from cooperative game theory. These core properties ensure that feature attributions are consistent, fair, and uniquely determined, making them the gold standard for explaining individual predictions in high-stakes fraud detection models.

01

Local Accuracy

The sum of all feature attributions for a specific prediction exactly equals the difference between the model's output for that instance and the average model output over the dataset. This property, also known as efficiency, guarantees that the explanation is a complete decomposition of the prediction.

  • Equation: f(x) = φ₀ + Σᵢ φᵢ, where φ₀ is the base value (expected prediction) and φᵢ are the SHAP values.
  • Fraud Context: For a transaction scored at 0.85 (high risk) against a baseline of 0.10, the SHAP values for 'transaction amount', 'time of day', and 'device fingerprint' will sum to exactly +0.75, leaving no unexplained variance.
02

Missingness

A feature that is not present in the input—either because it is genuinely missing or was intentionally excluded—is assigned an attribution of exactly zero. This prevents the model from inventing importance for absent data.

  • Mechanism: SHAP handles missing features by integrating them out using the background dataset's distribution, ensuring they contribute nothing to the explanation.
  • Practical Impact: If a fraud model does not receive a 'billing address match' signal for a transaction, that feature's SHAP value is strictly zero, preventing spurious attributions that could mislead investigators.
03

Consistency

If a model is changed so that a feature's contribution increases or stays the same regardless of other features, the SHAP value for that feature will never decrease. This monotonicity property ensures that explanations track logically with model updates.

  • Formal Definition: If f'(S ∪ {i}) - f'(S) ≥ f(S ∪ {i}) - f(S) for all subsets S, then φᵢ(f') ≥ φᵢ(f).
  • Fraud Model Evolution: When retraining a model to rely more heavily on 'login geolocation velocity', the SHAP value for that feature will only increase or stay flat—it will never contradict the model's increased reliance by showing a lower importance.
04

Symmetry

Two features that contribute identically to every possible subset of features receive identical SHAP values. This ensures that the attribution method treats functionally equivalent features with absolute fairness.

  • Implication: If 'card present flag' and 'chip read success' always have the same marginal effect on the fraud score when added to any coalition of other features, their SHAP values will be exactly equal.
  • Audit Utility: This property is critical for regulatory compliance, as it guarantees that no feature is arbitrarily favored over another that provides the same predictive signal, supporting defensible adverse action reason codes.
05

Additivity

The SHAP value for a feature in an ensemble model is the weighted average of its SHAP values from each constituent model. This allows explanations to be decomposed across complex model architectures.

  • Ensemble Decomposition: For a fraud detection ensemble combining a gradient boosting model and a neural network, the global SHAP value for 'transaction velocity' is the weighted sum of its SHAP values from each sub-model.
  • Layer-wise Analysis: This property extends to explaining multi-stage pipelines, where the final attribution can be traced back through feature transformations, enabling granular debugging of feature engineering steps.
06

Uniqueness

SHAP values are the only additive feature attribution method that simultaneously satisfies local accuracy, missingness, and consistency. This result, proven by Lundberg and Lee, unifies several prior explanation methods under a single optimal framework.

  • Unified Methods: KernelSHAP, TreeSHAP, and DeepSHAP are all specific implementations that compute Shapley values efficiently for different model types, all converging to the same unique solution.
  • Regulatory Gold Standard: For generating adverse action reason codes under regulations like the Fair Credit Reporting Act (FCRA), the uniqueness property provides a legally defensible, mathematically singular explanation that cannot be contradicted by another valid method.
SHAP EXPLAINABILITY

Frequently Asked Questions

Clear answers to the most common questions about SHAP values, their game-theoretic foundations, and their practical application in making complex fraud detection models fully auditable.

SHAP (SHapley Additive exPlanations) is a game-theoretic framework that assigns each input feature an importance value for a specific prediction. It works by calculating the Shapley value for each feature, which represents the feature's average marginal contribution to the model's output across all possible subsets (coalitions) of features. For a fraud detection model, SHAP decomposes a transaction's risk score into additive contributions: the sum of all feature SHAP values plus a base value (the average model output) equals the final prediction. This additive property ensures that the explanation is locally accurate and consistent. SHAP unifies several existing feature attribution methods—including LIME, DeepLIFT, and layer-wise relevance propagation—under a single theoretical umbrella, providing a mathematically rigorous foundation for why a specific transaction was flagged as fraudulent.

FEATURE ATTRIBUTION COMPARISON

SHAP vs. Other Explainability Methods

A technical comparison of SHAP against LIME, Integrated Gradients, and Counterfactual Explanations across key dimensions relevant to financial fraud model governance and auditability.

FeatureSHAPLIMEIntegrated GradientsCounterfactuals

Theoretical Foundation

Game-theoretic Shapley values with axiomatic guarantees

Local surrogate model approximation

Path-integrated gradients from baseline to input

Minimal perturbation to alter prediction

Model Agnostic

Global Interpretability

Local Interpretability

Handles Feature Interactions

Consistency Guarantee

Computational Cost

High (exponential in features, mitigated by KernelSHAP/TreeSHAP)

Moderate (per-instance sampling)

Moderate (requires gradient computation and integration steps)

Moderate to High (requires optimization per instance)

Regulatory Audit Readiness

High (axiomatic, consistent, additive)

Moderate (approximation-dependent, unstable)

Moderate (baseline choice sensitivity)

High (actionable, minimal-change logic)

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.