SHAP Value Logging

SHAP values are derived from Shapley values in cooperative game theory. Each feature is treated as a 'player' in a coalition, and its marginal contribution is calculated by evaluating the model's output with and without that feature, averaged over all possible feature orderings. This guarantees a fair and consistent attribution of the prediction among all inputs.

SHAP unifies local and global explanations. Local explanations show exactly why a specific decision was made for a single instance, which is critical for audit logs. By aggregating these local SHAP values across a dataset, you can derive global feature importance, revealing the model's overall behavior. This dual utility makes it the standard for both debugging and compliance.

SHAP is an additive feature attribution method, meaning the sum of all feature SHAP values plus a base value (the average model output) equals the final prediction. This linear decomposition is highly intuitive for auditors: Prediction = Base Value + SHAP(Feature A) + SHAP(Feature B) + .... It provides a complete, non-overlapping breakdown of the decision logic.

Logging SHAP values alongside the prediction creates a decision provenance record. The log entry typically includes:

• The model inference fingerprint (model version, input hash)
• The raw feature vector
• The SHAP value vector
• The final prediction score This immutable dataset allows auditors to deterministically replay and verify the fairness of every automated decision.

Different model types require different SHAP implementations for efficient logging:

• KernelSHAP: A model-agnostic method that works with any black-box model but is computationally intensive, making it suitable for low-throughput, high-stakes decisions.
• TreeSHAP: A highly optimized algorithm for tree-based models (XGBoost, LightGBM, Random Forest) that computes exact SHAP values in polynomial time, enabling real-time logging in high-throughput systems.

SHAP value logging directly supports key regulatory requirements:

• GDPR Article 22: Provides 'meaningful information about the logic involved' in automated decisions.
• EU AI Act: Serves as a technical measure for transparency and explainability in high-risk AI systems.
• ECOA/FCRA: Enables the generation of adverse action reasons by identifying the top negative contributing features for a denied application.

A global interpretability method that measures the decrease in model performance when a feature's values are randomly shuffled. Unlike SHAP, this provides aggregate importance rather than per-prediction explanations.

• Breaks the correlation between the feature and the target
• Computed on a held-out validation set, not training data
• Limitation: can be misleading when features are highly correlated
• Often logged alongside SHAP distributions for model-level transparency reports

A visualization technique showing the marginal effect of one or two features on the predicted outcome, averaged over the data distribution. PDPs complement SHAP by revealing the functional form of feature relationships.

• Reveals whether relationships are linear, monotonic, or non-linear
• Assumes feature independence, which can produce unrealistic data points
• Individual Conditional Expectation (ICE) plots address PDP limitations
• Useful for validating that SHAP patterns align with domain knowledge