SHAP Interaction Values

The interaction value between feature i and feature j is symmetric, meaning Φ_{i,j} = Φ_{j,i}. This property ensures that the captured interaction effect is a shared, undirected contribution. The total interaction effect is split equally between the two features in the decomposition, preventing double-counting and providing a consistent attribution of the joint effect.

The sum of all main effects and all pairwise interaction effects exactly equals the difference between the model's prediction f(x) and the baseline value. This extends the standard SHAP efficiency property:

• Main effects capture independent feature contributions
• Interaction effects capture the remaining joint contributions
• The decomposition is exhaustive with no unexplained residual

SHAP dependence plots reveal interactions through vertical dispersion. When a scatter plot of feature value vs. SHAP value shows spread rather than a clean curve, an interaction is present. Coloring points by a second feature's value exposes the interacting variable:

• Clustered colors indicate strong interaction
• Vertical spread at fixed x quantifies interaction magnitude
• Enables discovery of non-linear feature synergies

Interaction values are computed by considering all coalitions of features that exclude both feature i and feature j. The interaction effect is the marginal contribution of adding the pair together, minus their individual marginal contributions. This requires evaluating the model on 2^(M-2) subsets for M features, making exact computation exponentially expensive and necessitating sampling approximations.

If two features contribute to the model's output completely independently—meaning their joint effect equals the sum of their individual effects—their SHAP interaction value is zero. This property makes interaction values a precise detector of non-additive model behavior, distinguishing between models that treat features independently and those that learn complex feature conjunctions.

For tree-based models, TreeSHAP computes exact interaction values in O(TLD^2) time, where T is the number of trees, L is the maximum number of leaves, and D is the maximum tree depth. The algorithm tracks how pairs of features are split across tree paths simultaneously, avoiding the exponential complexity of model-agnostic methods and providing exact, deterministic interaction matrices.

The mathematical generalization of SHAP interaction values to arbitrary orders of feature interactions. While standard SHAP values capture main effects and pairwise interactions, the Shapley-Taylor index quantifies the importance of interactions among any specified number of features (e.g., three-way or four-way interactions).

• Order 1: Equivalent to standard Shapley values (main effects).
• Order 2: Equivalent to SHAP interaction values (pairwise effects).
• Higher Orders: Captures complex synergistic effects in deep models.

This framework allocates credit fairly among all interaction sets, ensuring the efficiency property holds at each interaction order.

The primary visualization for detecting and analyzing SHAP interaction values. This scatter plot displays a feature's value on the x-axis against its SHAP value on the y-axis, with an optional color axis for a second interacting feature.

• Vertical dispersion at a given x-value indicates interaction effects—if no interactions existed, all points would lie on a single curve.
• Color gradients reveal the direction of the interaction: a systematic shift from blue to red shows how the interacting feature modulates the main effect.
• Pattern recognition: Clustered color bands suggest threshold interactions where the effect of one feature flips based on another's value.

The mathematical foundation underlying SHAP interaction values. In cooperative game theory, the Shapley value answers how to fairly distribute a coalition's total payout among players. SHAP extends this by treating features as players and the model prediction as the payout.

For interaction values, the game is reformulated to consider pairs of players joining coalitions together, measuring their joint marginal contribution beyond their individual effects.

• Coalition: A subset of features present in the model.
• Characteristic function: The model's prediction given a specific feature subset.
• Marginal contribution: The change in prediction when a feature pair joins a coalition.

A highly optimized algorithm that computes exact SHAP interaction values for tree-based models in O(TLD^2) time, where T is the number of trees, L is the maximum number of leaves, and D is the maximum tree depth.

Unlike model-agnostic methods that require sampling, TreeSHAP exploits the internal structure of decision trees:

• It tracks how pairs of features split the data down each tree path.
• It computes the conditional expectation of the model output given any feature subset analytically.
• It produces exact values without sampling variance, making it the gold standard for XGBoost, LightGBM, and random forest models.

A critical distinction in SHAP interaction values is decomposing a feature's total effect into its main effect and its pure interaction effects.

• Main effect: The isolated contribution of a feature, averaged over all possible values of other features.
• Pure interaction effect: The additional contribution arising specifically from the synergy between two features, beyond the sum of their individual main effects.
• Total effect: The sum of the main effect and all interaction effects involving that feature.

The SHAP interaction value matrix is symmetric, with diagonal entries representing main effects and off-diagonal entries representing pure pairwise interactions.

The output structure of SHAP interaction values is an M×M matrix for each prediction, where M is the number of features. This matrix decomposes the prediction into additive components:

• Diagonal elements (Φᵢ,ᵢ): The main effect of feature i, representing its contribution in isolation.
• Off-diagonal elements (Φᵢ,ⱼ): The pure interaction effect between features i and j, split equally between them.
• Sum property: The total prediction equals the baseline plus the sum of all matrix elements.

This matrix can be aggregated across instances to produce a global interaction summary, revealing which feature pairs consistently interact across the dataset.