Marginal Contribution

The marginal contribution of a feature is not an absolute property; it is strictly defined relative to a specific coalition (subset) of other features.

• Formula: MC(i, S) = f(S ∪ {i}) - f(S)
• Interpretation: The change in the model's prediction when feature i joins the subset S.
• Example: The value of adding 'Age' to a model that already knows 'Income' may be small, but its value when added to an empty set may be large.

A single marginal contribution is inherently unfair because it depends on the arbitrary order in which features are added. The Shapley value solves this by averaging the marginal contribution over all possible feature orderings.

• Mechanism: For n features, there are n! possible sequences.
• Fairness: This exhaustive averaging ensures each feature's final attribution is independent of any arbitrary insertion order.
• Weighting: Coalitions of the same size receive identical weight in the Shapley kernel.

Marginal contributions capture interaction effects between features. If the value of adding feature i depends heavily on whether feature j is already present, they are interacting.

• Synergy: MC(i, S) is larger when j is in S.
• Redundancy: MC(i, S) is smaller or zero when j is in S.
• SHAP Interaction Values: Extend this concept to explicitly decompose predictions into main effects and pairwise interaction terms.

Calculating the exact marginal contribution for every possible coalition is computationally prohibitive for high-dimensional data.

• Complexity: The number of subsets grows exponentially (2^n).
• Solution: Algorithms like KernelSHAP and TreeSHAP use sampling or model-specific optimizations to approximate the weighted average without exhaustive enumeration.
• Variance Reduction: Techniques like paired sampling are used to stabilize these estimates with fewer model evaluations.

A critical property enforced on marginal contributions is missingness: a feature that is not present in the original input must have a marginal contribution of zero.

• Implementation: This is handled by imputing 'missing' features with values from a background dataset.
• Observational vs. Interventional: The method of imputation (conditional expectation vs. marginal sampling) determines whether the resulting SHAP values respect the data manifold or a causal intervention.

The sum of all marginal contributions, when properly weighted, must exactly equal the difference between the model's specific prediction and the average baseline prediction.

• Additivity: f(x) = Baseline + Σ SHAP_i(x)
• Guarantee: This efficiency property ensures the explanation is a perfect decomposition of the prediction, leaving no residual unexplained variance.
• Baseline: The expected model output E[f(X)], representing the prediction when no feature information is known.

The solution concept from cooperative game theory that directly formalizes marginal contribution. A player's Shapley value is the weighted average of their marginal contributions across all possible coalitions they could join. It uniquely satisfies four axioms: Efficiency (total gain is fully distributed), Symmetry (identical contributions earn equal payoffs), Dummy (zero contribution yields zero payoff), and Additivity (values combine across games). In ML, each feature is a 'player' and the prediction is the 'payout'.

A fundamental Shapley axiom stating that the sum of all feature attributions must exactly equal the difference between the model's prediction f(x) and the baseline expected value E[f(X)]. This property guarantees that the total marginal contribution of all features accounts for the entire prediction shift. In SHAP, this means the explanation is additively complete—no predictive force is lost or double-counted. It transforms marginal contributions into a zero-sum decomposition of a single prediction.

A Shapley axiom ensuring that if a model changes so that a feature's marginal contribution increases (or stays the same) in every coalition, its assigned value must not decrease. This formalizes fairness: a feature that becomes more important to the model should never receive a lower attribution. In practice, this property guarantees that SHAP explanations are monotonically aligned with the model's true reliance on each feature, preventing counterintuitive importance reversals.

An extension of standard SHAP that decomposes predictions into pairwise feature interaction effects in addition to main effects. The total marginal contribution of a feature is split into its direct contribution and its synergistic contributions with every other feature. Computed via the Shapley-Taylor index, these values capture how features amplify or suppress each other's impact. Visualized in dependence plots, they reveal non-linear interactions that simple additive explanations miss.

The computational process of approximating exact Shapley values when exhaustive enumeration over the power set of 2^N coalitions is infeasible. Techniques include Monte Carlo sampling of random coalitions, antithetic sampling to reduce variance, and paired sampling where a coalition with and without the target feature is evaluated together. The estimation error decreases with more samples, trading computational budget for precision. KernelSHAP uses a weighted linear regression approach to efficiently approximate these values.