Variance Reduction

A variance reduction technique that pairs each random sample with its complement to create a more balanced set of coalitions.

• For every sampled coalition S, the antithetic coalition N \ S \ {i} is also evaluated.
• This ensures that each feature's marginal contribution is estimated from both sides of the coalition spectrum.
• Reduces variance by exploiting negative correlation between paired estimates.
• Particularly effective because Shapley value estimation involves symmetric averaging over all subsets.

A strategy that evaluates the model on two inputs that differ only by the inclusion of a single feature, directly computing the marginal contribution.

• Instead of sampling coalitions independently, pairs are constructed to isolate the effect of feature i.
• Each pair yields one direct estimate of f(S ∪ {i}) - f(S).
• Eliminates the noise introduced by comparing unrelated coalitions.
• Forms the computational backbone of efficient SHAP approximation algorithms like KernelSHAP.

A sampling design that ensures proportional representation of coalitions across all possible sizes.

• The power set of features is partitioned into strata by coalition cardinality |S|.
• Samples are drawn from each stratum to guarantee coverage of small, medium, and large coalitions.
• Prevents the estimator from being dominated by mid-sized coalitions, which are most numerous.
• The Shapley kernel in KernelSHAP implicitly applies a form of stratified weighting to prioritize coalitions with high information content.

A technique that draws coalitions from a proposal distribution that over-samples high-information subsets rather than uniformly.

• Coalitions are weighted by the inverse of their sampling probability to maintain an unbiased estimator.
• The optimal proposal distribution assigns higher probability to coalitions where the feature's marginal contribution has high variance.
• Can dramatically reduce the number of model evaluations needed for convergence in high-dimensional feature spaces.
• Requires careful design of the proposal distribution to avoid introducing bias.

Replaces random sampling with low-discrepancy sequences that fill the coalition space more evenly than independent random draws.

• Sequences like Sobol or Halton ensure deterministic, space-filling coverage.
• Achieves a convergence rate of approximately O(1/N) compared to O(1/√N) for standard Monte Carlo.
• Eliminates the clustering and gaps inherent in random sampling.
• Particularly effective when the number of model evaluations is strictly budgeted and reproducibility is required.

A method that leverages a correlated surrogate function with a known expectation to reduce the variance of the Shapley value estimator.

• A simpler model or a linear approximation of the target model serves as the control variate.
• The difference between the target model and the control variate is estimated, then the known expectation of the control is added back.
• If the control variate is highly correlated with the true model output, the residual variance is significantly smaller.
• Exploits the additive structure of Shapley values to decompose the estimation problem.

A variance reduction technique that pairs each random sample with its complement to create negatively correlated estimates.

• For every sampled coalition, the opposite coalition is also evaluated
• The paired estimates are averaged, canceling out first-order noise
• Reduces the variance of the marginal contribution estimator
• Particularly effective when the value function is monotonic

In SHAP estimation, antithetic pairs ensure that if one sample overestimates a feature's contribution, its complement underestimates it, pulling the average closer to the true Shapley value.

A strategy that evaluates coalitions in matched pairs that differ only by the inclusion or exclusion of a single target feature.

• Each pair directly estimates the marginal contribution of one feature
• Eliminates the need to average over unrelated coalition variations
• Reduces the Monte Carlo error by focusing computation on the contrast of interest
• Works synergistically with antithetic sampling for compounded variance reduction

This approach transforms the estimation problem from sampling the full power set to strategically sampling informative pairwise comparisons.

A method that partitions the coalition space into homogeneous strata and samples proportionally from each, ensuring representative coverage.

• Strata are defined by coalition size (number of features included)
• Guarantees that small and large coalitions are both adequately sampled
• Prevents the estimator from being dominated by mid-sized coalitions
• Reduces variance by eliminating sampling imbalance across strata

In SHAP, stratified sampling ensures that the marginal contribution of a feature is evaluated across the full spectrum of coalition sizes, from empty to full, without random clustering.

A technique that draws samples from a biased proposal distribution that over-represents coalitions with higher variance contributions, then reweights to correct the bias.

• Focuses computational budget on high-impact coalitions
• Reweighting uses the likelihood ratio between the target and proposal distributions
• Dramatically reduces the number of samples needed for convergence
• Requires careful design of the proposal distribution to avoid weight degeneracy

Applied to SHAP, importance sampling prioritizes coalitions where a feature's marginal contribution is most uncertain, accelerating convergence to stable Shapley values.

A deterministic alternative to random sampling that uses low-discrepancy sequences (e.g., Sobol, Halton) to achieve more uniform coverage of the coalition space.

• Sequences are designed to minimize gaps and clustering
• Achieves a convergence rate of nearly O(1/N) versus O(1/√N) for standard Monte Carlo
• Eliminates the random noise inherent in pseudo-random sampling
• Particularly effective in low-to-moderate dimensional feature spaces

In SHAP estimation, quasi-Monte Carlo methods provide faster, more reproducible convergence by systematically exploring the space of possible coalitions.

A technique that leverages a correlated auxiliary variable with a known expected value to reduce the variance of the primary estimator.

• The auxiliary variable must have a known analytical expectation
• Its deviation from the known mean is used to correct the primary estimate
• The stronger the correlation, the greater the variance reduction
• Common control variates in SHAP include the baseline value and subset-averaged predictions

By subtracting the weighted error of the control variate, the estimator's variance is reduced without introducing bias.