Partial Dependence Plots (PDP)

ICE plots are the disaggregated counterpart to PDPs. While a PDP shows the average marginal effect of a feature, an ICE plot visualizes the functional relationship for each individual instance in the dataset.

• Heterogeneity Detection: ICE plots reveal when the average effect masks divergent behaviors across subgroups.
• Visual Structure: A PDP is the average of all ICE curves. If all ICE curves share a similar shape, the PDP is a trustworthy summary.
• Centered ICE (c-ICE): A variant that normalizes all curves to start at zero, making it easier to compare the relative change in prediction across instances.

ALE plots are an unbiased alternative to PDPs specifically designed to handle correlated features. PDPs can produce unreliable extrapolations when features are dependent.

• Conditional vs. Marginal: ALE calculates differences in predictions over the conditional distribution of a feature, not the marginal distribution.
• Bias Correction: By isolating the local effect within small intervals and accumulating them, ALE avoids averaging over unrealistic synthetic data points.
• Computational Speed: ALE is generally faster to compute than PDPs because it bins the feature space rather than iterating over every unique value.

Permutation feature importance measures the decrease in model performance when a single feature's values are randomly shuffled, breaking its relationship with the target.

• Global Metric: Unlike PDPs which show the functional form of a relationship, permutation importance provides a single scalar value representing a feature's overall utility.
• Correlation Sensitivity: If two features are highly correlated, shuffling one may not significantly degrade performance because the model can rely on the other, leading to an underestimation of importance.
• Model-Agnostic: Works with any fitted model and any performance metric (e.g., log-loss, AUC, RMSE).

SHAP is a game-theoretic framework that assigns each feature an importance value for a specific prediction, unifying several additive feature attribution methods.

• Local vs. Global: SHAP provides local explanations for individual predictions, while PDPs provide global, average effects. SHAP dependence plots can visualize the effect of a feature similar to a PDP.
• Additive Property: The sum of all SHAP values for a prediction equals the difference between the model's output and the average prediction.
• Interaction Effects: SHAP dependence plots can color points by a second feature to reveal interaction effects that a standard PDP would obscure.

The H-statistic, developed by Friedman and Popescu, quantifies the degree to which the effect of one feature on a prediction depends on the value of another feature.

• Complement to PDP: A PDP assumes features are independent and additive. The H-statistic formally tests this assumption by measuring the variance of the interaction component.
• Computation: It compares the variance of the two-dimensional PDP (which includes the interaction) to the sum of the two one-dimensional PDPs.
• Practical Use: A high H-statistic between two features indicates that a 2D PDP or an ICE plot colored by the interacting feature is necessary for a complete understanding.

A surrogate model is an inherently interpretable model (like a linear regression or shallow decision tree) trained to approximate the predictions of a complex black-box model.

• Global Surrogate: Trained on the original dataset with the black-box model's predictions as the target. You can then analyze the surrogate's coefficients or tree structure instead of using a PDP.
• Fidelity Measure: The R-squared between the black-box predictions and the surrogate's predictions measures how faithfully the surrogate approximates the original model.
• Limitation: A surrogate model explains the model's behavior, not the underlying data-generating process, which is the same goal as a PDP.