Individual Conditional Expectation

Unlike a Partial Dependence Plot (PDP) which shows a single average line, an ICE plot draws one line per instance in the dataset. Each line represents the change in that specific instance's prediction as the feature of interest is varied. This reveals heterogeneous treatment effects where the functional form differs across subgroups, preventing the ecological fallacy of assuming the average effect applies to all individuals.

A variant called Centered ICE subtracts each curve's value at a chosen anchor point (often the minimum feature value). This forces all curves to originate at zero, making it easier to compare the relative change in prediction across instances. c-ICE is particularly useful when the absolute prediction levels vary widely across instances, obscuring the shape of the partial relationship.

Derivative ICE plots the partial derivative of the prediction with respect to the feature for each instance. If all d-ICE curves are flat and identical, there is no interaction between the feature of interest and other features. Substantial variation in the slopes of d-ICE curves visually confirms the presence of feature interactions that a standard PDP would completely mask.

To generate an ICE plot for feature (X_S):

• For each instance (i), hold all other features (x_C^{(i)}) constant at their observed values.
• Create a grid of values for (X_S).
• Predict (\hat{f}(x_S, x_C^{(i)})) for each grid point.
• Plot the resulting curve. This is repeated for every instance, creating a grid of counterfactual predictions that shows how each individual's outcome would change under different values of (X_S).

When ICE plots reveal distinct clusters of curves with different slopes or shapes, this indicates the presence of latent subgroups in the data. For example, in a medical risk model, increasing age might increase risk for most patients but decrease it for a specific subgroup with a particular genetic marker. Visual inspection of ICE curves can guide subsequent subgroup analysis and feature engineering.

ICE plots become visually overwhelming with more than a few hundred instances, requiring subsampling or transparency techniques. They also suffer from the same extrapolation risk as PDPs when the feature of interest is correlated with other features—the held-constant values (x_C^{(i)}) may represent unrealistic combinations. Additionally, ICE plots only visualize one or two features at a time, limiting their ability to capture higher-order interactions.