Partial Dependence Plot

A PDP isolates the relationship between a target feature and the predicted outcome. It does this by marginalizing over the complementary set of features. For a single feature x_s, the partial dependence function is computed as:

f(x_s) = E_{x_c}[f(x_s, x_c)]

• This expectation is estimated by averaging predictions over all instances in the dataset, holding x_s fixed.
• The result is a curve showing how the model's output changes on average as x_s varies.

The practical computation of a PDP relies on a brute-force averaging method rather than analytical integration:

• For each unique value of the feature of interest, the feature is set to that value for every instance in the dataset.
• The model generates predictions for this synthetically modified dataset.
• The average prediction is computed and plotted.
• This process is repeated for a grid of values across the feature's range, producing the characteristic PDP curve.

While one-way PDPs show the effect of a single feature, two-way PDPs visualize the joint effect of two features simultaneously:

• The output is a contour or surface plot where the axes represent the two features of interest.
• This can reveal interaction effects that are invisible in one-way plots.
• If the two-way plot shows non-parallel contours, it indicates the features do not act independently on the prediction.
• A common diagnostic is to compare the two-way PDP against the sum of the individual one-way PDPs to quantify interaction strength.

The primary limitation of PDPs is the assumption of feature independence. The averaging process creates unrealistic data points if features are correlated:

• For example, a PDP for 'height' might average predictions for a 2-meter-tall person with a weight of 50kg, a combination that rarely exists in reality.
• This gives equal weight to regions of the feature space with very low probability density.
• Accumulated Local Effects (ALE) plots are a direct alternative designed to be unbiased under feature correlation by using conditional distributions instead of marginal ones.

A PDP shows the average marginal effect, which can conceal a heterogeneous relationship where the effect of a feature is positive for some instances and negative for others:

• If a feature has a positive effect on half the population and an equally negative effect on the other half, the PDP will show a flat horizontal line.
• This is a critical failure mode, as the plot suggests the feature has no influence when it is, in fact, highly predictive.
• Individual Conditional Expectation (ICE) plots solve this by plotting the curve for every single instance, revealing the full spectrum of effects hidden by the PDP average.

PDPs are a post-hoc, global interpretability method that can be applied to any predictive model, treating it as a black box:

• The technique requires only the ability to generate predictions from input data; it does not inspect model internals.
• This makes it applicable to random forests, gradient boosted trees, support vector machines, and deep neural networks alike.
• In RF machine learning, a PDP can reveal how a neural network's classification confidence changes as a signal's signal-to-noise ratio (SNR) varies, averaged over all modulation schemes and channel conditions in the test set.

A disaggregated counterpart to the partial dependence plot. Instead of averaging the effect of a feature over all instances, ICE plots display one line per instance, showing how each individual prediction changes as the selected feature varies. This reveals heterogeneous effects and interactions that the PDP's averaging can mask. When ICE curves have different slopes or cross, it indicates the presence of a strong interaction between the feature of interest and another variable.

An alternative to PDPs that is unbiased under feature correlation. While PDPs average predictions over the marginal distribution—which can extrapolate into unrealistic regions when features are correlated—ALE plots compute the local effect of a feature by accumulating differences in prediction over small conditional intervals. This makes ALE plots faster to compute and more reliable than PDPs when input features are not independent, a common scenario in real-world datasets.

A global model inspection technique that measures the increase in prediction error after randomly shuffling a single feature's values. The shuffling breaks the association between the feature and the target, and the resulting drop in model performance quantifies the feature's importance. Unlike PDPs, which show the functional form of the relationship, permutation importance provides a single scalar value per feature. It is model-agnostic but can be misleading when features are correlated, as importance may be split among collinear variables.

A quantitative metric developed by Friedman and Popescu to measure the strength of interaction between two features. The H-statistic compares the variance of the partial dependence function of the joint feature distribution to the sum of the individual PDPs. A value of 0 indicates no interaction, while a value near 1 suggests the prediction is driven entirely by the interaction. This metric guides analysts on when a 2D partial dependence plot is necessary to fully understand the model's behavior.