Partial Dependence Plot (PDP)

PDPs isolate the effect of a target feature by averaging out the influence of all other features. The algorithm substitutes the target feature's value for every instance in the dataset, obtains predictions, and computes the average. This process answers: What is the average prediction if every instance had this specific feature value?

• Mathematical Definition: (\hat{f}S(x_S) = E{X_C}[\hat{f}(x_S, X_C)])
• Estimation: Uses Monte Carlo integration by averaging over actual instances in the training set.
• Result: A curve (1D) or surface (2D) showing the average prediction change across the feature's domain.

Unlike local explanations like LIME or SHAP, a PDP provides a global view of a feature's average influence across the entire dataset. It is completely model-agnostic, meaning it can be applied to any predictive model—from gradient-boosted trees to deep neural networks—as long as the model produces a prediction function.

• Interpretation: The plot's shape reveals the nature of the relationship (linear, monotonic, non-linear).
• Comparison: Contrast with Individual Conditional Expectation (ICE) plots, which disaggregate this global average to show individual instance-level effects.

The primary statistical limitation of PDPs is the assumption of feature independence. When features are highly correlated, the marginalization process averages predictions over unrealistic synthetic data points.

• Extrapolation Risk: The model is forced to predict for combinations of features that rarely or never occur in the real joint distribution (e.g., a person with 10 pregnancies who is marked as male).
• Mitigation: For correlated features, Accumulated Local Effects (ALE) plots are preferred, as they compute effects based on the conditional distribution, avoiding this extrapolation bias.

A PDP shows an average effect, which can obscure complex, heterogeneous relationships. If half the instances have a strong positive relationship with a feature and the other half have a strong negative one, the PDP may show a flat, zero-effect line.

• Diagnostic Tool: Always pair a PDP with its corresponding ICE plots to visually check for such interactions.
• Interaction Detection: A flat PDP with highly variant ICE curves is a strong indicator of a feature interaction that is being averaged out.

PDPs are highly effective for visualizing the effect of categorical features. Instead of a continuous line, the plot displays a bar or point for each category level, representing the average prediction when all instances are forced into that category.

• Interpretation: The vertical difference between category bars directly quantifies the expected change in the prediction when switching from one category to another, holding all else constant.
• Example: In a churn model, a PDP can show the average predicted churn probability for 'Contract Type: Month-to-month' versus 'Contract Type: Two-year'.

A PDP can be extended to two features simultaneously, producing a three-dimensional surface or a contour/heatmap. This visualizes the joint marginal effect of a feature pair, revealing their interaction.

• Mechanism: The algorithm varies both features on a grid and averages the predictions over all other features.
• Use Case: A 2D PDP can show if the effect of one feature (e.g., temperature) depends on the level of another (e.g., humidity) in predicting energy consumption, a pattern invisible in separate 1D plots.