Explainable AI (XAI)

Interpretability refers to the degree to which a human can consistently predict a model's result, often achieved through inherently transparent models like linear regression. Explainability involves post-hoc methods that approximate the reasoning of complex, opaque models. In supply chain contexts, explainability is critical when using deep learning for lead time prediction, as planners need to audit why a specific Temporal Fusion Transformer flagged a shipment as high-risk.

Feature attribution methods quantify the marginal contribution of each input variable to a specific prediction. For a delayed shipment forecast, this reveals whether the primary driver was port congestion, carrier performance, or weather anomalies. Key techniques include:

• SHAP (SHapley Additive exPlanations): Uses game theory to assign fair credit to each feature
• LIME (Local Interpretable Model-agnostic Explanations): Approximates the model locally with a simpler surrogate
• Integrated Gradients: Computes the path integral of gradients for deep neural networks

Global explanations describe the overall behavior of a model across all predictions, identifying which features are generally most influential for lead time forecasting. Local explanations focus on a single prediction, answering the question: 'Why was this specific purchase order predicted to be 5 days late?' Effective XAI systems provide both perspectives, allowing strategic model validation and tactical exception management.

Counterfactuals identify the minimal changes required to alter a prediction outcome. For a predicted late delivery, a counterfactual might state: 'If the carrier had been switched to Air Freight and the order quantity reduced by 10%, the delivery would be on time.' This technique is particularly valuable for prescriptive analytics, offering planners actionable recourse rather than just diagnostic information.

Model-agnostic explanation techniques work independently of the underlying algorithm, providing flexibility across diverse AI architectures. This is essential in supply chain environments where Gradient Boosting Machines, LSTMs, and Temporal Fusion Transformers may all be deployed simultaneously. Methods like SHAP and LIME can be applied uniformly, ensuring consistent auditability across the entire model ecosystem.

Some models are designed to be transparent by nature, offering direct insight into their decision logic without post-hoc approximation. Examples include:

• Decision Trees: Explicit if-then rules for lead time classification
• Generalized Additive Models (GAMs): Reveal the shape of each feature's effect
• Attention-based models: Highlight which time steps or features the model focused on These are preferred in high-stakes procurement decisions where regulatory compliance demands full transparency.

A game-theoretic approach that decomposes a lead time prediction into the marginal contribution of each input feature. SHAP (SHapley Additive exPlanations) values quantify exactly how much a specific factor—such as a carrier's historical on-time performance or a seasonal demand surge—pushed the forecasted delivery date forward or backward.

• Computes fair feature attribution by averaging over all possible feature combinations
• Provides both global importance (which features matter most overall) and local explanations (why this specific order is predicted late)
• Example: A SHAP waterfall plot reveals that a 3-day delay prediction is driven +1.2 days by port congestion and +1.8 days by the supplier's recent reliability decline

A perturbation-based technique that explains individual predictions by creating a simple, interpretable surrogate model around the specific data point. LIME generates synthetic samples near the prediction of interest and fits a linear model to approximate the complex model's local decision boundary.

• Model-agnostic: Works with any black-box predictor including GBMs and LSTMs
• Highlights which features, if changed, would most alter the predicted lead time
• Example: For a shipment predicted to arrive late, LIME might show that increasing the buffer stock allocation by 10% would shift the prediction to on-time

A global interpretability method that visualizes the marginal effect of one or two features on the predicted lead time outcome, averaged over the distribution of all other features. PDPs answer questions like "How does predicted lead time change as supplier distance increases?"

• Reveals directional relationships and potential non-linear thresholds
• Identifies saturation points where additional feature changes yield diminishing returns
• Example: A PDP may show that lead time remains stable up to 500km of transit distance, then increases sharply beyond that threshold due to cross-border customs complexity

A straightforward technique that measures a feature's importance by randomly shuffling its values and observing the resulting degradation in model accuracy. If scrambling a feature like historical lead time variability causes a large drop in predictive performance, that feature is deemed critical.

• Model-agnostic and computationally efficient
• Directly measures the feature's impact on the chosen error metric such as MAPE
• Example: Permuting the port congestion index causes a 22% increase in forecast error, confirming it as the dominant predictor for maritime shipments

A what-if analysis technique that generates the minimal set of changes required to flip a prediction from an undesirable outcome to a desirable one. Counterfactuals answer: "What would need to change for this predicted late delivery to become on-time?"

• Produces actionable recommendations grounded in real feature values
• Respects feasibility constraints—won't suggest impossible changes like negative transit times
• Example: "If the supplier's reliability score improved from 0.72 to 0.85 and the order was placed 2 days earlier, the predicted delay would be eliminated"

For deep learning models like the Temporal Fusion Transformer (TFT), attention weight visualization reveals which historical time steps and which input features the model focused on when making a specific forecast. This provides native interpretability without post-hoc approximation.

• Shows temporal patterns the model learned, such as weekly seasonality or holiday effects
• Identifies regime shifts where the model's attention pattern changes due to disruptions
• Example: Attention heatmaps reveal the model heavily weights the past 7 days of transit data and ignores older history when predicting for stable lanes, but expands its attention window during port strikes

A game-theoretic approach to model interpretability that assigns each feature an importance value for a particular prediction. SHAP (SHapley Additive exPlanations) computes the marginal contribution of each input variable—such as carrier type or port of origin—to the final predicted delay.

• Based on Shapley values from cooperative game theory
• Satisfies consistency, local accuracy, and missingness properties
• Can explain any model: gradient boosting, random forest, or deep learning
• Outputs both global feature importance and local per-prediction explanations

Example: A SHAP waterfall plot shows that a specific supplier's recent 3-day processing delay contributed +2.1 days to the predicted lead time, while the chosen premium carrier subtracted -0.8 days.

A technique that explains individual predictions by approximating the complex model locally with an interpretable surrogate model—typically a linear regression or decision tree. LIME perturbs the input data around a specific instance and observes how predictions change.

• Model-agnostic: Works on any classifier or regressor
• Generates locally faithful explanations, not global ones
• Identifies which words in a disruption alert or which numerical features drove a specific delay classification
• Particularly useful for text-based features in supplier communication analysis

Limitation: Local explanations may not generalize to the model's overall behavior, requiring careful validation against domain knowledge.

A global interpretability method that visualizes the marginal effect of one or two features on the predicted outcome, averaged over the distribution of all other features. PDPs reveal the directional relationship between an input and the model's output.

• Shows whether increasing order volume monotonically increases predicted lead time
• Can expose non-linear relationships and threshold effects
• Assumes feature independence, which can produce misleading results with correlated logistics variables
• Best used alongside Individual Conditional Expectation (ICE) plots to reveal heterogeneous effects

Supply Chain Use: A PDP might reveal that predicted transit time drops sharply when shipment weight exceeds a full-container-load threshold, triggering direct routing.

A set of techniques that rank input variables by their overall contribution to model performance. In supply chain forecasting, this answers the critical question: What actually drives delivery delays?

• Permutation importance: Measures performance drop when a feature's values are randomly shuffled
• Impurity-based importance: Derived from decision tree splits (fast but biased toward high-cardinality features)
• SHAP-based importance: Mean absolute SHAP value across all predictions

Key Insight: A model may show that supplier historical reliability has 3x more influence on lead time than shipping distance, redirecting procurement strategy toward vendor qualification rather than nearshoring.

Explanations that answer the question: What would need to change for the outcome to be different? A counterfactual identifies the minimal set of input alterations required to flip a prediction from delayed to on-time.

• Provides actionable recourse for planners
• Generates statements like: "If the order were placed 3 days earlier, the predicted delay would be eliminated"
• Must satisfy feasibility constraints—cannot suggest impossible changes like altering a supplier's geographic location
• Often generated through optimization algorithms or generative modeling

Operational Value: Directly informs exception management by prescribing the smallest intervention needed to meet a delivery deadline.

High-precision rule-based explanations that provide sufficient conditions for a prediction. An anchor is an "if-then" rule that guarantees the prediction with high confidence, regardless of changes to other feature values.

• Derived from reinforcement learning and bandit algorithms
• Example anchor: "IF carrier is OceanFreight Ltd AND origin port is Shanghai THEN predicted delay > 5 days (precision: 97%)"
• More intuitive for non-technical stakeholders than SHAP values
• Rules are self-contained and do not require understanding of the underlying model

Advantage: Anchors provide deterministic guarantees that a specific combination of conditions will result in a delay, enabling precise risk flagging in control tower dashboards.