Explainable AI (XAI)

Distinguishes between models that are inherently transparent and those requiring post-hoc analysis. Interpretability refers to intrinsic understanding (e.g., a linear regression weight), while explainability involves applying external methods to decode a black-box model.

• White-box models: Decision trees and logistic regression are inherently interpretable.
• Black-box models: Deep neural networks require surrogate models or feature attribution for explainability.
• Trade-off: Higher accuracy in complex systems often necessitates sacrificing direct interpretability for post-hoc explainability.

Methods that highlight which parts of the input data most influenced a model's output. In predictive maintenance, this translates to identifying the specific sensor channels and time steps driving a failure classification.

• Gradient-based methods: Compute the partial derivative of the output with respect to each input feature.
• Attention weights: In Transformer models, attention scores reveal which past sensor readings the model deems critical.
• Application: Visualizing that a specific frequency band in a Fast Fourier Transform (FFT) spectrum is the primary driver of a bearing fault classification.

Generates hypothetical scenarios that answer the question: 'What minimal changes to the input would have changed the prediction?' This is critical for prescriptive maintenance action planning.

• Mechanism: Finds the smallest perturbation to sensor values that flips the model's decision from 'failure' to 'normal'.
• Actionable insight: 'If the rotational speed had been reduced by 50 RPM, the predicted failure probability would drop below the threshold.'
• Constraint handling: Realistic counterfactuals respect physical constraints, avoiding impossible suggestions like negative vibration.

An interpretable model trained to approximate the predictions of a black-box model. This provides a high-level, simplified view of the complex model's overall logic for auditing purposes.

• Process: Train a decision tree on the original inputs and the black-box model's outputs.
• Fidelity metric: R-squared measures how well the surrogate mimics the original model.
• Limitation: Surrogates can misrepresent complex, non-linear decision boundaries, requiring careful validation against the original model.

A game-theoretic method that assigns each input feature an importance value for a particular prediction. Based on Shapley values from cooperative game theory, SHAP fairly distributes the 'payout' (the prediction) among the features.

• How it works: Calculates the marginal contribution of each sensor reading (e.g., vibration amplitude, temperature) to the final failure probability.
• Use case: Explaining why a specific bearing was flagged for imminent failure by showing that the high-frequency vibration spike contributed 70% to the alert, while temperature rise contributed 20%.
• Key benefit: Provides consistent, theoretically grounded explanations that satisfy both engineering curiosity and regulatory audit requirements.

An algorithm that explains individual predictions by approximating the complex model locally with a simpler, interpretable 'surrogate' model (like a linear regression or decision tree).

• Mechanism: Perturbs the input data around a specific prediction, observes how the black-box model responds, and fits a simple model to that local neighborhood.
• Maintenance example: For a single 'high-risk' pump alert, LIME might reveal that the model is focusing on erratic pressure fluctuations rather than the overall runtime hours, guiding the technician to check for cavitation.
• Limitation: Explanations are local and can be unstable; two very similar inputs might yield slightly different surrogate explanations.

A technique specific to Transformer models and LSTMs that highlights which parts of an input sequence the model 'attended to' when making a prediction. It creates a heatmap over the time-series sensor data.

• How it works: Extracts the internal attention weights from the neural network layers. Higher weights indicate stronger influence on the output.
• Practical application: When a Transformer predicts a motor failure 48 hours out, the attention map might show the model is intensely focused on a subtle, 2-second current inrush anomaly that occurred 12 hours prior, which a human analyst missed.
• Key insight: Transforms the model's internal reasoning into a visual narrative of temporal dependencies.

A global interpretability method that shows the marginal effect of one or two features on the predicted outcome, averaged over the distribution of all other features.

• Function: Isolates the relationship between a sensor (e.g., operating temperature) and the failure probability, holding all other variables constant at their average values.
• Diagnostic value: A PDP can reveal that failure risk remains flat until a critical temperature threshold of 85°C, after which it increases exponentially. This confirms or challenges domain engineering assumptions.
• Caveat: Assumes feature independence, which can produce misleading results if sensors are highly correlated (e.g., temperature and pressure in a boiler).

An explanation method that answers 'what if' questions by identifying the minimal changes to the input features required to flip a model's prediction from one outcome to another.

• Mechanism: Searches for the smallest possible perturbation to the sensor data that would change a 'failure predicted' alert to 'normal operation'.
• Actionable insight: The model predicts a gearbox failure. A counterfactual explanation states: 'If the input shaft speed were reduced by 50 RPM and the oil debris count were below 15 ppm, the failure risk would drop below the threshold.'
• Value: Directly translates a complex prediction into a concrete, prescriptive operational adjustment for the plant operator.

An attribution method designed specifically for deep neural networks that satisfies a key axiom called implementation invariance—two functionally identical networks always get the same attributions.

• Process: Computes the average gradient of the model's output with respect to the input features along a straight-line path from a baseline (e.g., zero signal) to the actual input.
• Why it matters: Unlike raw gradients, Integrated Gradients avoid saturation problems where a feature clearly influences the prediction but has a near-zero gradient.
• Maintenance context: Accurately attributes a high Remaining Useful Life (RUL) prediction to the cumulative vibration energy across all historical time steps, not just the final spike, providing a more faithful picture of degradation.

A technique specific to Transformer models and attention-based architectures that visualizes the weighting the model assigns to different time steps in a sensor sequence.

• Reveals which temporal segments the model focuses on when forecasting Remaining Useful Life (RUL)
• Validates that the model attends to physically meaningful precursors, such as a spike in vibration before bearing failure
• Critical for debugging Long Short-Term Memory (LSTM) and Transformer-based degradation models

A global interpretability method that shows the marginal effect of one or two features on the predicted outcome, averaged over the distribution of all other features.

• Illustrates the relationship between a sensor reading and failure probability
• Helps engineers verify monotonic relationships, such as rising temperature consistently increasing failure risk
• Can be combined with Individual Conditional Expectation (ICE) plots to reveal heterogeneous effects across instances

A method that generates hypothetical scenarios by minimally altering input features to flip a model's prediction. For predictive maintenance, it answers: 'What sensor values would have changed the failure classification to normal?'

• Provides actionable guidance for operators: 'Reduce RPM by 15% to exit the critical zone'
• Uses optimization techniques to find the smallest feature perturbation that changes the outcome
• Essential for prescriptive maintenance workflows where explanations must drive immediate action

Models designed with inherent transparency, avoiding the need for post-hoc explanation tools. In industrial contexts, these often outperform black-boxes when data is structured.

• Decision Trees: Explicit rule paths that map sensor thresholds to failure modes
• Generalized Additive Models (GAMs): Capture non-linear sensor relationships while maintaining additive feature independence
• Explainable Boosting Machines (EBMs) : Modern glass-box models that rival gradient boosting in accuracy while remaining fully auditable