SHAP vs LIME for Explaining Black-Box Anomaly Detection Alerts to Engineers

Theoretical Guarantee: SHAP values uniquely satisfy three desirable properties (local accuracy, missingness, consistency) based on Shapley values from cooperative game theory. This means if a feature's contribution increases, its SHAP value will never decrease.

Stability: Provides deterministic, reproducible explanations. Running SHAP on the same input always yields the same result, which is critical for building trust with engineers who need to verify the logic behind an alert.

Global Insight: KernelSHAP and TreeSHAP offer powerful global interpretability methods (summary plots, dependence plots) that reveal the model's overall behavior across the entire fleet, not just a single anomaly.

Performance Cost: Exact SHAP calculation is NP-hard. KernelSHAP, the model-agnostic method, requires an exponential number of model evaluations, making it extremely slow for real-time alert explanation on high-dimensional sensor data (e.g., 100+ vibration frequencies).

Practical Workaround: TreeSHAP is fast for tree-based models (like Isolation Forests or XGBoost), but DeepSHAP for neural networks (like LSTM Autoencoders) is an approximation that can introduce errors.

Engineering Friction: Integrating SHAP into a low-latency alert pipeline often requires pre-computing background datasets and accepting a trade-off between explanation accuracy and speed.

Speed and Simplicity: LIME is model-agnostic and computationally cheap. It creates a local surrogate model (e.g., a sparse linear model) by perturbing the input, making it fast enough to generate explanations in near real-time for an incoming anomaly alert.

Customizable Explanations: Engineers can easily define what constitutes an 'interpretable' feature (e.g., 'vibration_spike_in_band_50-60Hz') rather than raw sensor values, making the explanations more aligned with their diagnostic workflow.

Broad Applicability: Works out-of-the-box with any black-box model (LSTM, TCN, Transformer) without needing access to model internals, making it a versatile first choice for heterogeneous model deployments.

Non-Deterministic Output: LIME's perturbation process is random. Running it twice on the same anomaly can produce different explanations. This inconsistency can severely damage an engineer's trust in the system, leading them to dismiss valid alerts as 'the AI being glitchy.'

Local Fidelity Issues: The definition of 'locality' (how big the neighborhood of perturbations is) is a critical hyperparameter. A poorly chosen kernel width can lead to explanations that don't faithfully represent the model's true decision boundary, missing the real root cause of the anomaly.

Feature Correlation Blindness: LIME's sparse linear model can struggle when sensor readings are highly correlated (e.g., temperature and pressure rising together), often arbitrarily selecting one feature over another, which can misdirect the troubleshooting engineer.

Theoretical Guarantee: SHAP values are based on Shapley values from cooperative game theory, ensuring a fair, consistent distribution of feature importance across all predictions. This matters for audit-ready root cause analysis where you need to prove why a specific vibration frequency triggered an alert.

• Global Explanations: Aggregating SHAP values provides a reliable model-level view of degradation patterns.
• Stability: Identical inputs always produce identical explanations, critical for regulatory compliance in high-stakes environments.

Computational Efficiency: LIME creates local surrogate models (e.g., linear regression) around a single prediction, making it orders of magnitude faster than SHAP for complex black-box models. This matters for real-time edge inference where an engineer needs an instant, on-screen justification for a shutdown alarm.

• True Model-Agnosticism: LIME treats the model as a complete black box, working seamlessly with any custom deep learning architecture without needing internal access.
• Interpretable Sparse Explanations: LIME forces a limited number of features (e.g., top 3 sensors) into the explanation, reducing cognitive load for operators.

Exponential Complexity: Exact SHAP calculation is computationally expensive, scaling exponentially with the number of features. For high-dimensional sensor streams (e.g., 100+ IoT tags), KernelSHAP approximations can introduce significant latency (seconds to minutes). This makes it unsuitable for sub-second alert triage on edge gateways like NVIDIA Jetson.

Sampling Variance: LIME's explanations depend on random perturbations of the input data. Running LIME twice on the same anomaly can produce different feature importance rankings, eroding engineer trust. This instability is a major liability in safety-critical predictive maintenance where consistent diagnostic logic is mandatory.