SHAP vs LIME: Local Model Explainability

SHAP (SHapley Additive exPlanations) excels at theoretical consistency because it is grounded in cooperative game theory. By calculating Shapley values, it provides a mathematically unified measure of feature importance that fairly distributes the prediction among the input variables. For example, in a benefits eligibility model, SHAP can guarantee that the sum of individual feature contributions exactly equals the model's output, a property crucial for satisfying the 'meaningful information' requirement of legal mandates like the EU AI Act.

LIME (Local Interpretable Model-agnostic Explanations) takes a different approach by prioritizing speed and flexibility over theoretical guarantees. It works by creating a simpler, interpretable 'surrogate' model around a specific prediction. This results in a significant trade-off: LIME can generate explanations for virtually any black-box model in milliseconds, but the explanations are inherently local approximations that can be unstable. Two similar applicants might receive slightly different explanations due to the random sampling process LIME uses to build its surrogate model.

The key trade-off: If your priority is legal defensibility and mathematical consistency for high-stakes decisions like permit denials, choose SHAP. If you prioritize rapid prototyping and model flexibility across diverse, low-risk agency applications, choose LIME. For government CTOs, SHAP is often the safer default for automated decision-making transparency engines where auditability is non-negotiable.

SHAP excels at providing legally defensible, consistent explanations because of its foundation in game theory. For example, its Shapley values guarantee that the contribution of each feature fairly sums to the final prediction, a property critical for withstanding appeals in benefits denial cases. This consistency, however, comes at a steep computational cost, with exact KernelSHAP calculations scaling exponentially with the number of features, making it impractical for real-time citizen-facing portals without significant approximation.

LIME takes a different approach by creating a simple, interpretable 'surrogate' model around a single prediction. This results in a significant speed advantage, often generating explanations in milliseconds rather than seconds. The trade-off is fidelity; a LIME explanation is a local approximation that can be unstable, where two similar cases might yield slightly different explanations due to the random sampling process, a potential liability in high-stakes, precedent-driven government decisions.

The key trade-off: If your priority is legal defensibility and global consistency for high-stakes decisions like parole or benefits allocation, choose SHAP. If you prioritize speed and flexibility for exploratory analysis or lower-stakes triage where a 'good enough' local approximation is acceptable, choose LIME. For many agency CTOs, a hybrid strategy—using SHAP for offline model validation and audit trails, and LIME for rapid, on-the-fly citizen inquiries—provides the optimal balance of rigor and responsiveness.