Local Interpretable Model-agnostic Explanations (LIME)

LIME does not attempt to explain the entire global model. Instead, it generates a new dataset of perturbed samples around the instance of interest, weighting them by their proximity to the original input. It then trains a simple, inherently interpretable model—such as a sparse linear model or a decision tree—on this local neighborhood. The coefficients or feature splits of this surrogate model serve as the explanation, revealing which features were most influential for that specific prediction.

A defining characteristic of LIME is its complete independence from the underlying model's internal structure. The algorithm treats the original classifier as a black box, requiring only the ability to query it with inputs and observe the outputs. This means the same explanation pipeline can be applied to:

• Deep neural networks
• Gradient-boosted trees
• Support vector machines
• Any proprietary API This agnosticism makes LIME a universal tool for auditing diverse model portfolios without requiring access to gradients or architecture specifics.

To make explanations human-understandable, LIME operates on interpretable representations rather than raw features. For text classification, this means the presence or absence of individual words or n-grams. For image classification, the input is segmented into super-pixels (contiguous patches of similar pixels). The explanation then identifies which super-pixels or words were most critical. This abstraction bridges the gap between the model's complex feature space and a format a clinician or regulator can intuitively grasp.

To prevent information overload, LIME imposes a constraint on the complexity of the surrogate model. By default, it selects a small number of features, K, to present to the user. This is achieved through L1 regularization (Lasso) during the fitting of the local linear model. The result is a sparse explanation that highlights only the top predictive factors—for example, identifying that a specific pixel region and the absence of a certain word were the primary drivers of a diagnostic decision, filtering out noise.

LIME explicitly balances two competing goals through its objective function. Local fidelity measures how accurately the simple surrogate model mimics the complex model's behavior in the neighborhood of the instance. Interpretability is enforced by limiting the complexity of the surrogate (e.g., number of features). The algorithm optimizes a loss function that penalizes unfaithful approximations while rewarding simplicity. This mathematical framework allows developers to tune the explanation's strictness, ensuring the output is both trustworthy and comprehensible.

While LIME is inherently local, the accompanying Submodular Pick (SP-LIME) algorithm provides a global perspective. It selects a diverse, representative set of individual instances to explain, maximizing the coverage of important features across the dataset. This avoids redundancy—explaining ten nearly identical patients—and instead presents a non-redundant portfolio of archetypal cases. For a clinical diagnostic team, this means efficiently auditing model behavior across different patient phenotypes without manually sifting through thousands of individual explanations.

Quantitative measures that assess how accurately an explanation reflects the model's true reasoning process. Key metrics include:

• Comprehensiveness: How much the prediction changes when top-attributed features are removed
• Sufficiency: Whether the top-attributed features alone can produce a similar prediction
• Monotonicity: Whether adding features in attribution order consistently increases prediction probability These metrics are essential for comparing LIME against SHAP or Integrated Gradients in validation studies for FDA submissions.

A property of models that are understandable by design, such as linear regression, decision trees, or generalized additive models (GAMs). Unlike LIME's post-hoc approach of approximating a black box, intrinsically interpretable models provide explanations directly from their structure. In high-stakes diagnostics, regulators often prefer inherently transparent models over post-hoc explanations of opaque neural networks. The trade-off is that simpler models may sacrifice predictive accuracy, making LIME a pragmatic bridge when complex models are clinically necessary.

The specific application of feature attribution methods—including LIME—to identify which biological measurements most strongly influence a diagnostic model's prediction for an individual patient. This goes beyond statistical significance to reveal patient-specific biomarker importance. For example, LIME might show that HER2 expression dominates one patient's risk score while tumor size drives another's. This granularity supports personalized treatment planning and helps clinicians validate that the model's reasoning aligns with established biological pathways.