Local Fidelity

Local fidelity is defined at the level of a single data point. An explanation with high local fidelity approximates the decision boundary of the complex model only in the immediate vicinity of that specific instance. It does not claim to explain the model's global behavior. For example, a LIME explanation fits a simple linear model to the black-box model's predictions on a dataset of perturbed samples generated around the instance of interest.

The concept is model-agnostic, meaning it applies to any machine learning model (e.g., deep neural networks, gradient-boosted trees) regardless of its internal architecture. The explanation method itself (e.g., SHAP, LIME) is separate from the model being explained. This allows evaluation teams to use a consistent faithfulness metric like infidelity or sufficiency across a heterogeneous model portfolio.

Local fidelity is not binary but a spectrum, measured by quantitative faithfulness scores. Key metrics include:

• Infidelity: Measures the expected squared error between the explanation's importance scores and the actual change in model output when the input is perturbed.
• Sufficiency: Assesses if the top-K features identified by the explanation are sufficient for the model to make its original prediction.
• Completeness: Checks if the explanation accounts for the total change in prediction from a baseline. Low scores indicate the explanation is an unreliable proxy for the model's local logic.

It is crucial to distinguish local fidelity from global interpretability. A globally interpretable model (like a small decision tree) is understandable in its entirety. A post-hoc explanation with high local fidelity only provides a trustworthy 'snapshot' of model behavior for one input. An explanation method can have high local fidelity but poor global consistency, as the local approximations may not cohere into a single global narrative.

The primary technical method for assessing local fidelity is perturbation analysis. The core assumption: if an explanation's feature importance scores are correct, then systematically perturbing important features should cause a large change in the model's output, while perturbing unimportant features should cause little change. This is the operational basis for metrics like infidelity. Automated sensitivity analysis frameworks execute these perturbations at scale to generate fidelity scores.

In enterprise settings, local fidelity is a non-negotiable prerequisite for human-AI agreement and simulatability. If an explanation lacks fidelity, a data scientist cannot reliably use it to debug a model error, nor can a regulatory auditor trust it to verify compliance. High local fidelity ensures that the explanation provided to a human is a truthful account of 'what the model saw' for that specific case, forming the basis for actionable insight and governance.

A quantitative metric that directly measures how accurately an explanation reflects the true reasoning process of the underlying model for a specific prediction. It is the primary operationalization of local fidelity.

• Core Concept: A high faithfulness score indicates the explanation's feature importance rankings would correctly predict how the model's output changes if those features were modified.
• Measurement: Often calculated using perturbation analysis, where input features are altered based on the explanation and the resulting prediction change is compared to the explanation's expectation.

A foundational explanation validation technique that systematically modifies or removes input features to empirically test an explanation's claims about local fidelity.

• Method: The input is perturbed (e.g., a word is removed from text, a region is masked in an image) according to the explanation's importance scores.
• Validation: The change in the model's prediction is observed. A faithful explanation should predict this change: if a feature is deemed important, perturbing it should cause a large prediction shift.

An explanation metric that quantifies the degree to which an explanation fails to accurately reflect the model's local behavior. It is the inverse of faithfulness.

• Calculation: Formally defined as the expected squared error between the explanation's prediction of model output change and the actual change, under a meaningful perturbation distribution.
• Purpose: Provides a single, comparable score. Lower infidelity is better, indicating higher local fidelity.

A model-agnostic explanation method whose core premise is the construction of a locally faithful explanation. LIME explicitly optimizes for local fidelity.

• Mechanism: For a given prediction, LIME generates perturbed samples around the instance, queries the complex model for these samples, and fits a simple, interpretable model (e.g., linear regression) to this local dataset.
• Output: The coefficients of the simple model serve as the feature importance explanation, which, by design, approximates the complex model's behavior locally.

A complementary explanation metric that evaluates whether the subset of features identified as most important by an explanation is, by itself, sufficient for the model to make its original prediction.

• Test: The top-k most important features (according to the explanation) are presented to the model, while other features are masked or set to baseline values.
• Interpretation: If the model's prediction remains the same or very similar using only these top features, the explanation is deemed sufficient. It measures comprehensiveness alongside local fidelity.

In explainability, this evaluates how small changes in input features affect both the model's prediction and the generated explanation. It tests the stability and robustness of local fidelity.

• Dual Assessment: It measures: 1) Prediction sensitivity (does the output change?), and 2) Explanation sensitivity (does the importance ranking change?).
• Goal: A robust, locally faithful explanation should be relatively stable under small, semantically-preserving perturbations, ensuring the explanation is not an artifact of noise.