Sufficiency

Sufficiency is a faithfulness metric that validates feature attribution explanations. The core test is: if you provide only the top-k most important features (as identified by the explanation) to the model, does it still make the same prediction with high confidence?

• Procedure: 1) Generate an explanation (e.g., SHAP, LIME) for a prediction. 2) Isolate the top-k ranked features. 3) Mask or ablate all other features. 4) Feed this minimal, explanation-derived input back into the model. 5) Measure if the original predicted class probability remains high (e.g., > 95%).
• A high sufficiency score indicates the explanation has identified a truly predictive subset, making it a credible summary of the model's logic for that instance.

Sufficiency and completeness are complementary but distinct validation metrics.

• Sufficiency asks: "Are the highlighted features enough to cause the prediction?" It's a test of predictive power.
• Completeness asks: "Do the highlighted features account for all reasons for the prediction?" It's a test of explanatory coverage.

An explanation can be sufficient but not complete (the top features cause the prediction, but other minor features also contributed). It can also be complete but not sufficient (the explanation lists all contributing features, but the top-ranked ones alone aren't decisive). Ideal explanations score highly on both axes.

A central tension in explanation design is between sparsity (fewer features highlighted) and sufficiency (the highlighted features must be predictive).

• High Sparsity, Low Sufficiency: An explanation is overly simplistic. The one or two features it highlights are not, by themselves, enough for the model to be confident.
• Low Sparsity, High Sufficiency: An explanation lists many features. While this subset is sufficient, it is not a concise or human-interpretable summary.

Practitioners often plot a sufficiency-sparsity curve: as k (the number of top features selected) increases, sufficiency scores typically rise. The optimal k is where the curve begins to plateau, achieving a parsimonious yet faithful explanation.

The sufficiency metric is calculated as the model's output probability for the original predicted class when given only the explanation-selected features.

Formula: Suff(f, x, E, k) = f_y(x_E^k) Where:

• f is the model.
• x is the original input.
• E is the explanation method (e.g., SHAP).
• k is the number of top features selected.
• x_E^k is a modified input where only the top-k features from E are retained (others are set to a baseline).
• f_y is the model's output probability for the original class y.

A score of 1.0 means the minimal feature subset perfectly reproduces the original prediction confidence. Scores below ~0.8 suggest the explanation may be missing critical factors.

Sufficiency is a powerful tool for model debugging and regulatory auditing.

• Detecting Clever Hans Predictors: If a model makes a correct prediction for the wrong reason (e.g., a radiology model uses a hospital watermark to predict disease), sufficiency scores will be low. The explanation will highlight spurious features (the watermark) which, when isolated, do not support the prediction.
• Validating for High-Stakes Decisions: In credit lending or medical diagnostics, auditors require explanations that are not just plausible, but causally sufficient. A low sufficiency score flags an explanation as unreliable for justifying an automated decision.
• Comparing Explanation Methods: By measuring the average sufficiency score across a dataset, you can objectively rank explanation techniques (SHAP vs. LIME vs. Integrated Gradients) for a given model.

While crucial, sufficiency has key limitations that must be accounted for in practice.

• Baseline Sensitivity: The score depends heavily on how non-selected features are masked (e.g., set to zero, mean, or a neutral value). The choice of baseline must be semantically meaningful for the data type.
• Model Dependence: The test uses the original model f as the arbiter of truth. If f is itself flawed or non-robust, sufficiency measures faithfulness to a flawed process.
• Correlated Features: In datasets with high multicollinearity, many subsets of features may be sufficient, making it hard to pinpoint a single 'correct' explanation.
• Computational Cost: Requires k forward passes per explanation to create a full sufficiency curve, which can be expensive for large models or datasets.

It is therefore best used in conjunction with other metrics like completeness, stability, and human-AI agreement.

A faithfulness score is a quantitative metric that measures how accurately an explanation reflects the true reasoning process or causal factors of the underlying model for a given prediction. It directly assesses whether the features identified as important by the explanation method are genuinely influential to the model's internal computation.

• Core Principle: A faithful explanation should have a high correlation between the importance scores it assigns and the actual impact on the model's output when those features are perturbed.
• Contrast with Sufficiency: While sufficiency asks 'Is this subset enough for the prediction?', faithfulness asks 'Do these features truly cause the prediction?' A subset can be sufficient without being faithful if the model uses a different, correlated set of features.

A completeness score is a metric that evaluates whether an explanation accounts for all features or factors that contributed significantly to a model's prediction. It is often considered the conceptual complement to sufficiency.

• Mathematical Relationship: In many axiomatic frameworks, a good explanation should satisfy both completeness (the sum of importance scores equals the model's output deviation from a baseline) and sufficiency (the top-ranked features reproduce the output).
• Practical Implication: An explanation with high completeness but low sufficiency may identify many relevant features but fail to isolate a compact, decisive subset. Conversely, a sufficient subset may not fully account for the entire prediction's rationale.

Perturbation analysis is a foundational explanation validation technique that systematically modifies or removes input features to observe the resulting changes in the model's output. It is the experimental basis for calculating both sufficiency and faithfulness scores.

• Methodology: Features are ablated (set to zero, masked, or replaced with baseline values) based on the order of importance provided by an explanation. The resulting drop in model prediction confidence or change in logit scores is measured.
• Sufficiency Test: To measure sufficiency, only the top-k features identified by the explanation are retained (all others are perturbed to baseline). If the model's prediction remains unchanged, the explanation is deemed sufficient.

Infidelity is an explanation metric that quantifies the degree to which an explanation fails to accurately reflect the model's output when the input is perturbed according to the explanation's importance scores. It is a formal measure of unfaithfulness.

• Calculation: Infidelity is defined as the expected squared error between the dot product of the explanation's importance vector and a meaningful input perturbation, and the actual difference in the model's output caused by that perturbation.
• Relationship to Sufficiency: A low-infidelity explanation is generally more faithful. However, an explanation can have low infidelity (be locally faithful) but not be sufficient if the selected important features, while correctly weighted, are not the minimal set required to maintain the prediction.

Anchors are a model-agnostic explanation method that provides a high-precision rule (an 'anchor') consisting of a set of if-then conditions on input features that sufficiently 'anchors' the prediction, making it locally robust to other feature changes. The method directly operationalizes the concept of sufficiency.

• Output: An Anchor explanation takes the form: "IF [feature A = value X] AND [feature B > value Y], THEN the prediction is Z with high probability (e.g., 95%), even if all other features are changed."
• Sufficiency Guarantee: By construction, an Anchor identifies a sufficient condition for the prediction. The precision parameter of the Anchor algorithm controls how confident it must be that the rule holds when other features are perturbed, making sufficiency a core design goal.

Local fidelity is a property of a post-hoc explanation that measures how well the explanation approximates the behavior of the complex model in the immediate vicinity of a specific input instance. It is a prerequisite for both sufficiency and faithfulness.

• Scope: Unlike global interpretability, which seeks to explain the entire model, local fidelity concerns only the model's behavior for inputs similar to the instance being explained.
• Connection to Sufficiency: A locally faithful explanation (e.g., from LIME) builds a simple surrogate model that mimics the complex model locally. The sufficiency of an explanation can be seen as a stricter test of this local fidelity: not only must the explanation approximate the model's output function, but its highlighted features must also be capable of reproducing the exact prediction in isolation.