Model-Agnostic

Model-agnostic methods require no visibility into a model's internal architecture, weights, or gradients. They interact exclusively through the prediction function f(x), querying the model with inputs and receiving outputs or probability scores. This constraint ensures compatibility with any model—from deep neural networks to gradient-boosted trees—and even with models exposed only via a remote API. The method treats the model as an oracle, making it universally applicable in production environments where internal access is restricted.

These methods generate explanations after a model has been fully trained, without altering the original training procedure or architecture. They are retrofitted onto existing black-box systems, making them ideal for auditing legacy models or third-party APIs. Key implications:

• No retraining or architectural changes required
• Can explain models that are already in production
• Enables independent third-party auditing without model access
• Contrasts with intrinsic interpretability, where the model is designed to be transparent from the start

The explanation system interacts with the model through a single, well-defined interface: the prediction function. This function maps an input instance to an output prediction or probability distribution. The method is agnostic to whether this function is a neural network forward pass, a tree traversal, or an ensemble vote. This abstraction layer is what enables the same explanation algorithm—such as LIME or SHAP—to explain a random forest one day and a transformer the next without any code changes.

To understand a model's decision boundary, model-agnostic methods systematically perturb the input instance and observe how predictions change. This creates a synthetic local neighborhood of labeled data points—the model's predictions serve as labels. The perturbation strategy varies by data modality:

• Text: Token masking or word removal
• Images: Superpixel occlusion or blurring
• Tabular: Feature value randomization or quantile shifting
• Audio: Spectrogram masking The resulting input-output pairs train a local surrogate model that reveals which features drive predictions.

At the core of model-agnostic explanation is the training of an intrinsically interpretable surrogate model—typically a sparse linear model or shallow decision tree—on the perturbed dataset. This surrogate approximates the black-box model's decision boundary in a local region around the instance of interest. The surrogate's coefficients or feature splits become the explanation. The key trade-off is local fidelity vs. global simplicity: the surrogate must be simple enough for humans to understand while accurately mimicking the complex model's behavior in the immediate neighborhood.

The model-agnostic property extends beyond model type to data modality. The same explanation framework can handle tabular data, images, text, and time series by swapping only the perturbation strategy and interpretable representation. For images, superpixels serve as interpretable features; for text, individual words or n-grams; for tabular data, the original features or discretized bins. This universality makes model-agnostic methods the standard choice for enterprise platforms that must explain heterogeneous model portfolios.

The fundamental interaction paradigm for model-agnostic methods. The explanation system can only query the model with inputs and receive outputs or prediction probabilities, with zero visibility into internal parameters, gradients, or architecture. This is the lowest common denominator of model access, making it universally applicable.

• Key distinction: Contrasts with white-box access, which requires internal weights and gradients
• Practical implication: Works on proprietary APIs, vendor models, and legacy systems
• Limitation: Requires many queries to probe the decision boundary, increasing computational cost

An explanation generated after a model has been trained, applied to an existing black-box system without requiring any modification to the model's internal architecture or training procedure. This is the defining temporal characteristic of model-agnostic methods.

• Contrast with: Intrinsic interpretability, where the model itself is designed to be transparent (e.g., decision trees)
• Advantage: Can be retrofitted to any deployed model without retraining or approval from the model owner
• Trade-off: May not perfectly capture the true reasoning process, only an approximation of it

A type of model explanation that provides the specific reasons for a single, individual prediction, as opposed to a global explanation that describes the average behavior of the entire model. Model-agnostic methods typically operate at this granularity.

• Use case: Debugging a specific loan denial or medical diagnosis
• Mechanism: Builds a local surrogate model around the single instance of interest
• Complementary concept: Global explanations summarize overall feature importance across all predictions

A class of explanation methods, including LIME and SHAP, that decompose a model's prediction into a sum of individual feature contributions relative to a baseline. This provides a linear explanation of the output, making it inherently interpretable to humans.

• Mathematical form: f(x) = φ₀ + Σ φᵢ where φᵢ is the contribution of feature i
• Unifying framework: Both LIME and SHAP are additive methods, though SHAP adds game-theoretic consistency guarantees
• Baseline choice: The reference point (often the average prediction) critically shapes the explanation

An intrinsically interpretable model, such as a linear regression or decision tree, trained to mimic the predictions of a complex black-box model within a specific local region. The surrogate's simplicity is what makes the explanation human-readable.

• Global vs. local: Global surrogates approximate the entire model; local surrogates approximate a single prediction's neighborhood
• Fidelity requirement: Must accurately replicate the black-box's behavior in the region of interest
• Common choices: Sparse linear models (Lasso), shallow decision trees, rule lists

The process of generating a synthetic neighborhood of data points by randomly altering or masking features of the original instance. This creates the training set for the local surrogate model, enabling it to learn the black-box decision boundary.

• Text data: Randomly removing words or tokens from a document
• Image data: Masking out superpixels (contiguous pixel groups)
• Tabular data: Sampling from feature distributions around the instance
• Critical parameter: The number of samples determines the fidelity of the local approximation