Surrogate Model

The surrogate must be a model class that is natively transparent to humans. Common choices include sparse linear models (e.g., Lasso regression), shallow decision trees with limited depth, or rule lists. The model's structure itself serves as the explanation—coefficients directly indicate feature importance, and tree paths map to decision logic. This contrasts with post-hoc saliency maps on complex networks, which require secondary interpretation.

The surrogate does not attempt to explain the entire global model. It is trained to be accurate only in a tightly bounded neighborhood around the single instance being explained. This is enforced by an exponential kernel that weights perturbed samples by their proximity to the original instance. High local fidelity ensures the explanation reflects the model's decision boundary at that specific point, even if the global behavior is highly non-linear.

A defining architectural advantage is that the surrogate treats the original model as a complete black box. It requires only the ability to query a prediction function f(x)—no access to gradients, internal weights, or architecture is needed. This makes the technique universally applicable across model types, from gradient-boosted trees and random forests to proprietary APIs and ensemble stacks.

The surrogate rarely operates on raw input features. Instead, it uses an interpretable representation of the data. For images, raw pixels are converted to superpixel segments; for text, documents become a bag-of-words presence vector. This binary or simplified space ensures that the explanation's components map directly to human-understandable concepts like 'the presence of the word 'not'' or 'this patch of the image,' rather than abstract latent dimensions.

Human attention is limited, so the surrogate actively penalizes complexity. Techniques like L1 regularization (Lasso) drive the coefficients of less important features to exactly zero. The final explanation is a sparse linear combination of only a handful of features (typically 5-10). This sparsity is not a side effect but a core design goal, ensuring the output is a concise, non-overwhelming summary of the most critical decision drivers.

The surrogate model embodies a fundamental balancing act. A highly complex surrogate (e.g., a deep decision tree) could achieve near-perfect local fidelity but would be unreadable. A perfectly simple model (e.g., a single rule) is readable but may poorly approximate the black-box boundary. The kernel width hyperparameter directly controls this trade-off by defining the size of the local neighborhood, forcing a design choice between precise local replication and stable, simple explanations.