Linear Proxy Model

The model's prediction is a direct linear combination of input features, making the contribution of each feature immediately visible through its learned coefficient.

• Coefficient as Importance: The magnitude and sign of each weight directly quantify the feature's impact on the prediction.
• No Interaction Terms: Standard linear proxies assume feature independence, providing a clear, additive explanation without complex interactions.
• Auditability: A human can trace the exact arithmetic path from input to output, satisfying regulatory requirements for transparent decision-making.

The Least Absolute Shrinkage and Selection Operator (LASSO) is a preferred linear proxy when only a few features truly drive the complex model's decision.

• L1 Regularization: LASSO adds a penalty equal to the absolute value of the coefficients, driving irrelevant feature weights to exactly zero.
• Automatic Pruning: This produces a sparse model that highlights only the most critical factors, reducing cognitive load for human reviewers.
• Example: In a credit risk black-box, a LASSO proxy might retain only 'debt-to-income ratio' and 'delinquency count' while zeroing out dozens of other variables.

A linear proxy can be trained to explain the entire model behavior or just a specific prediction, depending on the audit scope.

• Global Surrogate: Trained on a large sample of the black-box's input-output pairs to approximate its overall decision boundary. Useful for high-level model validation.
• Local Surrogate: Trained on perturbed samples around a single instance to explain that specific prediction. This is the core mechanism behind LIME.
• Fidelity Metric: The R-squared value between the proxy's predictions and the black-box's outputs measures how faithfully the linear model captures the original logic.

Linear proxies are often trained using knowledge distillation, where they learn from the teacher model's probability distribution rather than hard labels.

• Soft Targets: The complex model outputs a probability for each class (e.g., [0.1, 0.8, 0.1]). These contain 'dark knowledge' about class similarities.
• Rich Supervision: A logistic regression student trained on these probabilities learns a more nuanced decision boundary than one trained on binary 0/1 labels.
• Calibration: The resulting linear model often produces better-calibrated probability estimates, as it inherits the teacher's uncertainty quantification.

For a linear proxy to be trustworthy, its coefficients must be stable and statistically significant, not artifacts of noise.

• Confidence Intervals: Standard linear model theory allows calculation of p-values and confidence intervals for each weight, quantifying uncertainty in the explanation.
• Multicollinearity Check: High correlation between input features can cause coefficient instability. Variance Inflation Factor (VIF) analysis is used to diagnose this.
• Regularization Path: By varying the LASSO penalty, one can observe the order in which features enter the model, indicating their relative importance to the black-box.

A linear model cannot capture non-linear interactions or heterogeneous effects present in the original black-box, leading to fidelity gaps.

• Interaction Blindness: If the true model relies on an XOR-like relationship (e.g., 'high income AND low debt'), a purely additive linear proxy will misrepresent this logic.
• Low Fidelity Warning: A low R-squared value signals that the black-box's decision surface is too complex for a linear approximation, and a more expressive surrogate like an Explainable Boosting Machine (EBM) should be considered.
• Extrapolation Risk: The proxy's coefficients are valid only within the data distribution it was trained on; predictions on outliers may be meaningless.