Transparent-by-Design Student

The defining characteristic is that the model's architecture is self-explanatory. Unlike a compressed neural network, these students are chosen from glass-box model families where the reasoning mechanism is directly visible.

• Generalized Additive Models (GAMs): The final prediction is a sum of univariate functions, allowing direct visualization of each feature's contribution via shape plots.
• Shallow Decision Trees: The logic is a human-readable flowchart of if-then rules.
• Explainable Boosting Machines (EBMs): An additive model that also includes pairwise interaction terms, remaining fully intelligible.
• Rule Lists: A sparse set of prioritized if-then statements.

The student must be a high-fidelity global surrogate, accurately approximating the teacher's decision boundary across the entire input space, not just locally. The training objective is to minimize the divergence between the teacher's soft targets and the student's predictions.

• Fidelity Metric: Measured by the agreement rate between teacher and student predictions on unseen data, not just accuracy against ground truth.
• Trade-off Navigation: The distillation process explicitly manages the interpretability-accuracy tradeoff, accepting a small, measurable drop in fidelity to gain complete transparency.
• Dark Knowledge Transfer: The student learns the teacher's generalization structure by matching soft targets, which encode inter-class similarities.

The student provides native feature importance scores without requiring external post-hoc tools like SHAP or LIME. The explanation is a direct byproduct of the model's mathematical structure.

• Linear Proxy Models: A distilled logistic regression student provides a coefficient for each feature, directly quantifying its global influence.
• GAM Shape Functions: A distilled GAM outputs a graph for each feature showing how the prediction changes as the feature value varies.
• Decision Tree Paths: A distilled decision tree provides the exact sequence of splits that led to a prediction, with no ambiguity about feature interactions.

The student architecture is selected to be amenable to deterministic rule extraction, converting the learned weights into a set of human-readable logical statements.

• Rule-Regularized Distillation: A penalty term is added to the loss function to encourage the student to form simple, axis-aligned decision boundaries that translate cleanly into if-then rules.
• SIRUS Algorithm: A specific distillation target that produces a stable and sparse rule list from the teacher's outputs.
• Logical Form: The final output is a set of conjunctive rules (e.g., IF age < 40 AND income > $80k THEN approve), providing a complete and auditable logic flow.

A transparent student must not only mimic the teacher's predictions but also faithfully represent the teacher's confidence calibration. The student's probability estimates should reflect the true likelihood of correctness.

• Temperature Scaling: The softmax temperature used during distillation is tuned so the student learns the teacher's calibrated uncertainty, not just its overconfident hard predictions.
• Reliable Probabilities: A distilled logistic regression or EBM student provides well-calibrated probability outputs that can be used for downstream risk assessment.
• Epistemic vs. Aleatoric: While simple students may not fully decompose uncertainty sources, their transparent structure allows engineers to clearly identify regions where the model is uncertain.

The student is designed to serve as the auditable artifact for governance and compliance workflows. Its transparent structure allows a human operator to trace and contest any decision.

• Complete Decision Trace: Every prediction can be decomposed into the sum of feature contributions (GAM) or a logical path (decision tree), satisfying regulatory requirements for adverse action reasons.
• Bias Detection: The explicit feature effects allow compliance officers to directly inspect the model for unwanted discriminatory patterns without relying on proxy fairness metrics.
• Stable Explanations: Unlike local surrogate methods that can produce unstable explanations for similar inputs, a global transparent student provides a consistent, deterministic explanation framework.

A statistical model where the prediction is a sum of smooth functions of individual features: g(E[y]) = β₀ + Σ fᵢ(xᵢ). When used as a distillation student, a GAM provides a fully transparent mapping from each input feature to its contribution, enabling auditors to inspect the exact effect of any variable on the output.

• Shape functions visualize how each feature influences predictions
• Modular interpretability: Each fᵢ(xᵢ) can be examined in isolation
• No interactions by default, ensuring complete feature independence

A globally interpretable tree-based model trained on the input-output pairs of a black-box teacher. The resulting tree provides a human-readable flowchart of decision logic, where each path from root to leaf represents a conjunctive rule.

• CART or C4.5 algorithms commonly used as student architectures
• Fidelity metric measures how closely the tree mimics the teacher
• Tradeoff: Deeper trees increase fidelity but reduce interpretability
• Rule extraction: Each leaf node translates directly to an if-then rule

A simple linear model—such as LASSO or logistic regression—trained to mimic a complex teacher's predictions. The learned coefficients serve as global feature importance scores, providing a first-order approximation of the teacher's decision boundary.

• LASSO regularization enforces sparsity, selecting only the most influential features
• Coefficient sign and magnitude directly indicate direction and strength of influence
• Calibrated probabilities when trained on teacher soft targets via temperature scaling

A student model that produces a stable and sparse set of if-then rules extracted from a random forest teacher. The SIRUS algorithm narrows the vast rule space of an ensemble into a compact, human-auditable rule list with statistical stability guarantees.

• Rule pruning removes redundant or low-frequency rules
• Stability: Narrow rule selection ensures consistent explanations across similar inputs
• Output format: Ordered decision list where the first matching rule determines the prediction

A training method that adds a penalty term to the standard distillation loss to encourage the student model's decision boundaries to be simple and amenable to extraction as a compact set of logical rules. This approach bakes interpretability directly into the optimization objective.

• Loss = Distillation Loss + λ · Rule Complexity Penalty
• Encourages axis-aligned or low-curvature decision boundaries
• Post-training rule extraction becomes more faithful and concise