Self-Explaining Neural Networks (SENN)

SENNs enforce a globally interpretable functional form where the final prediction is a linear combination of concept activations. Unlike post-hoc methods, this structure is a hard architectural constraint, not an approximation.

• Prediction formula: f(x) = Σ θ(x)ᵢ · cᵢ(x)
• θ(x) generates relevance scores for each concept
• cᵢ(x) represents the activation of human-specified concepts
• Linearity ensures the contribution of each concept is trivially separable

A dedicated sub-network maps raw input features to a concept bottleneck — a latent space where each dimension corresponds to a predefined, human-understandable concept. This forces the model to reason in terms of interpretable primitives.

• Concepts can be supervised (annotated) or unsupervised (discovered)
• The bottleneck prevents the model from using opaque feature interactions
• Example: For loan approval, concepts might include 'credit history length' and 'debt-to-income ratio'

The θ(x) network dynamically computes the importance weight of each concept for a specific input. This is the mechanism that makes explanations input-dependent — the same concept can be highly relevant for one prediction and irrelevant for another.

• Outputs a sparse weight vector over the concept space
• Enables local interpretability within a globally transparent model
• Regularized to encourage sparsity, producing concise explanations

SENNs are trained with a specialized robustness penalty that enforces explanation stability. The loss function penalizes large changes in concept relevance scores when the input is perturbed by small, semantically meaningless noise.

• L_θ constraint: ||θ(x) - θ(x + δ)|| must be small for small δ
• Prevents adversarial explanation manipulation
• Produces rationales that are consistent under minor input variations
• Critical for building user trust in the generated explanations

Unlike post-hoc rationalization, SENN explanations are faithful by design. The concept weights θ(x) are the exact coefficients used in the forward computation — there is no secondary explainer model that can introduce misrepresentation.

• Eliminates the explanation-prediction gap found in LIME or SHAP
• The rationale is a direct readout of the model's internal computation
• Auditable: regulators can verify that the explanation matches the arithmetic
• Contrast with plausible rationales that sound convincing but may be fabricated

The entire SENN architecture — concept encoder, relevance parameterizer, and linear aggregator — is a single differentiable computation graph. This enables joint optimization of both prediction accuracy and explanation quality through standard gradient descent.

• No alternating optimization or surrogate model training required
• Backpropagation flows through the concept bottleneck
• Allows fine-tuning on domain-specific interpretability requirements
• Compatible with modern deep learning frameworks without custom infrastructure

A generated explanation that accurately reflects the true internal reasoning process of the model, not just a plausible-sounding story. In the context of SENN, faithfulness is architectural: the explanation is a direct function of the model's own latent concepts, not a post-hoc approximation. This contrasts sharply with models where an external explainer guesses at the logic.

Justifications structured around high-level, human-understandable concepts rather than low-level input features like individual pixels. SENNs learn to map raw inputs to a sparse set of interpretable basis concepts during training. For example, a medical imaging SENN might explain a diagnosis using concepts like 'tissue density' and 'margin irregularity' instead of highlighting arbitrary pixel clusters.

The degree to which a generated rationale mirrors the true computational logic used by the model. SENNs are designed to maximize faithfulness by making the explanation a constrained, differentiable component of the forward pass. Key properties include:

• Fidelity: The explanation must causally influence the prediction.
• Stability: Similar inputs must yield similar explanations.
• Completeness: The explanation must account for the entire output.

The technique of generating an explanation after the model has made its decision, often using a secondary explainer model like LIME or SHAP. This is the antithesis of the SENN approach. Post-hoc methods are susceptible to confirmation bias, where the explainer constructs a convincing but unfaithful story. SENNs eliminate this gap by making the explanation a first-class output of the primary model.

The ability of a human observer to use a model's explanation to correctly anticipate the model's output on a new, unseen input. This is a strict, empirical test of interpretability. A highly simulatable SENN allows an auditor to internalize the model's logic by reading the generated concept weights and reasoning trace, then accurately predict how it will classify a held-out sample.

The field of reverse-engineering the internal computations and learned algorithms within a neural network's weights. While mechanistic interpretability dissects a black box after training, SENN represents a transparency-by-design approach. It constrains the architecture so that the mechanism is legible by construction, mapping latent representations directly to a human-specified concept basis.