GSAT

Unlike deterministic attention, GSAT samples attention weights from a learned Bernoulli distribution for every edge. This injection of randomness acts as a regularizer, forcing the model to distinguish between robust, predictive edges and spurious correlations. The stochasticity prevents the model from collapsing into trivial solutions where all edges are deemed important.

• Randomness as a filter: Only edges consistently sampled with high probability survive.
• Gumbel-Softmax: Enables differentiable sampling for end-to-end training.

GSAT formalizes explanation as an information constraint problem. The objective maximizes mutual information between the selected subgraph and the label, while strictly minimizing mutual information between the subgraph and the original input graph. This mathematically guarantees the extraction of a minimal sufficient statistic for prediction.

• Compression: Penalizes large, uninformative subgraphs.
• Sufficiency: Rewards subgraphs that alone achieve high accuracy.

The core output of GSAT is a label-relevant subgraph—the minimal set of nodes and edges that drives the prediction. By training the stochastic attention with the information bottleneck, the model automatically learns to drop task-irrelevant structure, such as background noise in molecular graphs or social connections unrelated to a specific classification.

• Direct interpretability: The selected subgraph is the explanation.
• No post-hoc steps: Explanation is intrinsic to the forward pass.

To control the degree of sparsity, GSAT regularizes the learned attention distribution by minimizing the Kullback-Leibler (KL) divergence between the edge sampling probabilities and a prior distribution (typically a low-probability Bernoulli). This prevents the attention from saturating at 1.0 and ensures only the most critical edges are retained.

• Tunable sparsity: A hyperparameter controls the trade-off between compression and prediction.
• Stable training: Prevents gradient vanishing in the attention module.

GSAT can be extended to learn invariant subgraphs that generalize across different environments or domains. By penalizing subgraphs that rely on environment-specific correlations, the model identifies causal structures that hold universally, not just in the training distribution. This is critical for scientific discovery tasks like drug response prediction.

• Causal discovery: Separates correlation from causation.
• Domain generalization: Explanations remain valid under distribution shift.

GSAT is trained as a single, end-to-end differentiable pipeline. The stochastic sampling is handled via the Gumbel-Softmax reparameterization trick, allowing gradients to flow through the discrete attention selection. This eliminates the need for separate explainer and predictor models, reducing computational overhead and ensuring the explanation is faithful to the predictor's internal logic.

• Joint optimization: Explainer and predictor are one model.
• Faithfulness by design: No approximation gap between explanation and prediction.

The foundational principle upon which GSAT is built. The information bottleneck objective compresses an input graph into a minimal subgraph that retains maximal mutual information with the label. GSAT implements this by injecting stochasticity into attention weights, treating edge selection as a variational approximation to the bottleneck. The model learns to discard irrelevant structural noise while preserving only label-relevant connections.

A self-explainable framework where a generator-predictor architecture produces explanations without post-hoc analysis. The generator extracts a concise, causal subgraph called the rationale, and the predictor makes decisions based solely on that rationale. GSAT functions as a rationalization method by using its stochastic attention to sample the rationale during training, ensuring the predictor only sees the explanation subgraph.

The primary evaluation criterion for GSAT's explanations. Faithfulness measures how accurately an explanation subgraph reflects the GNN's true reasoning process. It is quantified by the drop in prediction performance when the explanation is removed or when a random subgraph is used instead. GSAT explicitly optimizes for faithfulness through its information bottleneck objective, ensuring the selected edges are causally relevant, not merely correlated.

A fidelity assessment technique used to validate GSAT's explanations. The method systematically masks or removes the most important nodes and edges identified by the explainer and measures the resulting change in prediction. A faithful explanation causes a sharp performance drop when removed. GSAT's stochastic attention mechanism is trained to be robust to such perturbations, as the predictor learns to rely exclusively on the sampled subgraph.

A model-agnostic post-hoc explainer that identifies compact subgraphs and node features most influential for a GNN's prediction. Unlike GSAT, which is self-explainable and trained end-to-end with the bottleneck objective, GNNExplainer operates on a frozen model by maximizing mutual information between the subgraph and the prediction. GSAT generally produces sparser and more faithful explanations because the predictor is co-trained to depend on the rationale.

A technique for interpreting Graph Attention Networks by tracing how attention weights propagate across layers. GSAT extends this concept by making attention stochastic and regularized. Instead of deterministic aggregation, GSAT samples edges from a learned probability distribution, injecting noise that forces the model to identify only the most robust, label-relevant connections. The final attention probabilities serve as the explanation.