Graph Neural Network (GNN)

The fundamental mechanism enabling GNNs to learn. Each node aggregates feature vectors from its neighbors, combines them with its own state, and updates its representation through a neural network layer.

• Aggregation Function: A permutation-invariant operation like SUM, MEAN, or MAX that collects neighbor states.
• Update Function: A learnable neural network (e.g., MLP or GRU) that computes the new node embedding.
• Iterative Process: Stacking k layers allows a node to incorporate information from its k-hop neighborhood, expanding the receptive field.

A critical mathematical property ensuring that the output of a GNN layer is independent of the arbitrary ordering of input nodes. Since graphs lack a canonical node order, the aggregation function must produce the same result regardless of how neighbors are indexed.

• Achieved via: Symmetric aggregation functions (SUM, MEAN, MAX).
• Contrast: Standard CNNs rely on spatial order; GNNs operate on sets.
• Failure Case: Using a non-invariant aggregator (e.g., an RNN over a fixed neighbor sequence) breaks this property and leads to unstable training.

Two primary mathematical frameworks define how convolution is performed on graphs.

• Spectral Methods: Define convolution via the graph Laplacian's eigendecomposition in the Fourier domain. ChebNet approximates filters using Chebyshev polynomials to avoid costly eigenvector computation.
• Spatial Methods: Define convolution directly on the graph topology by aggregating neighbor features. GraphSAGE samples a fixed-size neighborhood, while GAT learns attention weights for each neighbor.
• Practical Dominance: Spatial methods are generally preferred due to scalability and the ability to handle evolving graph structures.

An architecture that introduces a self-attention mechanism to the aggregation step, allowing the model to implicitly assign different importance to different neighbors.

• Attention Coefficients: Computed for each edge using a shared attentional mechanism, followed by a softmax normalization.
• Multi-Head Attention: Stabilizes learning by running multiple independent attention mechanisms in parallel and concatenating (or averaging) their outputs.
• Key Advantage: Unlike GraphSAGE's uniform sampling, GATs dynamically weight neighbors, making them robust to noisy or irrelevant connections in the graph.

The ability of a trained GNN model to generate embeddings for entirely unseen nodes or new graphs without retraining. This is a defining characteristic of spatial GNNs like GraphSAGE.

• Mechanism: The model learns an aggregation function parameterized by the node's features, not a fixed embedding lookup table (as in transductive methods like DeepWalk).
• Use Case: A GNN trained on a static legal citation network can immediately compute an embedding for a newly published opinion based on its text features and initial citations.
• Contrast: Transductive node embedding methods require expensive retraining to incorporate new nodes.

A fundamental limitation where stacking too many GNN layers causes node representations to converge and become indistinguishable, losing local information.

• Cause: Repeated averaging of neighbor features acts as a low-pass filter, driving node embeddings toward a global equilibrium state.
• Mitigation Strategies:
  • Skip Connections: Residual connections (as in GCNII) add the initial layer's input to deeper layers.
  • DropEdge: Randomly removing edges during training to slow information propagation.
  • Layer Normalization: Stabilizing feature distributions across layers.
• Practical Limit: Most GNN architectures rarely exceed 2-4 layers effectively.

A spectral GNN variant that performs convolution operations directly on graph topology. GCNs use a localized first-order approximation of spectral graph convolutions, making them computationally efficient for large legal citation networks. Each layer applies a shared weight matrix followed by a normalized adjacency-based aggregation.

• Semi-supervised learning: Propagates label information across edges
• Transductive: Operates on fixed graph structure during training
• Legal application: Classifying case nodes by practice area using citation links

A GNN architecture that introduces self-attention mechanisms to learn the relative importance of neighboring nodes during aggregation. Instead of treating all neighbors equally, GATs compute attention coefficients that weight messages based on node feature similarity. In legal knowledge graphs, this enables the model to prioritize highly relevant precedent citations over tangential references.

• Multi-head attention: Parallel attention mechanisms for stability
• Dynamic weighting: Importance scores adapt per node pair
• Inductive capability: Generalizes to unseen graph structures

An inductive framework that generates embeddings by sampling and aggregating features from a node's local neighborhood. Unlike transductive GCNs, GraphSAGE learns aggregator functions (mean, LSTM, pooling) that generalize to entirely unseen nodes. Critical for dynamic legal graphs where new cases, statutes, and contracts are continuously added.

• Neighbor sampling: Fixed-size samples for computational efficiency
• Inductive learning: Embed unseen nodes without retraining
• Legal use case: Embedding newly filed complaints into existing case law graphs

An extension of GCNs designed for multi-relational knowledge graphs with typed edges. R-GCNs maintain separate weight matrices per relation type, enabling distinct transformations for different legal relationships like 'cites', 'overturns', 'distinguishes', or 'applies'. Essential for modeling the rich semantics of legal citation networks.

• Relation-specific weights: One transformation per predicate type
• Basis decomposition: Reduces parameter explosion for many relations
• Link prediction: Predict missing legal citations or statutory references

A theoretically grounded GNN architecture proven to be maximally expressive under the Weisfeiler-Lehman graph isomorphism test. GINs use a multi-layer perceptron on summed neighbor features with a learnable epsilon parameter. In legal contexts, this expressiveness enables precise distinction between structurally similar but semantically different contractual arrangements.

• WL-test equivalent: Distinguishes non-isomorphic graph structures
• Sum aggregation: Preserves multiset cardinality information
• Expressiveness guarantee: Captures subtle structural legal patterns