Graph Neural Network (GNN)

A Graph Convolutional Network (GCN) is a foundational GNN architecture that performs a localized spectral convolution. It aggregates feature information from a node's immediate neighbors, normalized by the nodes' degrees, to generate updated node embeddings. This operation is analogous to a learnable smoothing filter over the graph.

• Core Operation: Uses a first-order approximation of spectral graph convolutions.
• Key Feature: Employs a symmetric normalization (e.g., using the inverse square root of the degree matrix) to stabilize learning.
• Primary Use: Node classification, graph classification, and as a building block in more complex models.

A Graph Attention Network (GAT) introduces an attention mechanism to the neighborhood aggregation step. Instead of using fixed, pre-defined weights (like degree-based normalization in GCN), GAT computes dynamic, data-dependent attention coefficients for each edge. This allows the model to focus on the most relevant neighboring nodes.

• Core Operation: Uses multi-head self-attention over node features to compute aggregation weights.
• Key Feature: Provides interpretability through attention weights and handles varying importance of connections.
• Primary Use: Tasks where relationship strength is not uniform, such as protein interface prediction or social network influence analysis.

GraphSAGE (SAmple and aggreGatE) is an inductive framework designed to generate embeddings for unseen nodes. Unlike transductive models that require the entire graph during training, GraphSAGE learns aggregator functions (like mean, LSTM, or pooling) that can generalize to new graph structures.

• Core Operation: Samples a fixed-size neighborhood for each node and applies a learned aggregator function.
• Key Feature: Enables inductive learning and scalable, mini-batch training on large graphs.
• Primary Use: Dynamic graphs, production recommendation systems where new users/items are constantly added.

The Message Passing Neural Network (MPNN) framework provides a general, unifying abstraction for many GNNs. It formalizes the forward pass as two phases: a message passing phase where nodes exchange information along edges, and a readout phase to compute a graph-level representation.

• Core Operation: Message, Aggregate, and Update functions define the model.
• Key Feature: A flexible framework that generalizes GCN, GraphSAGE, and others; often used in molecular property prediction.
• Primary Use: Quantum chemistry, molecular graph regression, and any task requiring a flexible, physics-inspired model of interaction.

A Heterogeneous Graph Neural Network (HGNN) is designed for heterogeneous graphs containing multiple node types and edge types (e.g., a knowledge graph with entities like Person, Company, and relations like worksFor, competesWith). These models use type-specific parameters and meta-paths to handle semantic diversity.

• Core Operation: Employs network schema or meta-path-based neighbor sampling and type-aware transformations.
• Key Feature: Preserves and leverages rich semantic information inherent in multi-relational data.
• Primary Use: Knowledge graph completion, recommender systems with diverse entities, and academic citation networks.

A Temporal Graph Neural Network (TGNN) processes dynamic graphs where edges and node features change over time. These models incorporate temporal embeddings, recurrent units, or attention over time steps to capture evolving dependencies and forecast future states.

• Core Operation: Integrates sequential models (RNNs, Transformers) with spatial GNN layers or uses continuous-time dynamic models.
• Key Feature: Captures both structural topology and temporal dynamics for time-series predictions on graphs.
• Primary Use: Traffic forecasting, social network evolution, financial transaction network monitoring, and epidemic modeling.

Graph embedding is a foundational technique that maps nodes, edges, or entire graphs to low-dimensional vector representations (embeddings). These dense vectors preserve the structural and semantic properties of the original graph, enabling the application of standard machine learning algorithms. Unlike GNNs, which are deep learning models, many embedding methods (e.g., Node2Vec, DeepWalk) are shallow and generate static representations. GNNs often use or generate embeddings as part of their message-passing process for downstream tasks like node classification and link prediction.

A Graph Attention Network (GAT) is a specific and influential type of Graph Neural Network that incorporates an attention mechanism into the neighborhood aggregation step. Instead of treating all neighboring nodes equally, a GAT learns to assign different importance weights (attention scores) to each neighbor. This allows the model to dynamically focus on the most relevant connections when updating a node's representation. The key innovation is its computational efficiency and inductive capability, meaning it can generalize to graphs with unseen node structures.

A heterogeneous graph is a graph structure containing multiple types of nodes (entities) and/or multiple types of edges (relationships). This contrasts with a homogeneous graph, which has only one node and edge type. Heterogeneous graphs are ubiquitous in enterprise knowledge graphs (e.g., a graph containing Customer, Product, and Transaction node types). Specialized GNN architectures, like Heterogeneous Graph Neural Networks (HGNNs), are required to handle this rich semantic information by using type-specific parameters and meta-paths to propagate information across different relation types.

Message passing is the fundamental computational framework underpinning most Graph Neural Networks. It defines how information is propagated through the graph. The process operates in iterative layers:

• Message Function: Each node creates a "message" (a transformed feature vector) for its neighbors.
• Aggregation Function: Each node aggregates the messages received from its neighboring nodes (e.g., using sum, mean, or max).
• Update Function: The node combines its current representation with the aggregated message to produce an updated representation for the next layer. This local, iterative process allows nodes to incorporate information from their multi-hop neighborhood.

Link prediction is a core machine learning task on graphs where the goal is to predict the existence of a missing edge (link) between two nodes or the likelihood of a future connection forming. It is crucial for knowledge graph completion, recommendation systems, and network evolution modeling. GNNs excel at this task by learning low-dimensional representations for nodes that encode both their features and graph structural roles. The prediction is typically made by applying a decoder function (e.g., a dot product or MLP) to the learned embeddings of a pair of nodes to score the potential link.

Graph classification is a supervised learning task where the objective is to predict a class label or property for an entire graph. This contrasts with node-level or edge-level tasks. Applications include molecular property prediction (classifying a molecule graph as toxic or not), social network analysis, and document categorization. GNNs solve this by first performing node-level message passing, then using a global pooling (readout) function (e.g., sum, mean, or attention-based pooling) to aggregate all node representations into a single graph-level embedding, which is finally passed to a classifier.