Graphormer

A GNN variant that employs self-attention mechanisms to assign learnable importance weights to different neighboring nodes during message aggregation.

• Dynamic weighting: Unlike Graphormer's global structural encoding, GAT computes attention scores based solely on node features.
• Masked self-attention: Operates only over immediate neighbors, making it computationally efficient but limited in capturing long-range dependencies.
• Multi-head attention: Stabilizes learning by running multiple attention mechanisms in parallel, a concept shared with the Transformer architecture at Graphormer's core.

A classical graph isomorphism heuristic that iteratively refines node labels based on their neighbors, serving as the theoretical upper bound for the discriminative power of message-passing GNNs.

• Expressivity ceiling: Standard MPNNs cannot distinguish graphs that the 1-WL test cannot separate. Graphormer's spatial encoding via shortest-path distances allows it to break this limit.
• Centrality encoding: Graphormer injects node centrality measures, which the WL test ignores, providing additional discriminative signals.
• Theoretical grounding: Understanding WL is critical for evaluating why Graphormer can outperform conventional GNNs on graph classification benchmarks.

A general framework for graph neural networks where node representations are iteratively updated by aggregating information from neighboring nodes via message and update functions.

• Local neighborhood aggregation: The core operation that Graphormer replaces with a global attention mechanism over all node pairs.
• Permutation invariance: MPNNs are naturally invariant to node ordering, a property Graphormer must explicitly enforce through its structural encodings.
• Scalability trade-off: MPNNs scale linearly with edges, while Graphormer's full attention scales quadratically, motivating hybrid approaches for large molecular graphs.

A property of a function or neural network ensuring that its output transforms consistently with any input rotation and translation in 3D Euclidean space, critical for modeling molecular geometries.

• Geometric priors: Unlike Graphormer, which operates on 2D topological graphs, SE(3)-equivariant models like Equiformer directly process 3D atomic coordinates.
• Tensor product operations: Used by equivariant architectures to maintain rotational consistency, a complexity Graphormer avoids by using invariant spatial encodings.
• Complementary strengths: Graphormer excels at 2D molecular graph tasks, while equivariant models are essential for predicting energy and forces from 3D conformations.

A transformer architecture that integrates SE(3)/E(3) equivariance using tensor products and equivariant attention, achieving state-of-the-art performance on 3D molecular property prediction.

• Direct descendant: Equiformer extends Graphormer's core idea—applying transformers to molecular data—into the 3D geometric domain.
• Equivariant attention: Replaces standard dot-product attention with operations on irreducible representations to preserve rotational symmetry.
• Benchmark leader: On the OC20 dataset for catalyst discovery, Equiformer demonstrates the power of combining transformer architectures with physical symmetry constraints.

A fixed-length bit vector encoding the presence or absence of specific substructural features within a molecule, used as a standardized input representation for machine learning models.

• Pre-GNN era: Before Graphormer and GNNs, fingerprints like ECFP4 and MACCS keys were the dominant molecular descriptors for QSAR modeling.
• Learned vs. engineered: Graphormer learns task-specific molecular representations end-to-end, often outperforming fixed fingerprints on property prediction benchmarks.
• Interpretability trade-off: Fingerprints offer explicit substructure interpretability, while Graphormer's learned embeddings require post-hoc explanation tools like GNNExplainer.