Geometric Deep Learning

The defining property of geometric architectures. An equivariant layer guarantees that if the input atomic coordinates are rotated by a matrix R, the output features transform predictably by the same rotation. This is achieved through tensor field networks and spherical harmonics, ensuring the model's predictions are physically consistent regardless of the molecule's orientation in space. Without equivariance, a network would need to see every possible rotation during training—a computationally infeasible data augmentation task.

An extension of the standard transformer architecture that operates on 3D point clouds while maintaining roto-translational equivariance. Key innovations include:

• Self-attention with geometric kernels: Attention weights depend on both feature similarity and relative spatial positions
• Clebsch-Gordan tensor products: Used to combine higher-order geometric features without breaking symmetry
• Continuous convolution on groups: Enables message passing between atoms that respects the SE(3) symmetry group

These models form the backbone of state-of-the-art protein and RNA structure prediction systems.

Molecules are naturally represented as graphs where nodes are atoms and edges are covalent bonds or spatial proximity. GNNs learn through message passing: each atom iteratively aggregates information from its neighbors. For 3D structures, SchNet and DimeNet incorporate interatomic distances and angles into edge features. Equivariant GNNs like PaiNN and SEGNN extend this by ensuring that node features transform correctly under rotation, enabling accurate prediction of directional properties like atomic forces and dipole moments.

A critical architectural distinction:

• Invariant features: Do not change under rotation (e.g., interatomic distances, bond angles). Used when the output is a scalar property like binding energy or solubility.
• Equivariant features: Rotate with the input (e.g., vector features representing atomic forces). Required when predicting directional quantities or 3D coordinates.

Modern architectures like Equiformer and Allegro maintain both types of features simultaneously, using invariant representations for energy prediction and equivariant ones for force fields.

The mathematical engine of equivariant networks. A tensor product combines two geometric features of different orders (scalars, vectors, higher-order tensors) using Clebsch-Gordan coefficients to produce a new feature with well-defined transformation properties. This operation:

• Preserves the irreducible representations of the SO(3) rotation group
• Enables the construction of higher-order geometric features beyond simple distances
• Is computationally expensive, driving research into efficient approximations like e3nn and TFN implementations

Tensor products allow networks to capture complex angular relationships between atoms.

A pragmatic alternative to strict equivariance. Frame averaging computes predictions in multiple canonical coordinate frames and averages the results, guaranteeing exact invariance or equivariance without complex tensor algebra. The method:

• Stochastic Frame Averaging: Randomly samples frames during training for efficiency
• Deterministic averaging: Uses a fixed set of frames defined by the molecule's principal axes
• Enables the use of any standard neural network as a backbone while still respecting symmetries

This approach is particularly useful for rapid prototyping and when strict equivariance is computationally prohibitive.

A mathematical property where applying a transformation (e.g., rotation) to the input produces an equivalent transformation on the output. In geometric deep learning, equivariant layers ensure that rotating a molecule's coordinates rotates the predicted forces or properties identically, eliminating the need for data augmentation. This is distinct from invariance, where the output remains unchanged under transformation.

A class of neural networks that operate directly on graph-structured data, where nodes (atoms) are connected by edges (bonds or spatial proximity). GNNs learn node, edge, and global representations through iterative message passing: each node aggregates feature vectors from its neighbors and updates its own state. Variants include GCN, GAT, and MPNN, forming the backbone of most molecular deep learning architectures.

The fundamental computational mechanism in GNNs where information flows between connected nodes. In each layer, every node:

• Collects messages from neighboring nodes
• Aggregates them via a permutation-invariant function (sum, mean, or max)
• Updates its own feature vector using the aggregated message This framework naturally captures local chemical environments and long-range interactions through stacking multiple layers.

A pioneering architecture that builds SE(3)-equivariant convolutional layers using spherical harmonics and Clebsch-Gordan tensor products. It represents features as geometric tensors of different orders (scalars, vectors, higher-order tensors) that transform predictably under rotation. This enables the network to reason about directional information—critical for modeling atomic forces and dipole moments—while maintaining physical consistency.

The generalization of standard convolution to symmetry groups like SO(3) or SE(3). Instead of translating a filter across a grid, group convolution transforms the filter by all elements of the symmetry group and computes inner products. This guarantees equivariance to the group action by construction. In molecular applications, this means the model's predictions are physically valid regardless of the molecule's orientation in space.