Geometric Deep Learning

A fundamental design principle ensuring that a model's predictions transform consistently when the input 3D structure is rotated or translated. SE(3) Transformers process atomic coordinates such that rotating a protein in space produces an identically rotated output prediction. This is achieved by constraining network operations to the irreducible representations of the special Euclidean group SE(3), which captures all rigid-body motions in three dimensions.

• Equivariance means f(Rx) = R f(x) for any rotation R
• Invariance is a special case where the output remains unchanged under transformations
• Critical for tasks like force field prediction where directional quantities must rotate with the molecule

A class of architectures that represent biomolecules as graphs where nodes are atoms and edges represent spatial proximity or chemical bonds. Unlike standard GNNs, equivariant versions propagate both scalar features (atom types, charges) and vector features (positions, velocities) through the network while preserving their geometric transformation properties.

• Message passing updates node representations based on neighboring atoms within a cutoff radius
• Tensor field networks use spherical harmonics to capture directional information
• Enables learning of complex many-body interactions without data augmentation by random rotations

A foundational architecture that builds learnable, locally equivariant filters operating on continuous 3D point clouds. Each layer maps feature fields of one tensor order to another, using spherical harmonics and Clebsch-Gordan tensor products to couple geometric information across different rotational symmetries.

• Tensor order 0 = scalar (invariant), order 1 = vector (equivariant), order 2 = higher-rank tensors
• Point convolutions are parameterized by learnable radial functions and fixed angular basis functions
• Forms the theoretical backbone for many modern equivariant architectures including SE(3) Transformers

An extension of standard self-attention that operates on geometric features while maintaining SE(3) equivariance. Attention weights are computed using invariant features (distances, angles), while value messages carry directional information that transforms correctly under rotation.

• Query and key vectors are constructed from invariant scalar features to ensure rotation-invariant attention weights
• Value vectors incorporate tensor products of directional information with learned filters
• Enables the model to dynamically focus on geometrically relevant atomic neighborhoods for each prediction

Geometric deep learning powers state-of-the-art structure prediction by iteratively refining atomic coordinates. Models learn a pairwise representation encoding residue-residue relationships and a single representation for each residue's local geometry, then use equivariant layers to predict and update 3D positions.

• AlphaFold2 uses invariant point attention (IPA) to reason about 3D geometry without explicit equivariance constraints
• RosettaFold employs a three-track architecture processing 1D sequence, 2D distance, and 3D coordinate information simultaneously
• Recycling iteratively refines predictions by feeding outputs back as inputs

The task of designing an amino acid sequence that will fold into a specified 3D backbone structure—the reverse of structure prediction. Geometric deep learning models process the target backbone coordinates with equivariant encoders to capture local and non-local geometric features, then decode these into sequence probabilities.

• ProteinMPNN uses an equivariant message-passing framework to achieve state-of-the-art sequence recovery rates
• Models learn to satisfy geometric constraints like hydrogen bonding patterns and hydrophobic packing
• Enables de novo design of proteins with novel functions not found in nature

A mathematical property where applying a transformation (e.g., rotation) to the input produces an equivalent transformation on the output. In geometric deep learning, SE(3)-equivariant layers ensure that rotating a protein structure rotates the model's predictions identically, eliminating the need for data augmentation and guaranteeing physically consistent outputs. This contrasts with invariance, where the output remains unchanged under input transformations.

A neural architecture that operates directly on graph-structured data, where atoms are represented as nodes and chemical bonds or spatial proximity as edges. GNNs learn node, edge, and global representations through message passing: each node aggregates feature information from its neighbors, enabling the model to capture local chemical environments and long-range interactions in molecular structures.

An extension of the transformer architecture that builds equivariance to 3D rotations and translations directly into the attention mechanism. By using tensor field representations and Clebsch-Gordan tensor products, the SE(3) Transformer processes atomic coordinates while respecting physical symmetries, making it particularly effective for protein structure prediction and molecular dynamics tasks where orientation matters.

A framework that combines graph neural network message passing with group equivariance constraints. During each layer, messages between atoms are computed using functions that are equivariant to the Euclidean group E(3) or its subgroups. This ensures that learned representations transform correctly under rotation, reflection, and translation, enabling accurate prediction of vector-valued properties like atomic forces.

A locally equivariant 3D point cloud architecture that represents features as geometric tensors living on atomic positions. Each layer uses learnable radial filters and spherical harmonic-based filters to construct messages that preserve SO(3) equivariance. Tensor Field Networks form the theoretical foundation for many subsequent equivariant models used in quantum chemistry and structural biology.

The core geometric reasoning module in AlphaFold2 that operates on protein backbone frames. IPA augments standard attention by incorporating 3D spatial distances and relative coordinate frames directly into the attention computation, ensuring that the output is invariant to global rotations and translations while allowing the model to reason about local geometry for structure refinement.