Equivariant Neural Network

The network's predictions transform predictably under the Special Euclidean group SE(3)—all 3D rotations and translations. If you rotate a protein's input coordinates, the predicted structure rotates identically. This is not learned; it is mathematically guaranteed by the architecture. Unlike data augmentation, which only approximates symmetry, SE(3) equivariance eliminates an entire class of prediction errors where models output physically impossible orientations.

A core mechanism from AlphaFold2 that performs attention over 3D spatial relationships while maintaining invariance to global rotation and translation. IPA computes attention weights based on pairwise Euclidean distances and relative spatial orientations between residues, allowing the network to reason about local geometry without being confused by the protein's overall position in space. This enables the model to iteratively refine pairwise residue relationships.

Internal features are structured as geometric tensors with well-defined transformation properties under rotation. Key types include:

• Scalars (rank-0): Rotation-invariant features like atom types or charges
• Vectors (rank-1): Directional features that rotate like 3D coordinates
• Higher-order tensors: Capture complex angular dependencies This hierarchical representation allows the network to compose geometric relationships while preserving physical consistency at every layer.

Equivariant networks decompose features into irreducible representations (irreps) of the rotation group SO(3), expressed using spherical harmonics. Each irrep corresponds to a specific angular momentum (l=0,1,2,...) and transforms independently under rotation. The Clebsch-Gordan tensor product combines irreps to produce new features with predictable transformation properties, enabling the network to build complex geometric reasoning from mathematically sound primitives.

Atoms or residues are represented as nodes in a geometric graph, with edges encoding spatial proximity. Messages between nodes depend on:

• Interatomic distances (radial basis functions)
• Relative orientations (spherical harmonics of bond angles)
• Node features (atom types, residue identities) This graph structure naturally respects the locality of physical interactions, ensuring that distant atoms do not arbitrarily influence each other and that predictions scale efficiently with system size.

An alternative approach to achieving equivariance without explicit tensor algebra. The network averages predictions over a group of transformations or first canonicalizes the input into a standard orientation. For proteins, this might involve aligning to a reference frame defined by local backbone geometry. While computationally simpler than full tensor field networks, frame averaging can approximate equivariance with minimal architectural changes, making it attractive for rapid prototyping and deployment.

The mathematical property ensuring a model's output transforms identically to its input under any 3D rotation or translation. For protein structures, this means if you rotate the input coordinates, the predicted coordinates rotate identically. SE(3) is the Special Euclidean group in 3D, combining rotations (SO(3)) and translations (R^3). This guarantees predictions are physically consistent and independent of the arbitrary coordinate frame.

A foundational architecture for building SE(3)-equivariant point cloud networks. Layers operate on fields of geometric tensors (scalars, vectors, higher-order tensors) living at 3D point coordinates. Convolutions use learned radial filters and spherical harmonics to mix features while strictly preserving rotation and translation equivariance. This provides a continuous, learnable mapping between 3D geometry and feature representations.

The fundamental building blocks for constructing equivariant features. Features are decomposed into irreps (irreducible representations) of the rotation group SO(3), labeled by rotation order l (0=scalar, 1=vector, 2=matrix, etc.). Equivariant networks process these typed features using Clebsch-Gordan tensor products, which combine two irreps to produce new ones while strictly preserving rotational transformation laws.

A family of orthogonal functions defined on the surface of a sphere, serving as the angular basis functions for equivariant convolutions. They encode directional information in a way that transforms predictably under rotation. In architectures like SE(3)-Transformers, spherical harmonics are used to embed relative position vectors between atoms, enabling the network to reason about 3D geometry without losing directional information.

The core mechanism in AlphaFold2 that achieves SE(3) equivariance without explicit tensor products. IPA augments standard attention with 3D spatial proximity biases derived from the current coordinate frame. It operates on local frames defined by each residue's predicted orientation, making the attention weights invariant to global rotation while the coordinate updates remain equivariant. This is a pragmatic, highly scalable alternative to formal group-equivariant convolutions.

A lightweight equivariant layer that processes scalar and vector features jointly. A GVP transforms a tuple of (scalar features, vector features) into a new tuple, applying learned linear and nonlinear operations. Vector features are rotated by learned weight matrices, while scalar features gate and modulate them. This provides a simple, efficient building block for SE(3)-equivariant graph neural networks on protein backbones.