Equivariant Neural Network

A function f is equivariant if applying a transformation (e.g., rotation) to the input x and then applying f is the same as applying f first and then the transformation: f(g·x) = g·f(x). This preserves geometric information. Invariance is a special case where the output is unchanged: f(g·x) = f(x). For predicting a molecule's energy (invariant) versus its dipole moment (equivariant), the architecture must be designed accordingly.

A foundational architecture that builds SE(3)-equivariant features by operating on fields of geometric tensors. Key characteristics include:

• Point Convolution: Weights are learned functions of relative position vectors, not absolute coordinates.
• Clebsch-Gordan Tensor Products: The core operation that combines two feature tensors to produce a new one, preserving rotation order.
• This allows the network to propagate directional information (like forces) through layers without being destroyed by arbitrary rotations of the input.

The mathematical backbone of 3D equivariant networks. Spherical harmonics (Y^l_m) are functions on the sphere that serve as a basis for representing rotationally equivariant features. Features are typed by their rotation order l (scalar=0, vector=1, tensor=2). An irreducible representation (irrep) is a feature space that transforms in a well-defined way under rotation, and all equivariant features are built from them.

Equivariant networks typically operate on a graph where nodes are atoms and edges represent spatial proximity. A message function computes an interaction vector between neighbors using their relative positions and feature tensors. Crucially, the message is constructed using rotationally equivariant operations (like tensor products of spherical harmonics), ensuring the aggregated update to a node's feature tensor is itself a valid geometric tensor.

A more general framework where feature fields can have an orientation defined in a local frame (a gauge). Steerable CNNs guarantee equivariance to transformations of both the base space and this local gauge. This is critical for processing data on curved surfaces or when features have a directional sense (like a vector field on a sphere) that standard 3D rotations don't fully capture.

A streamlined architecture achieving E(n) equivariance—equivariance to rotations, translations, and reflections. It simplifies the tensor product machinery by operating on scalar and vector features. A node i's vector feature is updated by a weighted sum of relative position vectors. This architecture, used in models like EGNN, is computationally efficient while still guaranteeing that output coordinates transform correctly with the input.

Equivariant networks like DeepEMhancer's convolutional architecture implicitly respect local rotational symmetries when performing map sharpening and local amplitude scaling. By guaranteeing that a rotated density map produces a consistently sharpened output, these models avoid introducing directional artifacts. This is critical for interpreting atomic features in regions of anisotropic resolution, where traditional Fourier-based sharpening can amplify noise along preferred orientations.

ModelAngelo employs a graph neural network with equivariant message-passing layers to trace the protein backbone directly into cryo-EM density maps. The network's SE(3)-equivariance ensures that:

• Predicted Cα atom positions transform rigidly with the input map
• Amino acid sequence assignment is invariant to the map's global orientation
• The model generalizes across maps solved in different coordinate frames without data augmentation

Tools like CryoDRGN and 3D Variability Analysis (3DVA) model continuous molecular motions from cryo-EM particle images. Equivariant architectures enable these methods to learn a latent space of conformational states where:

• Rigid-body domain rotations are decoupled from internal deformations
• The learned representation is invariant to the particle's arbitrary in-plane pose
• Transitions along latent coordinates correspond to physically plausible, symmetry-respecting motion trajectories

MDFF uses molecular dynamics simulation to flexibly fit atomic models into cryo-EM density maps by applying forces derived from the map's potential. Equivariant neural network potentials can replace classical force fields in this workflow, providing:

• Quantum-mechanical accuracy at near-classical computational cost
• Guaranteed energy conservation under rigid-body rotation of the entire system
• Consistent treatment of non-bonded interactions regardless of molecular orientation within the density map

In cryo-electron tomography (cryo-ET), subtomogram averaging aligns and averages 3D sub-volumes to achieve high-resolution structures in situ. Equivariant networks address two key challenges:

• Missing wedge correction: The network's SO(3)-equivariance allows it to infer missing Fourier space information by learning the consistent 3D shape manifold from partial observations
• Template-free alignment: Rotation-equivariant features enable simultaneous pose estimation and structure refinement without requiring an initial reference model, reducing model bias

Denoising autoencoders trained with Noise2Noise principles leverage equivariant architectures to restore cryo-EM micrographs and tomograms. Key advantages include:

• Rotation-consistent denoising: A rotated noisy input yields an identically rotated clean output, preventing the introduction of spurious directional features
• Frame alignment integration: Equivariant features can simultaneously correct for beam-induced motion while denoising, treating translation as a continuous symmetry group
• Improved CTF correction by learning to invert the contrast transfer function as an equivariant deconvolution operation

The mathematical foundation of equivariant networks. A group defines a set of transformations (e.g., rotations, translations) under which a system's behavior is predictable. SO(3) describes all 3D rotations, while SE(3) adds translations. Equivariant networks explicitly encode these group actions into their layers, guaranteeing that rotating the input produces an equivalently rotated output—a property standard CNNs must learn from data augmentation.

A pioneering architecture that builds SE(3)-equivariant point cloud networks using spherical harmonics and tensor products. Key characteristics:

• Neurons are geometric tensors that rotate according to irreducible representations
• Convolution filters are learned functions of distance, restricted to preserve rotation equivariance
• Outputs are vector fields (e.g., forces, velocities) that rotate correctly with the molecule
• Used extensively in molecular force fields and protein structure refinement

A family of orthogonal functions on the sphere that serve as the angular basis functions in equivariant networks. They decompose 3D signals into frequency components analogous to Fourier series on a circle. In practice:

• l=0 captures scalar (rotation-invariant) information like charge
• l=1 captures vector (dipole-like) information
• l=2 captures quadrupolar patterns
• Higher l values capture increasingly fine angular detail, at increased computational cost

The mathematical operation that combines two geometric tensors of different angular momenta to produce new tensors with defined transformation properties. This is the core equivariant operation in architectures like TFN and SE(3)-Transformers. It ensures that when two features interact (e.g., atom pairs), the resulting feature transforms correctly under rotation. The operation is parameterized by Clebsch-Gordan coefficients from quantum mechanics.

An extension of the Transformer architecture to 3D point clouds that maintains SE(3)-equivariance. Key innovations:

• Attention weights are invariant to rotation (computed from distances)
• Value vectors are geometric tensors that transform equivariantly
• Self-attention operates on tensor products of keys, queries, and values
• Achieves state-of-the-art on molecular property prediction and protein structure tasks by attending over atomic neighborhoods while respecting physical symmetries

The framework used in equivariant graph neural networks (GNNs) for molecular systems. Messages between atoms are constructed using equivariant features (scalars, vectors, tensors) rather than just scalar distances. This allows the network to reason about directional information like bond angles and dihedral angles natively. Architectures like NequIP and MACE use this to achieve quantum-level accuracy in interatomic potential prediction with orders of magnitude less data than classical force fields.