Equivariant Neural Network

Equivariance means that applying a symmetry operation (like a rotation) to the input produces a predictable, corresponding transformation in the output. For a force vector, rotating the molecule rotates the predicted force vector identically. Invariance, by contrast, means the output remains unchanged under transformation—essential for scalar properties like energy. An equivariant network internally maintains geometric tensor representations to guarantee this behavior by construction, not by hoping the model learns it from data.

Standard neural networks process scalar features. Equivariant networks operate on geometric tensors of varying orders:

• Scalars (order-0): Invariant under rotation (e.g., atomic energy contributions)
• Vectors (order-1): Rotate like 3D coordinates (e.g., atomic forces)
• Higher-order tensors: Capture complex angular dependencies (e.g., quadrupole moments) Layers transform these tensors while preserving their rotational properties, using Clebsch-Gordan tensor products to combine them correctly.

To encode directional information, atomic neighborhoods are projected onto spherical harmonics Y_l^m—a set of orthogonal basis functions on the sphere analogous to Fourier series for angular coordinates. This provides a complete, systematic representation of the local geometry around each atom. The order l controls angular resolution: l=0 captures radial density, l=1 captures dipole-like patterns, and higher l captures increasingly fine orientational detail.

Equivariant architectures typically operate on a radius-cutoff graph where nodes are atoms and edges connect neighbors within a cutoff distance. Messages between nodes carry tensor information:

• Node features are updated by aggregating messages from neighbors
• Edge features encode interatomic distance and direction vectors
• Convolution filters are learned functions of distance, making them rotationally invariant The combination of graph message passing with tensor algebra ensures both permutation and SE(3) equivariance.

The Special Euclidean group SE(3) encompasses all rigid-body motions in 3D: rotations (SO(3)) and translations (R^3). Equivariant networks are designed to be equivariant under the full SE(3) group. Translation invariance is trivially achieved by using relative position vectors. Rotation equivariance is enforced through the mathematical structure of tensor products and spherical harmonic representations, ensuring predictions are physically consistent regardless of molecular orientation.

By hard-coding physical symmetries into the architecture, equivariant networks achieve dramatic data efficiency. A standard network must learn rotational invariance from augmented training data—seeing the same molecule in thousands of orientations. An equivariant network knows this a priori. This reduces training data requirements by orders of magnitude and improves generalization to unseen molecular configurations, making high-accuracy training feasible with expensive quantum mechanical reference data.

The mathematical foundation of equivariance. A symmetry group is a set of transformations—such as the SE(3) group of 3D rotations and translations—that leave a physical system's properties invariant. Equivariant networks constrain their internal representations to transform according to irreducible representations of these groups, ensuring that rotating the input molecule rotates the predicted forces identically.

The core mathematical toolkit for building equivariant layers. Spherical harmonics are angular functions that serve as the basis for representing directional information in 3D space. Equivariant architectures use tensor product operations to combine these representations while preserving their transformation properties under rotation, allowing the network to reason about geometric relationships between atoms.

Equivariant networks typically operate on geometric graphs where nodes represent atoms and edges carry both scalar distances and directional vectors. During message passing, information flows between nodes through equivariant convolutions that update node features using neighboring atom information. Unlike standard graph neural networks, these messages carry directional information encoded in spherical harmonic coefficients.

A critical distinction in geometric deep learning:

• Invariance: The output remains completely unchanged under a symmetry operation. Example: the total energy of a molecule is rotationally invariant.
• Equivariance: The output transforms predictably with the input. Example: atomic forces rotate with the molecule. Equivariant networks can produce both types of outputs by using appropriate final aggregation layers.

The Clebsch-Gordan decomposition provides the mathematical rules for combining two angular momentum representations into a direct sum of new representations. In equivariant networks, this operation governs how feature vectors of different rotational orders interact. A tensor product between features of order l₁ and l₂ produces output features spanning orders from |l₁-l₂| to l₁+l₂, preserving rotational equivariance throughout the network.

By hard-coding physical symmetries into the architecture, equivariant networks achieve dramatically better data efficiency than conventional models. A standard neural network must learn rotational invariance from augmented training data, while an equivariant network has this property built into its mathematical structure. This inductive bias reduces training data requirements by orders of magnitude and improves generalization to unseen molecular configurations.