Equivariant Neural Network

The network's internal operations are constrained to respect the Special Euclidean group SE(3) — the set of all 3D rotations and translations. When the input atomic coordinates are rotated by a matrix R and translated by a vector t, the output transforms identically: f(Rx + t) = Rf(x) + t. This is achieved through architectures like tensor field networks and SE(3)-Transformers, which use spherical harmonics and Clebsch-Gordan tensor products to build features that transform predictably under group actions.

Equivariant layers decompose features into irreducible representations (irreps) of the rotation group SO(3), labeled by angular momentum ℓ = 0, 1, 2, ... (scalars, vectors, tensors). Each feature channel carries a specific type-ℓ that dictates how it rotates:

• ℓ = 0: Scalar features (invariant to rotation, e.g., atom type embeddings)
• ℓ = 1: Vector features (rotate like 3D vectors, e.g., forces)
• ℓ ≥ 2: Higher-order tensor features (capture directional bonding patterns) This structured representation prevents information loss during geometric transformations.

The core mathematical operation in equivariant networks is the tensor product between a learnable spherical harmonic filter and neighboring node features, combined via Clebsch-Gordan coefficients. This operation:

• Mixes features of types ℓ₁ and ℓ₂ to produce outputs of type ℓ₃ (where |ℓ₁ - ℓ₂| ≤ ℓ₃ ≤ ℓ₁ + ℓ₂)
• Preserves angular momentum coupling rules from quantum mechanics
• Enables message passing that respects 3D rotational symmetry
• Replaces standard scalar dot-product attention with geometrically meaningful interactions

Unlike traditional methods that require aligning structures to a canonical reference frame, equivariant networks operate directly on raw 3D coordinates without coordinate pre-processing. The network learns functions on geometric tensors rather than absolute positions, making predictions:

• Independent of the arbitrary choice of coordinate system
• Consistent across different crystallographic unit cell conventions
• Robust to global rotations introduced during molecular dynamics sampling This frame-free property is critical for RNA, where no natural canonical orientation exists.

By baking geometric priors directly into the architecture, equivariant networks achieve dramatic improvements in data efficiency compared to non-equivariant baselines:

• No rotational data augmentation required — the symmetry is exact, not learned from augmented examples
• Fewer training examples needed to generalize to unseen orientations
• Parameter sharing across all rotated versions of the same local geometry
• For RNA structure prediction, this means effective learning from the limited set of experimentally determined structures in the PDB

In graph-based architectures, messages between nodes (atoms or residues) are constructed using equivariant operations that depend on relative position vectors rᵢⱼ. The message from node j to node i is computed as:

• A learnable function of the distance ‖rᵢⱼ‖ (invariant scalar)
• Combined with spherical harmonic projections of the direction r̂ᵢⱼ (equivariant features)
• Aggregated using tensor products that preserve transformation laws This ensures that the updated node features transform correctly when the entire molecular graph is rotated, enabling the network to reason about local geometric patterns like base-pairing geometries and A-minor motifs.

A neural network paradigm designed to operate on non-Euclidean data like graphs, manifolds, and 3D point clouds. It provides a unified framework for building architectures that respect the underlying symmetries of the data domain. For RNA structure prediction, this means building models that inherently understand that a molecule's function is unchanged by its rotation or translation in space. This field encompasses equivariant networks as a specific implementation of the broader symmetry-preserving principle.

The Special Euclidean group in 3 dimensions, representing all possible rigid-body motions—rotations and translations—in 3D space. An SE(3)-equivariant network guarantees that if you rotate and translate the input atomic coordinates, the predicted output coordinates will undergo the exact same transformation. This is the core symmetry group respected by models like AlphaFold 3 and RoseTTAFoldNA, ensuring predicted RNA structures are physically valid regardless of their initial coordinate frame.

A critical distinction in geometric deep learning:

• Invariance: The output remains completely unchanged when the input is transformed. For example, a model predicting the total energy of an RNA molecule should be invariant to rotation.
• Equivariance: The output transforms predictably with the input. For example, a model predicting 3D atomic coordinates should output rotated coordinates when given rotated inputs. Equivariant networks are essential for structure prediction because the output is a set of coordinates that must move with the molecule.

A foundational SE(3)-equivariant architecture that operates on point clouds using learnable, localized filters. It constructs features as tensor fields that transform predictably under rotation, using spherical harmonics and Clebsch-Gordan tensor products to combine features while preserving their geometric transformation properties. This approach was a key precursor to the SE(3)-Transformers and the invariant point attention mechanism used in AlphaFold 2 and 3.

A set of orthogonal functions defined on the surface of a sphere, analogous to Fourier series for angular data. In equivariant neural networks, spherical harmonics are used to encode directional information between atoms in a way that rotates mathematically correctly. They decompose a 3D vector (like the direction from one atom to another) into a set of coefficients that can be combined using tensor products, ensuring that the network's internal representations rotate in lockstep with the input molecule.

The core geometric mechanism in AlphaFold 2 and 3 that achieves SE(3)-equivariance without explicitly using spherical harmonics. IPA augments standard transformer attention with 3D spatial information by projecting local coordinate frames for each residue. It computes attention weights based on both sequence similarity and Euclidean distance in 3D space, ensuring that the attention pattern is invariant to global rotation while the output coordinates are equivariant. This design was a breakthrough in computational efficiency for protein and RNA structure prediction.