IPA (Invariant Point Attention)

IPA ensures that predictions transform predictably under global rotation and translation of the input structure. This is achieved by operating on invariant spatial features—pairwise distances and relative displacement vectors expressed in local residue frames—rather than absolute coordinates. If you rotate the entire protein, the attention weights and output features remain identical, while coordinate updates rotate accordingly. This property eliminates the need for data augmentation over rigid-body transformations and dramatically improves training efficiency.

Each residue defines its own local coordinate frame using the geometry of its backbone atoms (N, Cα, C). IPA computes attention between residue pairs by expressing the 3D displacement vector from residue i to residue j in the local frame of residue i. This representation is invariant to global orientation because only relative spatial relationships matter. The frame construction uses Gram-Schmidt orthogonalization to ensure a valid right-handed coordinate system at every residue position.

IPA extends standard multi-head attention by incorporating 3D point clouds as additional inputs. Each residue is represented by a set of learned invariant points—virtual 3D locations predicted from the residue's current representation. Attention weights between residues are modulated by the squared Euclidean distances between their corresponding invariant points:

• Query, key, and value projections operate on both sequence features and 3D point coordinates
• Distance-based bias terms are added to the attention logits
• The mechanism naturally attends to spatially proximal residues regardless of sequence distance

IPA produces residual updates to the 3D coordinates of each residue's backbone frame at every layer of the structure module. These updates are computed as weighted sums of relative displacement vectors to neighboring residues, where the weights come from the attention mechanism. The refinement is applied iteratively through 8 layers with weight sharing (recycling), allowing the model to progressively improve its structural predictions. Each layer sees the updated coordinates from the previous layer, creating a feedback loop that converges to accurate atomic positions.

IPA enriches pairwise representations by incorporating geometric edge features derived from the relative positions of residues in 3D space. These features include:

• Inter-residue distances between backbone Cα atoms
• Relative displacement vectors decomposed into local frame components
• Distance bins encoded via radial basis functions
• Backbone dihedral angles between residue pairs

These edge features are projected and added to the standard sequence-based pair representation, allowing the model to jointly reason about evolutionary and spatial constraints.

IPA achieves O(N²) complexity with respect to sequence length N, matching standard transformer attention. The key efficiency innovations include:

• Single-layer coordinate updates: Unlike iterative physics-based refinement, IPA predicts coordinate updates in a single forward pass per layer
• Shared invariant points: The same set of learned points is used across all attention heads
• No explicit pairwise distance matrix: Distances are computed on-the-fly from invariant point coordinates
• Memory-efficient implementation: Attention over 3D points uses optimized CUDA kernels for batched distance computation

The loss function used to train IPA that measures the distance between predicted and true atom positions after aligning local reference frames. Unlike global RMSD, FAPE is computed in each residue's local coordinate system, making it invariant to global rotation and translation. This directly complements IPA's invariance properties and penalizes local structural errors more effectively than global metrics.

The mathematical foundation underlying IPA's invariance properties. SE(3) is the Special Euclidean group representing all rigid-body transformations (rotations and translations) in 3D space. IPA operates on residue gas frames—local coordinate systems defined by each residue's backbone geometry—and computes attention in a way that is invariant to global SE(3) transformations while preserving relative spatial relationships.

An iterative refinement strategy where the Structure Module's output coordinates are fed back as input for multiple passes (typically 3-4). Each recycling iteration allows IPA to:

• Resolve initial prediction errors
• Refine side-chain packing
• Improve domain-domain orientations The IPA layers benefit from seeing progressively better initial coordinates, converging to higher accuracy predictions.

The per-residue confidence score output by AlphaFold2 that correlates strongly with IPA's prediction quality. pLDDT (predicted Local Distance Difference Test) ranges from 0-100 and is trained to predict the local accuracy of each residue. Regions with high pLDDT (>90) indicate IPA has converged to a high-confidence local structure, while low pLDDT (<50) often signals disordered regions or prediction uncertainty.