Contact Prediction

The foundational principle underlying contact prediction is the detection of correlated mutations across a multiple sequence alignment (MSA). When two residues are in close spatial contact, a mutation at one position often necessitates a compensatory mutation at the other to maintain structural stability. Transformer models learn to identify these epistatic couplings by analyzing the statistical dependencies between columns in an MSA, effectively disentangling direct physical contacts from transitive correlations caused by phylogenetic relationships. This is a core capability that emerged in models like AlphaFold's Evoformer block.

In protein language models, individual attention heads spontaneously specialize to perform contact prediction without explicit structural supervision. Analysis of attention weight patterns reveals that specific heads learn to attend to residues that are spatially proximal in the folded three-dimensional structure. These heads act as learned structural probes, and their attention maps can be directly interpreted as contact probability matrices. This emergent behavior demonstrates that the grammar of protein folding is implicitly encoded in the statistical patterns of evolutionary sequence data.

The output of a contact prediction system is a contact map, a symmetric L×L matrix where L is the sequence length. Each entry (i, j) represents the predicted probability that the Cβ atoms of residues i and j are within a distance threshold, typically 8 Ångströms. This two-dimensional representation distills the complex three-dimensional folding problem into a binary classification task for every residue pair. The contact map serves as a critical intermediate representation that can be converted into distance histograms and subsequently used as restraints for downstream 3D structure reconstruction algorithms.

Contact prediction is a binary classification task—residues are either in contact or not—while distance prediction is a regression task that estimates the precise physical distance between residue pairs. Modern systems like AlphaFold2 advanced beyond binary contacts to predict distance distributions using a distogram, which bins distances into fine-grained intervals. This richer signal provides more geometric constraints for structure modeling. The evolution from contact to distance prediction marked a significant leap in de novo structure prediction accuracy, enabling end-to-end differentiable folding pipelines.

Contact prediction performance is rigorously evaluated using metrics computed on the top-ranked predictions. Key benchmarks include:

• Precision@L: The fraction of correctly predicted contacts among the top L predictions, where L is the protein length
• Long-range contacts: Residue pairs separated by at least 24 positions in the sequence, which are the most informative for tertiary structure determination
• Mean Reciprocal Rank (MRR): Evaluates how highly the correct contact is ranked State-of-the-art models achieve precision values exceeding 70% on challenging de novo targets.

Contact prediction serves as the critical bridge between sequence analysis and ab initio tertiary structure determination. Predicted contacts function as spatial restraints that guide conformational sampling algorithms, dramatically reducing the search space for protein folding simulations. In deep learning pipelines like AlphaFold, the contact prediction module within the Evoformer iteratively refines pairwise representations that encode residue-residue relationships. These refined representations are then passed to the Structure Module, which directly predicts atomic coordinates in an end-to-end differentiable manner, eliminating the need for traditional folding simulations.

A transformer-based architecture trained on massive databases of protein amino acid sequences using self-supervised objectives. These models learn the underlying grammar of protein structure and function, with contact prediction emerging as a property of their attention heads.

• Pre-training datasets often contain hundreds of millions of sequences
• Models like ESM-2 and ProtTrans learn evolutionary and structural constraints
• Contact maps can be extracted from attention weight patterns without explicit structural supervision

The core computational operation in transformer architectures that computes a weighted representation of every position in a sequence by dynamically assessing the relevance of all other positions. For proteins, this mechanism naturally learns to attend to spatially proximal residues.

• Query, key, and value projections enable pairwise interaction scoring
• Long-range dependencies between distal sequence positions are captured
• Raw attention weights correlate with physical contact probability in 3D space

A visualization of the self-attention weights extracted from a transformer model's attention heads, used as an interpretability tool to identify which specific amino acids the model focuses on when making predictions.

• Each cell in the heatmap represents the attention score between residue pair (i, j)
• High-intensity regions often correspond to physical contacts in the folded structure
• Multi-head attention produces multiple heatmaps, each capturing different structural relationships

A self-supervised pre-training objective where a random subset of input amino acids is masked and the model learns to predict the original residues from the surrounding sequence context. This forces the model to learn structural and evolutionary constraints.

• Typically 15% of residues are masked during training
• The model must infer missing amino acids from context, implicitly learning which positions co-vary
• This co-variation signal is the foundation for emergent contact prediction

The computational task of predicting a protein's three-dimensional folded structure from its amino acid sequence alone, without relying on homologous template structures. Predicted contacts serve as critical distance restraints that guide folding simulations.

• Systems like AlphaFold2 integrate contact predictions with geometric reasoning
• Contact maps constrain the conformational search space dramatically
• Accuracy of predicted contacts directly determines the quality of the final 3D model