CASP (Critical Assessment of Structure Prediction)

CASP's defining feature is its blind prediction format. Organizers solicit sequences of proteins whose experimental structures have been recently solved but are not yet publicly released in the Protein Data Bank (PDB). Participating research groups then submit their computational predictions without any knowledge of the true answer. This eliminates the possibility of overfitting or cherry-picking results, providing an unvarnished assessment of method performance. The prediction season typically spans three months, after which the experimental structures are revealed and scoring begins.

CASP uses a suite of rigorous, automated metrics to numerically score predictions against the unreleased experimental structure. The primary metric is the Global Distance Test Total Score (GDT_TS), which measures the percentage of C-alpha atoms that can be superimposed under multiple distance cutoffs, capturing global topology. Other critical metrics include:

• RMSD (Root Mean Square Deviation): Average atomic distance after optimal superposition.
• LDDT (Local Distance Difference Test): A superposition-free score evaluating local environment correctness.
• CAD (Contact Area Difference): Assesses the accuracy of residue-residue contact maps.

While initially focused on predicting the 3D structure of single protein domains, CASP has continuously evolved its scope to push the boundaries of the field. Recent and ongoing experimental categories include:

• Quaternary Structure Prediction: Assessing the ability to predict the 3D assembly of multi-protein complexes.
• Ligand Binding Site Prediction: Evaluating the identification of small molecule interaction sites.
• RNA Structure Prediction: Extending the challenge to nucleic acid folding.
• Data-Assisted Modeling: Testing how well methods can integrate sparse experimental data (e.g., from Cryo-EM or crosslinking mass spectrometry) to improve predictions.

The primary scoring metric in CASP for assessing global topology. GDT_TS measures the similarity between a predicted model and the experimental structure by calculating the largest set of residues that can be superimposed under multiple distance thresholds (1, 2, 4, and 8 Å).

• Higher is better: Scores range from 0 to 100, with 100 indicating perfect agreement.
• Robustness: Less sensitive to local structural outliers than RMSD, making it ideal for ranking methods.
• Historical significance: AlphaFold2 achieved a median GDT_TS of 92.4 across all targets in CASP14, effectively solving the single-domain protein folding problem.

A per-residue confidence metric output by AlphaFold that estimates the local accuracy of the predicted structure on a scale from 0 to 100. It is a self-assessment of how well the prediction would agree with a local superposition of the experimental structure.

• Interpretation: Residues with pLDDT > 90 are modeled with high accuracy and are suitable for detailed structural analysis.
• Low-confidence regions: pLDDT < 50 often indicates intrinsically disordered regions or domains where the prediction is unreliable.
• Practical use: Researchers should always mask or cautiously interpret low-pLDDT regions in downstream applications like drug docking.

A 2D confidence metric that estimates the expected positional error between any two residues in a predicted structure. Unlike pLDDT, which is local, PAE assesses global domain packing and relative orientation confidence.

• Domain orientation: Low PAE values between two blocks of residues indicate confident relative positioning of domains.
• Visualization: PAE plots are essential for identifying rigid-body domain movements and assessing quaternary structure predictions.
• CASP relevance: PAE provides a more nuanced view of model quality than a single global score, revealing where a model is confident about local geometry but uncertain about domain arrangement.

A standard superposition metric measuring the average distance between corresponding atoms (typically Cα) in two aligned protein structures. It is the most widely reported but also most misunderstood metric in structural biology.

• Calculation: RMSD = sqrt(1/N * Σ d_i²), where d_i is the distance between atom i in the model and the reference.
• Limitations: Highly sensitive to a single poorly modeled loop or flexible region; a single outlier can dominate the score.
• CASP context: While still reported, CASP has largely shifted to GDT_TS and local distance difference tests because RMSD fails to capture the quality of partially correct models.

A prediction category in CASP for targets where a detectable homologous structure exists in the Protein Data Bank (PDB). Methods use the known experimental structure as a template to guide model construction.

• Historical dominance: Before deep learning, TBM was the most reliable prediction method, relying on sequence alignment and homology transfer.
• AlphaFold's impact: Modern methods like AlphaFold2 have blurred the line between TBM and free modeling by using co-evolutionary information from MSAs, often outperforming traditional template-based approaches even when templates are available.
• CASP classification: Targets are retrospectively classified as TBM or FM (Free Modeling) based on whether a structural template could have been identified.

The hardest prediction category in CASP, reserved for targets with no detectable structural homologs in the PDB. Methods must predict the 3D structure solely from the amino acid sequence and physicochemical principles.

• CASP14 breakthrough: AlphaFold2 achieved GDT_TS scores above 80 for many FM targets, a feat previously considered decades away.
• Methodology: Modern FM approaches rely on deep learning applied to multiple sequence alignments, extracting coevolutionary signals rather than using explicit physics-based simulations.
• Remaining challenges: De novo designed proteins and orphan sequences with very shallow MSAs still pose significant difficulties for FM methods.