Sequence Recovery Rate

Sequence recovery rate is calculated as the percentage of residues in a designed sequence that match the native amino acid at the corresponding position in the target structure. For a protein of length L, if C residues match the wild-type, the rate is (C / L) × 100.

• Per-target calculation: Computed independently for each protein in a test set, then averaged.
• Perplexity relationship: Lower perplexity on native sequences generally correlates with higher recovery.
• Baseline comparison: Random guessing achieves ~5% (1/20 amino acids); state-of-the-art models like ProteinMPNN achieve ~52-55% on native backbones.

Recovery rate is evaluated under two distinct conditions that test different model capabilities:

• Native backbone recovery: The model redesigns sequences for experimentally determined wild-type structures. High performance here indicates the model has learned fundamental sequence-structure relationships from evolutionary data.
• Hallucinated backbone recovery: The model designs sequences for computationally generated, non-natural backbone geometries. This tests generalization beyond the training distribution and is critical for true de novo design.
• Models typically show a 5-15% drop in recovery on hallucinated backbones compared to native ones.

While widely used, sequence recovery rate has important caveats that researchers must consider:

• Degeneracy of the folding code: Multiple distinct sequences can fold into identical structures. A low recovery rate does not necessarily indicate a poor design—the predicted sequence may still fold correctly.
• Functional blindness: Recovery measures structural compatibility, not whether the designed sequence retains catalytic activity, binding affinity, or other functional properties.
• Surface vs. core bias: Recovery is typically higher for buried hydrophobic residues (~70%) than surface-exposed residues (~40%), masking per-region performance differences.
• Complement with experimental validation: Always pair recovery metrics with circular dichroism, crystallography, or biochemical assays to confirm folding and function.

Standardized test sets and protocols ensure fair comparison across inverse folding models:

• CATH 4.2 test split: Proteins are partitioned by topology class to prevent homologous structures from appearing in both training and test sets, ensuring a rigorous evaluation of generalization.
• TS50 and TS500: Curated test sets of 50 and 500 diverse, high-resolution protein structures commonly used in the literature.
• Per-residue confidence: Advanced models like ESM-IF1 output per-position confidence scores, allowing users to identify regions where predicted residues are most reliable.
• Sequence diversity metrics: Recovery should be reported alongside sequence entropy or average pairwise identity to assess whether the model produces diverse or collapsed sequence distributions.

Recovery rate correlates with but does not guarantee experimental design success:

• In silico validation: High recovery (>50%) on native backbones indicates the model has captured fundamental physicochemical constraints like packing, hydrogen bonding, and hydrophobic burial.
• Experimental correlation: Proteins designed by ProteinMPNN with moderate recovery rates (~40-50%) on hallucinated backbones have been experimentally verified to fold correctly via X-ray crystallography and cryo-EM.
• Thermostability link: Higher recovery rates often correlate with improved thermostability and expression yield in recombinant systems.
• Iterative refinement: Recovery rate can guide iterative design cycles—residues with low model confidence are prime candidates for targeted mutagenesis and experimental optimization.

Different inverse folding architectures exhibit characteristic recovery rate profiles:

• ProteinMPNN (message-passing): Achieves ~52% native recovery by explicitly modeling k-nearest neighbor graphs of backbone coordinates, capturing local structural context.
• ESM-IF1 (GVP-Transformer): Combines geometric vector perceptrons with transformer attention, achieving ~51% recovery while providing per-residue confidence estimates.
• PiFold (graph-based): Uses a residue-based featurizer and global graph attention, reaching ~54% on CATH benchmarks.
• LM-Design (language model): Leverages pre-trained protein language model embeddings as structural priors, achieving competitive recovery without explicit 3D coordinate input.
• Recovery differences of 1-3% between top models are often statistically significant given large test sets but may not translate to meaningful experimental differences.

A core metric derived from protein language models that quantifies how 'surprised' the model is by a given sequence. Lower perplexity indicates the sequence is highly probable under the model's learned distribution of natural proteins. It is used to assess the native-like quality of designed sequences, complementing recovery rate by evaluating global sequence plausibility rather than exact positional matches.

A structural validation metric where a designed sequence is fed into a structure prediction model like AlphaFold2 or ESMFold. The predicted structure is then compared to the original target backbone. A high scTM score (>0.9) confirms that the sequence encodes the intended fold, providing a crucial orthogonal check to sequence recovery rate by verifying the design-structure consistency.

The specific task of predicting the exact wild-type residue at each position of a native protein backbone. This is the most common benchmark for inverse folding models like ProteinMPNN and ESM-IF1. High recovery rates indicate the model has learned the complex sequence-structure relationships governing protein folding, though it does not measure the model's ability to design novel, non-native sequences.

A measure of the variety of residues at each designed position across multiple sampled sequences. While high sequence recovery rate can indicate accuracy, it may also reflect a model that simply memorizes the native sequence. Evaluating diversity ensures the model generates a non-degenerate ensemble of viable sequences, which is critical for downstream experimental screening and avoiding trivial solutions.

The computational task of predicting an amino acid sequence that will fold into a specified three-dimensional protein backbone structure. It is the inverse problem of structure prediction. Sequence recovery rate is the primary benchmark metric for this task, measuring the percentage of correctly predicted residues when the model is asked to recover the native sequence of a known protein backbone.