Glossary

RNA-Puzzles

A community-wide blind assessment experiment that evaluates the state-of-the-art in RNA tertiary structure prediction by challenging participants to predict unpublished crystallographic or cryo-EM structures.

Get in touch Learn more

Consultant assessing AI maturity on laptop, assessment framework visible, professional office setup.

Community Benchmark

What is RNA-Puzzles?

A community-wide blind assessment experiment evaluating the state-of-the-art in computational RNA tertiary structure prediction.

RNA-Puzzles is a collective, community-wide experiment designed to critically and blindly assess the accuracy of computational methods for predicting RNA tertiary structure. It challenges participants to predict unpublished crystallographic or cryo-EM structures, providing an unbiased benchmark for the field.

The experiment evaluates predictions using standardized metrics like Root Mean Square Deviation (RMSD) and Template Modeling Score (TM-score). By comparing diverse algorithms against experimentally determined structures, RNA-Puzzles identifies systematic strengths and weaknesses, directly driving methodological innovation in geometric deep learning and molecular modeling.

COMMUNITY-WIDE BLIND ASSESSMENT

Key Features of RNA-Puzzles

RNA-Puzzles is a collective experiment that rigorously evaluates the state-of-the-art in RNA tertiary structure prediction through blind challenges, fostering collaboration and benchmarking progress in the field.

Blind Prediction Challenges

The core mechanism involves predicting unpublished RNA structures. Organizers solicit experimentally determined structures (via X-ray crystallography or cryo-EM) that are not yet public. Participants submit 3D models without knowledge of the answer, ensuring an unbiased assessment of algorithmic capability rather than data fitting.

Standardized Evaluation Metrics

Predictions are compared to the experimental structure using rigorous metrics:

RMSD: Measures global atomic deviation after optimal superposition.
Template Modeling Score (TM-score): A length-independent metric sensitive to overall topology.
Interaction Network Fidelity (INF): Quantifies the accuracy of predicted non-canonical base pairs and stacking interactions as defined by the Leontis-Westhof classification.

Diverse Target Categories

Challenges span a wide range of structural complexity to test different aspects of prediction pipelines:

Natural riboswitches and ribozymes with complex tertiary folds.
Designed nanostructures with non-natural topologies.
Protein-RNA complexes testing interface prediction.
Multiple conformational states of a single sequence.
Targets with pseudoknots and G-quadruplexes.

Collaborative Post-Mortem Analysis

After each round, a collective analysis is published dissecting the results. This identifies systematic failures, such as the inability to correctly predict long-range tertiary contacts or non-canonical motifs like the A-minor motif. These insights directly guide the development of next-generation algorithms and force fields.

Evolution of Methodological Approaches

The experiment tracks the shift in dominant methodologies over time:

Early rounds: Dominated by knowledge-based potentials and fragment assembly (e.g., Rosetta FARFAR2).
Current era: Increasingly driven by end-to-end deep learning models like AlphaFold 3 and RoseTTAFoldNA, alongside diffusion models that directly generate 3D coordinates.

Integration with CASP-RNA

RNA-Puzzles operates synergistically with the CASP-RNA experiment. While CASP runs on a fixed, time-limited biennial schedule, RNA-Puzzles provides a rolling, on-demand mechanism for testing specific structural hypotheses and challenging the community with unique targets between the major CASP seasons.

RNA-PUZZLES EXPLAINED

Frequently Asked Questions

RNA-Puzzles is a community-wide blind assessment experiment that evaluates the state-of-the-art in RNA tertiary structure prediction. Below are answers to common questions about how the challenge works, its metrics, and its impact on the field.

RNA-Puzzles is a community-wide blind assessment experiment designed to rigorously evaluate computational methods for predicting RNA three-dimensional structures. The process works as follows: experimental structural biologists who have solved an RNA structure by X-ray crystallography, NMR, or cryo-EM but have not yet published it submit the sequence to the RNA-Puzzles organizers. Participants worldwide then predict the 3D structure using their algorithms without any knowledge of the experimental coordinates. Submissions are collected, compared against the soon-to-be-released experimental structure, and ranked using standardized metrics like Root Mean Square Deviation (RMSD) and Template Modeling Score (TM-score). This blind format eliminates overfitting and provides an unbiased snapshot of the field's true predictive capability. Since its inception in 2010, RNA-Puzzles has grown to include over 30 target structures, ranging from small aptamers to large riboswitches and ribozymes, and has become the definitive benchmark alongside CASP-RNA.

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

RNA-PUZZLES ECOSYSTEM

Related Terms

Key concepts, metrics, and methodologies that define the RNA-Puzzles blind assessment experiment and its role in advancing RNA tertiary structure prediction.

CASP-RNA

The RNA-specific track of the Critical Assessment of Structure Prediction experiment. While RNA-Puzzles operates as a rolling, community-driven challenge, CASP-RNA provides a biennial, standardized benchmark for comparing computational methods. Both experiments share the core principle of blind prediction against unpublished experimental structures, but CASP-RNA enforces stricter time limits and uniform evaluation protocols.

Biennial

Assessment Frequency

Root Mean Square Deviation (RMSD)

The standard global similarity metric for quantifying the accuracy of a predicted 3D RNA structure against an experimentally determined reference. RMSD is calculated by performing an optimal rigid-body superposition of the predicted and native structures, then computing the average Euclidean distance between equivalent atoms. Lower values indicate better predictions, though RMSD is sensitive to local domain movements and can penalize otherwise correct global folds.

Ångströms

Unit of Measurement

Template Modeling Score (TM-score)

A length-independent metric for assessing global structural similarity that addresses RMSD's sensitivity to outliers. TM-score weights closer residue pairs more heavily than distant ones, making it more sensitive to overall topology than local deviations. It is commonly used in RNA-Puzzles evaluations because it provides a normalized score between 0 and 1, where values above 0.5 generally indicate the same fold as the native structure.

0–1

Score Range

>0.5

Same Fold Threshold

Leontis-Westhof Classification

A geometric ontology that systematically categorizes RNA base pairs by the interacting edges and glycosidic bond orientation. Each base pair is annotated by:

Edge: Watson-Crick, Hoogsteen, or Sugar
Orientation: cis or trans This classification enables precise annotation of 3D motifs and is essential for evaluating whether predictions correctly capture non-canonical interactions, a key differentiator in RNA-Puzzles assessments.

Predicted Local Distance Difference Test (pLDDT)

A per-residue confidence metric output by AlphaFold and related deep learning models. pLDDT estimates the local accuracy of the predicted structure by comparing predicted distances to those in the final model. In RNA-Puzzles contexts, pLDDT serves as a critical filter for interpreting model reliability, allowing researchers to identify well-predicted helical regions versus disordered loops without experimental validation.

0–100

Confidence Scale

>90

High Accuracy

Cryo-EM Density Map

A 3D Coulomb potential map reconstructed from cryo-electron microscopy images, representing the electron scattering density of an RNA molecule. In RNA-Puzzles, cryo-EM maps serve as target restraints for map-to-model fitting algorithms. Predictors must interpret these density maps to build atomic models, testing the integration of experimental data with computational prediction—a distinct challenge from purely de novo sequence-to-structure problems.

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.