3D Genome Folding

The primary supervised task where models predict the contact frequency matrix between pairs of genomic loci from DNA sequence alone. A Hi-C map is a square matrix where entry (i, j) represents the frequency of physical interaction between locus i and locus j.

• Input: One-hot encoded DNA sequence, often 1–2 megabases in length
• Output: A predicted contact probability map at a specific resolution (e.g., 5–10 kb bins)
• Loss function: Often Poisson or mean squared error loss, comparing predicted contacts to experimental Hi-C data
• Key challenge: The output matrix is sparse and high-dimensional, requiring specialized architectures to handle the quadratic scaling of genomic distance

A critical downstream analysis where models identify self-interacting genomic regions that are insulated from neighboring domains. TADs are fundamental structural units of chromosome folding, typically spanning hundreds of kilobases.

• Mechanism: Models learn to detect the boundaries where contact frequency sharply drops, often marked by CTCF binding sites in convergent orientation
• Insulation score: A quantitative metric computed from the predicted contact map, measuring the degree of boundary strength at each genomic bin
• Biological significance: TAD boundaries constrain enhancer-promoter interactions, and their disruption is linked to developmental disorders and oncogene activation
• Model output: A one-dimensional track of boundary probabilities across the entire input sequence

The central architectural challenge that distinguishes 3D genome folding models from standard genomic classifiers. Enhancers can regulate genes located megabases away in linear sequence, requiring models to capture dependencies across vast genomic distances.

• Dilated attention: Sparse attention patterns that skip intervening bins to directly connect distal elements
• Hierarchical architectures: Multi-scale models that process the sequence at multiple resolutions simultaneously, merging local motif detection with global chromatin domain awareness
• Relative positional encoding: Methods like Rotary Position Embedding (RoPE) that naturally encode the distance between two loci, helping the model learn distance-dependent contact decay
• State Space Models: Linear-complexity alternatives like Mamba that can process entire chromosomes without the quadratic memory cost of full attention

The learned mapping from primary nucleotide sequence to three-dimensional folding patterns. Models implicitly capture the cis-regulatory code that governs chromatin architecture without explicit feature engineering.

• Motif grammar: Attention heads learn to recognize binding motifs for architectural proteins like CTCF, cohesin, and YY1
• Orientation sensitivity: Models distinguish between convergent, tandem, and divergent CTCF motif pairs, which is essential for loop extrusion directionality
• Epigenetic integration: Advanced models incorporate DNase-seq or ChIP-seq tracks as additional input channels, learning the interplay between chromatin state and 3D structure
• Transfer learning: Pre-trained genomic language models like Enformer or Nucleotide Transformer provide rich embeddings that encode this grammar, which can be fine-tuned for folding prediction

A mechanistic modeling approach where transformers learn to emulate the cohesin-mediated loop extrusion process that shapes interphase chromatin. Rather than predicting static contacts, these models capture the dynamic process.

• Extrusion factors: Cohesin complexes load onto chromatin and extrude DNA bidirectionally until blocked by convergently oriented CTCF proteins
• Model integration: Some architectures combine a transformer for sequence feature extraction with a differentiable loop extrusion simulator for physically grounded contact map generation
• Dynamic prediction: Models can predict how contact maps change upon cohesin depletion or CTCF site deletion, enabling in-silico perturbation experiments
• Validation: Predicted extrusion dynamics are validated against Hi-C data from cohesin degradation time-course experiments

The ability of a 3D genome folding model trained on one cell type to accurately predict chromatin architecture in an unseen cell type, a critical test of whether the model has learned universal folding principles.

• Invariant features: Models learn that CTCF binding and cohesin loading are primary determinants of structure across cell types, while cell-type-specific transcription factor binding modulates local interactions
• Fine-tuning strategy: Pre-training on a large compendium of Hi-C data from diverse cell types, then fine-tuning on a specific lineage with limited data
• Evaluation metric: Genome-wide correlation between predicted and experimental contact maps in held-out cell types, often measured by Hi-C reproducibility metrics like HiCRep or Stratum-adjusted Correlation Coefficient (SCC)
• Failure modes: Models struggle with cell types undergoing large-scale genome reorganization, such as mitotic chromosome condensation or meiotic pairing

The primary supervised learning task for 3D genome folding models. A Hi-C contact map is a matrix quantifying the interaction frequency between every pair of genomic loci, measured via chromosome conformation capture. Transformer models learn to predict these maps directly from DNA sequence, implicitly learning the grammar of chromatin loop extrusion, CTCF binding, and cohesin complex dynamics. The predicted maps reveal megabase-scale organizational features including A/B compartments (active vs. inactive chromatin) and topologically associating domains (TADs).

A fundamental unit of 3D genome organization where genomic regions exhibit high internal interaction frequency but are insulated from neighboring regions. TAD boundaries are enriched for CCCTC-binding factor (CTCF) motifs and act as regulatory neighborhoods, constraining enhancer-promoter contacts. Deep learning models like Akita and DeepC predict TAD boundaries from sequence by learning the complex interplay between architectural proteins. Disruption of TAD boundaries is a known mechanism in developmental disorders and oncogene activation.

DeepMind's Enformer architecture extended the receptive field of genomic transformers to 200 kilobases, enabling direct prediction of 3D contact frequencies from sequence. By replacing local convolutions with self-attention across distant genomic positions, Enformer captures enhancer-gene interactions that span up to 100 kb. The model outputs multi-resolution contact tracks and epigenetic profiles, demonstrating that 3D folding information is encoded in the primary DNA sequence and can be decoded by sufficiently expressive architectures.

The biophysical mechanism by which the cohesin protein complex actively reels chromatin into loops until blocked by convergently oriented CTCF proteins. This process is the primary driver of TAD formation and enhancer-promoter proximity. Transformer models trained on Hi-C data implicitly learn the sequence determinants of loop extrusion, including CTCF motif orientation, spacing, and binding affinity. Understanding loop extrusion is critical for interpreting model predictions and designing synthetic regulatory circuits.

A key advantage of transformer models over convolutional baselines is the direct interpretability of attention weights. By visualizing attention heatmaps between distal genomic loci, researchers can identify which specific sequence elements the model uses to predict a contact. This has led to the discovery of novel enhancer-gene linkages and regulatory motifs. Techniques like attention rollout and gradient-based attribution map the model's reasoning back to specific nucleotides, providing mechanistic hypotheses for experimental validation.

While bulk Hi-C captures population-averaged contact maps, single-cell Hi-C reveals the stochastic, cell-type-specific nature of 3D genome folding. Transformer models adapted for single-cell data must handle extreme sparsity and learn to distinguish technical dropout from biological variation. These models enable the identification of cell-type-specific TADs and dynamic chromatin loops that differ between cell states, providing a structural basis for differential gene regulation during development and disease progression.