Sequence-to-Epigenome Modeling

A single neural network is trained to simultaneously predict thousands of epigenomic tracks across diverse cell types and assay types. The model head outputs a multi-dimensional tensor where each channel corresponds to a specific experiment.

• Shared trunk: A common convolutional or transformer backbone learns universal regulatory grammar from DNA sequence
• Task-specific heads: Separate output layers specialize for each epigenomic profile
• Benefit: Shared representations improve generalization to cell types with sparse training data
• Example: The Enformer model predicts 5,313 tracks covering CAGE, ChIP-seq, and DNase-seq assays across human and mouse genomes

Modern architectures ingest 100-200 kilobase input windows to capture distal regulatory interactions. Enhancers can regulate promoters located tens of thousands of bases away through chromatin looping.

• Dilated convolutions: Exponential dilation rates in convolutional layers expand the receptive field without quadratic parameter growth
• Transformer attention: Self-attention mechanisms model pairwise interactions across the full input window
• Basenji2 processes 131 kb sequences using dilated convolutions
• Enformer uses a hybrid CNN-transformer architecture with a 200 kb receptive field
• Contrast: Earlier models like DeepSEA used only 1 kb windows, missing distal regulation

Models learn intrinsic DNA sequence grammar through pretext tasks on unlabeled genomic data before fine-tuning on specific epigenomic predictions. This leverages the vast amount of sequenced genomes without requiring expensive experimental assays.

• Masked language modeling: Random nucleotides are masked and the model predicts them from surrounding context, analogous to BERT in NLP
• Next token prediction: Autoregressive modeling of nucleotide sequences
• Nucleotide Transformer pre-trains on diverse reference genomes and the human pangenome
• Benefit: Pre-trained embeddings capture evolutionary constraints, motif syntax, and sequence homology that transfer to downstream epigenomic tasks with limited labeled data

A computational perturbation technique that systematically introduces virtual mutations into input sequences and measures the predicted change in epigenomic output. This identifies nucleotides critical for regulatory function.

• Saturation mutagenesis: Every position is mutated to all three alternative bases, and the model's prediction delta is recorded
• ISM scores produce a mutation impact map across the sequence, highlighting transcription factor binding motifs
• Integrated Gradients attribute predictions to input bases by accumulating gradients along a path from a neutral reference sequence
• Application: Prioritizing non-coding variants in genome-wide association studies by predicting which single nucleotide polymorphisms disrupt chromatin accessibility or histone modification patterns

The ability of a model trained on epigenomic data from a source set of cell types to accurately predict regulatory activity in an unseen target cell type. This tests whether the model learns universal regulatory grammar versus memorizing cell-type-specific patterns.

• Zero-shot prediction: The model predicts tracks for a completely held-out cell type without any fine-tuning
• Few-shot adaptation: Fine-tuning on a small number of examples from the target cell type
• Evaluation metric: Pearson correlation between predicted and experimentally measured signal tracks in the held-out cell type
• Challenge: Cell-type-specific transcription factor expression creates distinct chromatin landscapes that may not be inferable from sequence alone
• Enformer demonstrates strong cross-cell-type generalization, suggesting it learns fundamental sequence determinants of regulation

Statistical methods that estimate a model's confidence in its predictions, distinguishing between two sources of uncertainty critical for scientific and clinical applications.

• Epistemic uncertainty: Model ignorance due to limited training data or capacity—reducible with more data
• Aleatoric uncertainty: Inherent noise in the biological system or measurement process—irreducible
• Monte Carlo dropout: Multiple stochastic forward passes at inference time produce a distribution of predictions; variance indicates uncertainty
• Deep ensembles: Training multiple independent models and measuring prediction spread
• Application: Flagging genomic regions where predictions are unreliable, preventing false conclusions in variant interpretation or drug target identification

A training strategy where a single neural network simultaneously predicts multiple epigenomic assays across different cell types. By sharing hidden representations, the model learns a unified regulatory grammar that generalizes better than single-task models. Key benefits include:

• Improved performance on data-scarce assays through shared feature extraction
• Implicit cross-cell-type regularization that reduces overfitting
• A single forward pass producing thousands of track predictions This approach is foundational to DeepSEA, Enformer, and Basenji2.

A computational perturbation technique that systematically introduces virtual mutations into a DNA sequence and measures the predicted change in the model's output. By performing a saturating mutagenesis scan—substituting every possible nucleotide at every position—researchers can identify regulatory variants with the largest predicted effect sizes. This method is widely used to prioritize non-coding variants from genome-wide association studies for experimental validation, bridging the gap between statistical association and mechanistic understanding.

The process of adapting a model pre-trained on a large, general epigenomic corpus to a specific, data-scarce target task. A typical workflow:

• Pre-train on thousands of tracks from ENCODE and Roadmap Epigenomics
• Fine-tune on a rare cell type or disease condition with limited data
• The pre-trained model provides a strong regulatory prior, dramatically reducing the number of target-specific training examples required This paradigm is central to genomic foundation models like the Nucleotide Transformer.

The statistical assessment of a model's confidence in its epigenomic predictions. Two distinct types are measured:

• Epistemic uncertainty: Model ignorance due to limited training data or capacity, reducible with more data
• Aleatoric uncertainty: Inherent noise in the biological measurement, irreducible Techniques like Monte Carlo dropout and deep ensembles produce prediction intervals that help researchers distinguish high-confidence regulatory annotations from speculative ones, critical for prioritizing experimental follow-up.