Enhancer-Gene Linking

The defining capability of enhancer-gene linking models is their ability to bridge genomic distances exceeding 100 kilobases. Unlike convolutional architectures with limited receptive fields, transformer models use self-attention mechanisms to directly model interactions between a gene's promoter and its distal regulatory elements. This allows the model to learn that a enhancer's regulatory effect on a gene is independent of the linear distance separating them on the chromosome, but dependent on the three-dimensional chromatin conformation.

A critical design principle is the reliance on raw DNA sequence as the sole input. The model does not require experimental data like Hi-C contact maps, ChIP-seq peaks, or chromatin accessibility assays to make predictions. Instead, it learns to infer the complex cis-regulatory grammar—including transcription factor binding motifs, their combinatorial logic, and spacing constraints—directly from the nucleotide sequence. This makes the model a truly predictive tool, capable of generating hypotheses for uncharacterized cell types or disease-associated non-coding variants.

Enhancer-gene linking models offer a unique window into their decision-making through attention weight analysis. By extracting the self-attention scores from the transformer layers, researchers can generate attention heatmaps that highlight which distal regions the model focuses on when predicting a gene's expression level. These high-attention regions often correspond to experimentally validated enhancers. This built-in interpretability mechanism transforms the model from a black-box predictor into a hypothesis-generation engine for discovering novel regulatory elements.

A practical application of these models is the systematic in-silico scoring of enhancer-gene pairs. For a given gene, the model can evaluate every candidate enhancer within its receptive field by measuring the predicted change in gene expression when that enhancer sequence is included or masked. This generates a quantitative regulatory potential score for each pair, enabling the ranking and prioritization of non-coding variants identified in genome-wide association studies (GWAS) that fall within putative enhancers but lack a known target gene.

While base models learn a general regulatory grammar, enhancer-gene linking is inherently cell-type-specific. An enhancer active in neurons may be silent in hepatocytes. To address this, models are fine-tuned using parameter-efficient techniques like LoRA on cell-type-specific epigenomic data. This adapts the model's predictions to the unique chromatin landscape of a particular cell type without retraining the entire architecture, enabling the construction of cell-type-resolved enhancer-gene maps for applications in single-cell genomics and disease-specific target discovery.

The core mechanism in transformer architectures that computes a weighted representation of every position in a sequence by dynamically assessing the relevance of all other positions. In the context of enhancer-gene linking, self-attention allows the model to learn that a regulatory element 50,000 base pairs away directly influences a gene's promoter.

• Computes pairwise interaction scores across the entire input
• Captures long-range dependencies without a fixed window size
• Attention weights can be visualized to reveal putative enhancer-promoter contacts

A class of transformer-based models pre-trained on vast quantities of unlabeled DNA sequence data using self-supervised objectives like Masked Language Modeling (MLM). These models learn contextual representations of nucleotides that encode regulatory grammar, enabling fine-tuning for enhancer-gene linking with limited labeled data.

• Pre-training on diverse genomes builds a universal understanding of sequence syntax
• Embeddings capture transcription factor binding motifs and evolutionary conservation
• Examples include DNABERT, Nucleotide Transformer, and HyenaDNA

An efficient approximation of full self-attention where each token attends only to a predefined subset of other tokens, such as a local window or a dilated pattern. This reduces the quadratic memory cost of standard attention, enabling the processing of extremely long genomic sequences necessary for whole-genome enhancer-gene linking.

• Local window attention captures nearby regulatory interactions
• Dilated or global tokens provide long-range context
• Critical for scaling models to megabase-scale inputs

A visualization of the self-attention weights from a transformer model, used as an interpretability tool to identify which specific nucleotides the model focuses on when predicting a gene's expression level. In enhancer-gene linking, attention heatmaps can reveal the specific enhancer sequences driving a prediction.

• Maps the strength of interaction between every pair of genomic positions
• Highlights putative enhancer-promoter loops
• Provides a hypothesis-generation tool for experimental validation of regulatory connections

The computational prediction of the three-dimensional spatial organization of chromatin inside the nucleus from DNA sequence alone. Transformer models trained on Hi-C contact maps learn that linear sequence features dictate physical proximity, which is the mechanistic basis for why distal enhancers can regulate distant genes.

• Predicts topologically associating domains (TADs)
• Models the physical proximity that enables enhancer-promoter looping
• Complements sequence-based linking with structural constraints