Basenji2 Architecture

The core feature extractor uses exponentially increasing dilation rates to capture regulatory syntax across multiple scales without pooling. This allows the receptive field to grow to 131 kb while maintaining single-nucleotide resolution.

• Layer 1-4: Standard convolutions for local motif detection
• Layer 5-11: Dilated convolutions with rates doubling from 2 to 64
• Result: Captures both proximal promoters and distal enhancers in a single pass

Basenji2 simultaneously predicts thousands of functional genomics tracks across multiple cell types and assays from a single shared representation. Each output task corresponds to a specific CAGE, DNase-seq, or ChIP-seq profile.

• Shared trunk: Dilated convolutional tower learns universal regulatory grammar
• Task-specific branches: Linear layers map shared features to individual assay predictions
• Benefit: Cross-task regularization improves performance on data-scarce cell types

The model optimizes a Poisson negative log-likelihood loss rather than mean squared error, explicitly modeling the count-based nature of sequencing read coverage. This handles the heteroscedastic noise inherent in genomic assays where variance scales with mean signal.

• Input: Raw sequence one-hot encoding
• Target: Log-transformed read counts per genomic bin
• Advantage: Naturally handles zero-inflated and over-dispersed count data

Basenji2 enforces strand invariance by averaging predictions from forward and reverse-complement sequence inputs during training and inference. This biological prior—that regulatory activity is strand-independent—acts as a built-in data augmentation and regularizer.

• Mechanism: Siamese-style twin pass through the network
• Output: Element-wise mean of both strand predictions
• Impact: Reduces overfitting and improves generalization to unseen sequences

The genome is partitioned into non-overlapping 128 bp bins, and the model predicts aggregate regulatory signal within each bin. This tiling strategy transforms the continuous genome into a discrete grid suitable for convolutional processing.

• Input: 131,072 bp centered on target bin (1,024 bins)
• Output: Scalar prediction per bin per assay
• Design choice: Balances resolution with computational tractability

Each dilated convolutional block employs residual connections that add the block's input to its output before batch normalization and ReLU activation. This architectural pattern enables stable gradient flow through the deep 11-layer tower and accelerates convergence.

• Pattern: Conv → BatchNorm → ReLU → Conv → BatchNorm → Add Input → ReLU
• Benefit: Mitigates vanishing gradients in deep genomic networks
• Origin: Adapted from ResNet architectures in computer vision

The core architectural innovation enabling Basenji2 to capture distal regulatory interactions. Unlike standard convolutions, dilated convolutions insert gaps between kernel elements, exponentially expanding the receptive field without increasing parameter count or losing resolution. Basenji2 uses 11 dilated convolutional layers with exponentially increasing dilation rates to process 131-kilobase sequences, allowing it to model enhancer-promoter loops spanning tens of thousands of base pairs. This design avoids the quadratic complexity of self-attention while maintaining long-range dependency modeling.

Basenji2 simultaneously predicts 4,863 genomic tracks spanning DNase-seq, ChIP-seq, and CAGE assays across multiple human and mouse cell types. Each output head is a separate linear layer operating on the shared latent representation from the dilated convolutional trunk. This multi-task setup enforces representation sharing, where features useful for predicting one assay (e.g., H3K27ac in liver) benefit predictions for related assays (e.g., H3K4me1 in liver). The loss function is a Poisson negative log-likelihood optimized jointly across all tracks.

Basenji2 belongs to the sequence-to-activity model class, where the sole input is raw DNA sequence (A, C, G, T one-hot encoded) and outputs are continuous epigenomic signal tracks. This contrasts with models requiring pre-computed genomic annotations or evolutionary conservation scores. The paradigm shift means the model must learn regulatory grammar de novo from sequence alone—including motif syntax, spacing constraints, and combinatorial logic—making it a powerful tool for interpreting non-coding variants where no prior functional annotation exists.

Basenji2 and Enformer represent two dominant approaches to long-range regulatory modeling. Basenji2 uses dilated convolutions with a 131 kb receptive field, while Enformer employs transformer self-attention with a 200 kb receptive field. Key trade-offs:

• Basenji2: faster training, lower memory footprint, deterministic receptive field
• Enformer: larger context window, dynamic attention patterns, higher computational cost Both models predict similar epigenomic tracks and are frequently benchmarked against each other for variant effect prediction tasks.

Basenji2 is tightly coupled with in-silico mutagenesis workflows for variant interpretation. By systematically introducing single-nucleotide substitutions into a reference sequence and measuring the predicted change in epigenomic tracks, researchers can compute allelic effect scores for non-coding variants. This approach has been applied to prioritize causal variants in genome-wide association study (GWAS) loci, linking regulatory disruptions to complex diseases. The model's efficiency—processing thousands of mutated sequences in parallel—makes genome-scale mutagenesis screens computationally feasible.

Basenji2 models epigenomic assay counts using a Poisson regression framework rather than mean squared error. This is critical because sequencing-based assays produce discrete count data with variance proportional to the mean. The Poisson negative log-likelihood loss naturally handles the heteroscedasticity of genomic signal—regions with high coverage have proportionally higher variance. The output layer uses a softplus activation to ensure non-negative predictions, and the loss is summed across all positions and tracks during training.