Basenji

Basenji replaces standard convolutions with dilated convolutions to exponentially expand the receptive field without a linear increase in parameters. This allows the model to integrate distal regulatory elements, such as enhancers, that are hundreds of kilobases away from target promoters.

• Captures long-range interactions critical for gene regulation
• Maintains high spatial resolution for precise activity prediction
• Reduces computational cost compared to dense attention mechanisms

A single shared trunk network feeds into multiple independent output heads, enabling simultaneous prediction of diverse genomic tracks. This multi-task learning paradigm forces the model to learn a generalizable representation of regulatory grammar.

• Jointly predicts CAGE expression, DNase-seq accessibility, and ChIP-seq histone marks
• Improves generalization by leveraging shared biological structure
• Reduces the need for separate, tissue-specific models

Basenji operates directly on raw one-hot encoded DNA sequence, eliminating the need for hand-crafted features or pre-computed alignments. The model autonomously learns to detect sequence motifs, splice sites, and regulatory syntax.

• Input: 131 kb windows of the reference genome
• Learns de novo motif representations in its first layer
• Enables prediction on any species with a sequenced genome

To handle the count-based nature of sequencing data, Basenji optimizes a Poisson negative log-likelihood loss function. This statistical framework is inherently suited for modeling discrete, over-dispersed read counts across genomic bins.

• Models the variance structure of sequencing assays correctly
• Prevents the model from being dominated by high-signal regions
• Provides calibrated uncertainty estimates for predictions

A core interpretability feature where every nucleotide in an input sequence is systematically mutated, and the change in the predicted output is recorded. This creates a high-resolution map of regulatory motif logic.

• Identifies causal nucleotides driving expression changes
• Recovers known transcription factor binding motifs
• Predicts the functional impact of non-coding genetic variants

The genome is segmented into non-overlapping 128 bp bins, and predictions are made for each bin. This tiling approach converts the continuous genome into a discrete grid suitable for convolutional processing.

• Standardizes input size for efficient batch processing
• Aligns with the resolution of typical epigenomic assays
• Enables direct comparison with binned experimental data tracks

A core architectural component of the Basenji framework. Dilated convolutions insert gaps between kernel elements, exponentially expanding the receptive field without increasing parameter count or sacrificing resolution. This allows the model to integrate regulatory signals across tens of thousands of base pairs while maintaining single-nucleotide prediction granularity. Key advantages include:

• Exponential receptive field growth
• No pooling-induced resolution loss
• Efficient parameter usage for long sequences

A training paradigm central to Basenji's design where a single neural network simultaneously predicts thousands of genomic tracks—including DNase-seq, ChIP-seq, and CAGE profiles—across multiple cell types and tissues. By sharing a common DNA sequence representation, the model learns universal regulatory grammar. This approach improves generalization, reduces overfitting on sparse assays, and creates a rich, transferable embedding space for downstream fine-tuning tasks.

A computational perturbation method used to interpret Basenji's predictions. Every nucleotide in an input sequence is systematically mutated, and the change in predicted regulatory activity is measured. This generates saliency maps that highlight causal regulatory variants and transcription factor binding motifs. The technique transforms a black-box predictor into a variant effect predictor, enabling the discovery of pathogenic non-coding mutations.

A mechanism that injects sequential order information into the input embeddings of a neural network. While Basenji relies primarily on dilated convolutions to capture distance, hybrid architectures incorporating attention layers use positional encoding to distinguish identical motifs at different genomic loci. This is critical for understanding syntax in regulatory grammar, where the spacing and order of transcription factor binding sites determine function.