Cross-Species Transfer Learning

The foundational assumption enabling cross-species transfer. Despite millions of years of divergence, the grammar of gene regulation—the spacing, orientation, and combinatorial logic of transcription factor binding sites—remains remarkably conserved. A model pre-trained on the human genome learns this deep syntax, allowing it to recognize functional promoters, enhancers, and insulators in a target species like zebrafish or pig without ever seeing those specific sequences during initial training.

The process of identifying orthologs—genes in different species that evolved from a common ancestral gene—serves as the bridge for transfer. Embedding models are fine-tuned on aligned syntenic blocks where sequence homology is high. This alignment provides a direct mapping between the source and target species' genomic coordinates, allowing the model to project learned regulatory annotations from human to mouse loci with high confidence.

A technique to prevent the model from overfitting to species-specific sequence biases like GC content or k-mer frequencies. A gradient reversal layer is added during fine-tuning, forcing the encoder to learn features that are simultaneously predictive of the biological task (e.g., gene expression) but uninformative about which species the sequence came from. This yields truly species-agnostic embeddings.

A training schedule that mitigates catastrophic forgetting of the pre-trained regulatory grammar. The process occurs in stages:

• Stage 1: Freeze all convolutional and attention layers; train only the final classification head on the target species.
• Stage 2: Unfreeze the top transformer layers, allowing adaptation of high-level regulatory logic.
• Stage 3: Unfreeze all layers with a very low learning rate for full model recalibration. This preserves low-level motif detectors while adapting high-level syntax.

The ultimate benchmark of transfer learning efficacy. A model pre-trained exclusively on human ATAC-seq and ChIP-seq data is evaluated on its ability to predict chromatin accessibility in a completely unseen species, such as the axolotl, without any fine-tuning. Performance is measured by the correlation between predicted and experimentally measured signal tracks, validating that the model has captured universal, sequence-intrinsic determinants of regulatory function.

The vocabulary used to tokenize the source and target genomes must be compatible. If the source model uses byte-pair encoding (BPE) trained on the human genome, the target species' sequences are tokenized using the identical merge rules. This ensures that a conserved motif like the TATA box maps to the same token IDs across species, preserving the semantic meaning learned during pre-training in the shared embedding space.

The foundational step in cross-species transfer that identifies evolutionarily conserved sequences across genomes. By aligning syntenic blocks and orthologous promoters between human, mouse, and target species, models can anchor their predictions on regions where regulatory grammar is most likely preserved. Tools like liftOver and whole-genome alignments create the coordinate bridges that allow embeddings from one species to be projected onto another.

A transfer learning technique that uses a gradient reversal layer to force the model to learn species-invariant features. During training, a domain classifier attempts to predict which species a sequence came from while the feature extractor is penalized for making that prediction easy. The resulting embeddings strip away species-specific signals while preserving conserved regulatory syntax, enabling a classifier trained on human data to generalize to zebrafish or Arabidopsis.

A strategy that weights training examples based on evolutionary distance from the target species. Rather than treating all source species equally, sequences from closely related species receive higher importance during fine-tuning. This approach uses a phylogenetic tree to compute distance-aware loss functions, ensuring that a model transferring to canine genomics prioritizes mouse regulatory patterns over those from more distant clades like insects or plants.

Low-Rank Adaptation enables parameter-efficient fine-tuning of large genomic language models for new species without catastrophic forgetting. By freezing the pre-trained weights of a model like DNABERT-2 and injecting trainable low-rank decomposition matrices into attention layers, only 0.1-1% of parameters are updated. This preserves the conserved regulatory grammar learned from human data while adapting to species-specific sequence motifs with minimal compute.

A self-supervised approach that pulls orthologous promoter embeddings close together in latent space while pushing apart non-homologous regions. By training with a contrastive loss on aligned human-mouse-zebrafish triples, the model learns a species-agnostic regulatory manifold. A classifier trained on human ChIP-seq peaks can then directly operate on the embedding of a target species' sequence without any labeled examples from that species.