Cross-Species Transfer Learning

The foundational premise of cross-species transfer is that functional genomic elements—promoters, enhancers, splice sites—are conserved across evolutionary time. A model pretrained on human or mouse genomes learns to recognize these conserved sequence motifs. When applied to a non-model organism, the model's contextualized sequence representations encode phylogenetic priors that generalize across species boundaries. The degree of transfer success correlates directly with evolutionary distance: human-to-chimpanzee transfer outperforms human-to-zebrafish transfer. This conservation signal acts as an implicit regularizer, preventing overfitting to species-specific noise.

Effective cross-species transfer depends on tokenization strategies that capture evolutionary homology. K-mer tokenization with k=6 provides a vocabulary of 4,096 possible tokens that can represent conserved motifs across species. More advanced approaches use Byte-Pair Encoding (BPE) to learn species-agnostic subword units from multi-species genomic corpora. The tokenizer must balance vocabulary sharing—enabling transfer of learned motif representations—with species-specific token granularity. A tokenizer trained exclusively on human sequences may fragment conserved elements in distant species, degrading transfer performance.

Rather than pretraining on a single species and transferring, multi-species joint pretraining builds a single model on aligned or concatenated genomes from dozens of species. This approach forces the model to learn species-invariant representations of regulatory grammar. Architectures like the Enformer model have been extended to multi-species training, learning to predict expression and epigenomic tracks across mammals. Key design decisions include:

• Species token prefixing to provide phylogenetic context
• Balanced sampling to prevent data-rich species from dominating gradients
• Shared embedding spaces that align orthologous regulatory elements

Cross-species transfer enables zero-shot variant effect prediction in organisms where labeled pathogenic variants are scarce. A genomic language model pretrained on human data can compute the sequence log-likelihood difference between reference and alternate alleles in a target species. The underlying assumption is that mutations disrupting conserved regulatory grammar will show similar likelihood drops regardless of species. This approach has been validated for:

• Prioritizing disease-causing variants in canine and feline genomes
• Identifying deleterious mutations in crop plant breeding programs
• Conservation scoring in endangered species with no population-level variant databases

When evolutionary distance is substantial—such as transferring from vertebrates to plants—standard fine-tuning degrades. Domain adaptation techniques bridge this gap. Adversarial domain adaptation trains a discriminator to predict species of origin from hidden representations while the encoder learns to produce species-invariant features. Gradual unfreezing strategies progressively unfreeze transformer layers during fine-tuning, allowing lower layers to retain universal motif detectors while upper layers adapt to species-specific regulatory logic. These methods have enabled transfer from mammalian models to arthropod and fungal genomes.

The initial phase of training a DNA language model on massive, unlabeled genomic corpora using self-supervised objectives. During this phase, the model learns universal sequence representations—including conserved motifs, regulatory grammar, and structural patterns—that are shared across species. These representations form the foundation that enables transfer learning to organisms with limited training data.

Dynamic nucleotide embeddings where the vector for a given k-mer changes depending on its surrounding sequence context. Unlike static embeddings, these representations capture regulatory syntax—the combinatorial rules by which transcription factor binding sites interact. This context-sensitivity is critical for transfer learning because regulatory logic is often conserved even when raw sequence identity diverges between species.

A modeling paradigm that leverages deep learning on vast protein or DNA sequence alignments to capture evolutionary constraints. By training across diverse taxa simultaneously, ESM-based approaches learn phylogenetically-aware representations that explicitly encode the evolutionary distances between species, enabling principled transfer of functional annotations from well-studied model organisms to newly sequenced genomes.

The capability of a genomic language model to predict the functional impact of a genetic variant without being explicitly trained on labeled variant effect data. The model computes the change in sequence likelihood between reference and alternate alleles. Cross-species transfer amplifies this capability by allowing models trained on human ClinVar data to score variants in non-model organisms where labeled pathogenic variants are scarce.

The probability assigned to a genomic sequence by an autoregressive model, measuring how well the sequence conforms to learned patterns of natural DNA. In cross-species contexts, a model trained primarily on mammalian genomes can compute log-likelihood scores for a fish or plant genome—regions with surprisingly low likelihood often indicate lineage-specific functional elements or novel regulatory innovations absent from the training distribution.

A set of adaptation techniques, such as LoRA, that update only a small fraction of a genomic foundation model's parameters for a downstream task. PEFT is essential for cross-species transfer because it allows a single pretrained model to be rapidly specialized to each target species' genome without catastrophic forgetting of the universal regulatory knowledge acquired during pretraining, dramatically reducing compute costs.