Cross-Species Transfer Learning

The foundational principle enabling cross-species transfer. During self-supervised pre-training on diverse genomes, models learn that evolutionarily conserved sequences are functionally critical. This sequence conservation signal creates a universal representation space where homologous regulatory elements—promoters, enhancers, splice sites—cluster together across species boundaries. A model pre-trained on human data can recognize a mouse promoter because it shares deep grammatical features with human promoters learned during pre-training, even when the raw nucleotide identity diverges significantly.

The dominant adaptation strategy for cross-species transfer. Instead of fine-tuning all parameters of a large genomic language model, techniques like Low-Rank Adaptation (LoRA) or adapter modules update only a tiny fraction of weights. This prevents catastrophic forgetting of the universal biological grammar learned during pre-training while allowing the model to specialize to the target species' specific regulatory syntax. Key benefits include:

• Reduces compute cost by 90-99% compared to full fine-tuning
• Prevents overfitting when target species has limited labeled data
• Enables a single pre-trained backbone to serve dozens of target species via lightweight adapters

The most striking capability of cross-species transfer learning. A genomic language model pre-trained on human data can predict variant effects, promoter activity, or enhancer locations in a completely unseen species without any target-species fine-tuning. This works because the model's likelihood scoring captures fundamental sequence fitness constraints that are evolutionarily conserved. For example, a mutation that disrupts a transcription factor binding motif will be scored as deleterious regardless of the host genome, enabling immediate functional annotation of newly sequenced species.

Advanced cross-species models incorporate synteny information—the conservation of gene order along chromosomes—into their attention mechanisms. By encoding positional relationships between orthologous genes, the model learns that regulatory logic is often preserved in syntenic blocks. This allows the attention heads to focus on conserved non-coding sequences that maintain their relative position to target genes across species, dramatically improving enhancer-gene linking accuracy in target organisms where chromatin conformation data is unavailable.

A technique borrowed from domain adaptation that explicitly aligns the embedding spaces of source and target species. A gradient reversal layer is inserted between the genomic encoder and a species classifier, training the model to produce embeddings that are simultaneously predictive of regulatory function and invariant to species identity. This forces the model to learn truly cross-species biological features rather than species-specific artifacts, resulting in embeddings where functionally equivalent elements from human and zebrafish occupy the same region of latent space.

The success of cross-species transfer correlates strongly with evolutionary distance between source and target species. Transfer from human to other mammals (mouse, dog) typically achieves near-human performance on regulatory tasks, while transfer to more distant vertebrates (zebrafish, chicken) shows moderate degradation. Transfer to invertebrates or plants requires specialized pre-training on more diverse phylogenetic datasets. This predictable decay curve allows practitioners to estimate expected performance before committing resources to target-species fine-tuning.

A measure of the degree to which a nucleotide or amino acid position remains unchanged across evolutionary time. This fundamental signal is learned by transformer models during self-supervised pre-training and correlates strongly with functional importance. Cross-species transfer learning exploits conserved regulatory grammar—such as transcription factor binding motifs and splice site signals—that persists across divergent lineages. Positions under strong purifying selection provide the phylogenetic anchor points that allow a model trained on human data to recognize functional elements in mouse or zebrafish genomes.

The computational task of identifying evolutionarily related sequences across species boundaries. Protein language model embeddings have proven superior to traditional alignment-based methods like BLAST for detecting remote homologs—sequences that share structural and functional similarity but have highly diverged at the nucleotide level. In cross-species transfer, homology detection validates whether a model's predictions in a target species correspond to genuinely orthologous regulatory elements, distinguishing true functional conservation from spurious sequence similarity.

A set of adaptation techniques that update only a tiny fraction of a pre-trained model's parameters, enabling cost-effective specialization for a new species without catastrophic forgetting. Methods include:

• Low-Rank Adaptation (LoRA): Injects trainable rank-decomposition matrices into attention layers
• Adapter modules: Small bottleneck layers inserted between transformer blocks
• Prefix tuning: Learns virtual tokens prepended to the input sequence

For cross-species transfer, PEFT allows a human-trained genomic language model to adapt to mouse regulatory grammar by updating less than 1% of parameters, preserving the universal biological syntax learned during pre-training.

A standardized suite of curated datasets and evaluation protocols designed to rigorously compare genomic language model performance across diverse nucleotide-level classification tasks. For cross-species transfer evaluation, benchmarks include paired orthologous datasets—identical task definitions (e.g., enhancer identification) applied to aligned genomic regions in human, mouse, and other model organisms. This enables direct measurement of transfer efficiency: the performance delta between a model fine-tuned on target-species data versus one relying solely on cross-species knowledge transfer.

The application of a pre-trained protein or genomic language model to predict the effect of a mutation using only the difference in sequence likelihood, without any supervised fine-tuning on labeled variant effect data. In cross-species contexts, this capability is critical: a model pre-trained on diverse vertebrate sequences can score the pathogenicity of mutations in a non-model organism by measuring how much a variant disrupts the learned evolutionary probability distribution of functional sequence space.

A predictive genomics task that maps distal regulatory elements to their target gene promoters by learning complex, long-range chromatin interaction patterns from DNA sequence alone. Cross-species transfer excels here because the syntax of enhancer-promoter communication—including insulator binding sites and cohesin loading sequences—is deeply conserved. A transformer model trained on human Hi-C and promoter-capture data can identify orthologous enhancer-gene pairs in species where experimental chromatin conformation data is sparse or unavailable.