Cross-Modal Translation

The foundational neural architecture for cross-modal translation. An encoder compresses the source modality (e.g., DNA sequence) into a latent representation, while a decoder reconstructs the target modality (e.g., chromatin accessibility).

• Convolutional encoders capture local sequence motifs
• Transformer encoders model long-range regulatory interactions
• U-Net variants preserve spatial resolution for track-level predictions
• The bottleneck forces the model to learn a compressed, biologically meaningful representation

The task of predicting continuous epigenomic tracks from discrete DNA sequence alone. Models learn the cis-regulatory grammar that governs chromatin state.

• Input: One-hot encoded DNA sequence (A, C, G, T)
• Output: Continuous signal tracks (e.g., DNase-seq, ATAC-seq, ChIP-seq)
• Key models: Basenji2, Enformer, and Sei
• Enables prediction of chromatin accessibility in unmeasured cell types
• Captures the effect of non-coding variants on regulatory element activity

The generative task of computationally inferring a completely absent omics layer from an available one. This addresses the missing modality problem common in clinical and research datasets.

• RNA → Protein: Predict proteomic abundance from transcriptomic data using models like TotalVI
• DNA → RNA: Infer gene expression from promoter and enhancer sequences
• scATAC → scRNA: Translate chromatin accessibility to transcriptomic profiles in single cells
• Enables retrospective analysis when certain assays were never performed

Probabilistic frameworks that learn a joint posterior distribution from multiple input modalities, enabling bidirectional translation. MVAEs can generate any modality from any other modality.

• Product-of-experts formulation combines modality-specific encoders
• Learns a shared latent space where modalities are aligned
• Handles arbitrary missingness patterns at inference time
• Applications include synthetic multi-omic data generation
• Key implementations: scMM, MultiVI, and Cobolt

A self-supervised learning paradigm that aligns paired observations from different modalities in a shared embedding space without requiring paired data during training.

• CLIP-inspired approaches adapted for genomics (e.g., GeneCLIP)
• Pulls paired profiles (same cell's RNA and ATAC) together
• Pushes unpaired profiles apart in the latent space
• Enables zero-shot cross-modal retrieval
• Facilitates annotation transfer between modalities without explicit translation

A training strategy where entire data modalities are randomly zeroed out during model training. This forces the model to learn robust representations that do not over-rely on any single modality.

• Improves generalization to incomplete clinical datasets
• Prevents the model from learning trivial modality-specific shortcuts
• Enables graceful degradation when assays are missing
• Critical for real-world deployment where full multi-omic panels are rare
• Often combined with product-of-experts inference networks

The foundational neural framework for cross-modal translation. An encoder compresses the source modality (e.g., DNA sequence) into a latent representation, while a decoder reconstructs the target modality (e.g., chromatin accessibility).

• U-Net variants with skip connections preserve fine-grained spatial patterns
• Transformer-based encoders capture long-range dependencies in sequences
• Bottleneck latent space forces the model to learn essential cross-modal mappings

Example: Enformer uses a convolutional encoder to map 200kb DNA context to predicted epigenetic tracks.

The generative task of computationally predicting a completely absent omics layer from available data. This addresses the practical reality that most biological samples lack comprehensive multi-omic profiling.

• Transcript-to-proteome: Predicting protein abundance from RNA-seq data
• Sequence-to-epigenome: Inferring chromatin states from DNA alone
• Single-cell imputation: Filling in unmeasured modalities in multi-omic atlases

Critical for clinical applications where cost or tissue limitations prevent exhaustive assays.

A generative probabilistic framework that learns a joint posterior distribution from multiple input omics layers. The MVAE encodes each modality into a shared latent space and decodes to reconstruct all modalities.

• Handles missing modalities through product-of-experts inference
• Enables synthetic multi-omic data generation for augmentation
• Learns modality-invariant representations for downstream tasks

Used extensively in cancer subtyping where some assays are unavailable for certain patients.

A self-supervised paradigm that aligns representations across modalities without requiring paired labels. The model learns to pull truly paired samples together in latent space while pushing unpaired samples apart.

• CLIP-inspired approaches for genomics: matching DNA to expression
• Requires careful negative sampling to avoid trivial solutions
• Produces aligned embeddings usable for zero-shot cross-modal retrieval

Example: Training on paired scRNA-seq and scATAC-seq from the same cells to learn a shared manifold.

A training technique where entire data modalities are randomly zeroed out during model training. This forces the network to learn robust representations that do not rely on any single omics layer.

• Prevents modality co-adaptation where the model overfits to one data type
• Simulates real-world missingness patterns in clinical settings
• Improves generalization to samples with incomplete profiling

Essential for deploying cross-modal models in production where assay availability varies.

A transformer component that allows one modality to selectively query contextual information from another modality. Queries from the source modality attend to keys and values from the target modality.

• Enables fine-grained alignment between sequence positions and epigenetic signals
• Dynamically weighs relevance of cross-modal features per prediction
• Scales to long-range interactions in genomic sequences

Used in architectures like Enformer and Borzoi for predicting expression from DNA sequence context.