Multi-Omic Autoencoder

Raw omics data from each assay type—genomics, transcriptomics, proteomics—is processed by dedicated encoder sub-networks before fusion. This design respects the distinct statistical properties and noise distributions of each data modality.

• DNA sequence encoders often use convolutional or transformer layers to capture motif-level patterns
• RNA expression encoders handle continuous count data, frequently employing negative binomial loss functions
• Epigenomic track encoders process signal-valued data across genomic coordinates

Each encoder compresses its input into a fixed-dimensional embedding vector, enabling downstream alignment in the shared latent space.

The central architectural constraint is a low-dimensional bottleneck layer where modality-specific embeddings are fused into a unified representation. This bottleneck forces the model to discard modality-specific noise and retain only shared biological signal.

• Dimensionality is typically orders of magnitude smaller than the input feature space (e.g., 32–256 dimensions)
• The bottleneck acts as an information filter, preserving variation driven by true biological processes
• Representations in this space are directly usable for downstream tasks such as clustering, survival analysis, or phenotype prediction

The joint latent space enables cross-modal comparison: cells or samples can be compared even when profiled with different assays.

During training, entire data modalities are randomly zeroed out with a specified probability. This stochastic regularization technique forces the model to learn robust representations that do not depend on any single omics layer being present.

• Prevents the model from over-relying on one high-quality modality while ignoring noisier but informative assays
• Enables missing modality imputation at inference time: the model can reconstruct an absent omics layer from the remaining available data
• Particularly valuable in clinical settings where not all assays are collected for every patient

Modality dropout transforms the autoencoder from a pure compression tool into a generative model capable of cross-modal translation.

The decoder stage consists of separate output heads, one per input modality, that reconstruct the original data from the shared latent representation. Each decoder head is specialized to the output distribution of its target modality.

• Count-based modalities (RNA-seq) use zero-inflated negative binomial or Poisson output layers
• Continuous modalities (methylation beta values) use Gaussian or Beta distribution outputs
• Binary modalities (mutation calls) use Bernoulli output layers

The reconstruction loss is a weighted sum across all modalities, with weights tuned to balance the contribution of each assay to the total training objective.

Technical confounders such as sequencing platform, laboratory protocol, or sample processing date introduce batch effects that can dominate biological signal. Multi-omic autoencoders combat this with adversarial regularization.

• A batch discriminator network attempts to predict the batch label from the latent representation
• The encoder is trained adversarially to maximize the discriminator's loss, producing batch-invariant embeddings
• This gradient reversal layer technique preserves biological variability while removing technical artifacts

The result is a latent space where samples cluster by biological state rather than by experimental origin, critical for multi-cohort studies.

Rather than simple concatenation of modality embeddings, advanced architectures employ cross-attention mechanisms that allow one modality to selectively query information from another before fusion.

• A transformer cross-attention block enables gene expression features to attend to relevant chromatin accessibility regions
• Attention weights provide interpretability: they reveal which genomic regulatory elements are most influential for a given gene's predicted expression
• Gated multi-modal units learn to dynamically weight each modality's contribution per sample, suppressing noisy inputs

This dynamic fusion outperforms static concatenation when modalities have varying signal-to-noise ratios across samples.