Tensor Fusion Network

The core innovation of TFNs is computing the Cartesian outer product between modality-specific embedding vectors. Given embeddings from genomics, transcriptomics, and proteomics, the network forms a multi-dimensional tensor where each entry represents a multiplicative interaction between features from different modalities. This captures trimodal correlations—for instance, how a specific DNA mutation interacts with gene expression levels and protein abundance simultaneously—that linear fusion methods cannot represent.

Unlike late fusion (concatenating final-layer features) or early fusion (concatenating raw inputs), TFNs perform mid-level multiplicative fusion. The outer product creates an exponentially large interaction space, so TFNs employ factorized tensor decompositions (e.g., CP decomposition or Tucker decomposition) to control parameter count. This factorization learns low-rank approximations of the full interaction tensor, making training tractable while preserving the capacity to model high-order cross-modal relationships.

Each omics modality passes through a dedicated encoder subnetwork before fusion. For example:

• DNA sequence → convolutional or transformer encoder producing a 128-dimensional embedding
• RNA expression → fully-connected encoder producing a 128-dimensional embedding
• Protein abundance → separate encoder producing a 128-dimensional embedding These embeddings are then combined via the outer product, ensuring that each modality's unique statistical properties are preserved before interaction computation.

TFNs excel at tasks requiring cross-omic reasoning:

• Cancer subtyping: Integrating somatic mutations, gene expression, and methylation to identify clinically distinct tumor subtypes
• Drug response prediction: Modeling how genomic variants interact with transcriptomic states to determine drug sensitivity
• Disease prognosis: Combining proteomic and metabolomic signals with genetic risk scores for survival analysis The multiplicative interactions enable detection of synergistic biomarker combinations that additive models overlook.

While cross-attention mechanisms learn to dynamically weight modalities, TFNs compute all pairwise and higher-order interactions explicitly. Key distinctions:

• TFNs: Capture multiplicative interactions via tensor products; computationally intensive but exhaustive in interaction coverage
• Attention fusion: Learns soft weighting of modalities; more scalable but may miss multiplicative effects
• Concatenation fusion: Simplest approach; cannot model interactions without downstream layers learning them implicitly TFNs are preferred when mechanistic interaction hypotheses exist between omics layers.

A full outer product of three 128-dimensional embeddings produces a tensor with 2,097,152 entries—prohibitively large. TFNs address this via:

• CP decomposition: Factorizes the tensor into a sum of rank-1 components, reducing parameters from O(d³) to O(kd) where k is the rank
• Tucker decomposition: Uses a core tensor with factor matrices, offering more flexibility than CP
• Tensor Train decomposition: Represents the tensor as a chain of low-rank matrices, suitable for very high-order interactions These approximations preserve the ability to model multiplicative correlations while keeping training feasible.

The mathematical core of Tensor Fusion Networks, computing the outer product between modality-specific embeddings to explicitly capture multiplicative interactions across omics layers.

• Expands feature dimensions from d1 + d2 to d1 × d2
• Captures bilinear correlations that additive fusion misses
• Produces a 2D interaction map for each sample
• Often followed by flattening and fully connected layers

Example: Fusing a 128-dim gene expression vector with a 64-dim methylation vector yields an 8,192-dim interaction tensor.

A computationally efficient approximation of Tensor Fusion that factorizes the full outer product into low-rank matrices, dramatically reducing parameter count while preserving multiplicative interactions.

• Decomposes weight tensor W into U × V^T
• Reduces parameters from O(d1 × d2 × output) to O((d1 + d2) × rank)
• Enables fusion with hundreds of modalities without memory explosion
• Used in visual question answering and multi-omic phenotype prediction

Key trade-off: Approximation fidelity vs. computational tractability.

A neural gating mechanism that dynamically controls information flow from distinct modality-specific encoders, allowing the model to suppress noisy or irrelevant omics inputs before fusion.

• Learns modality-wise scalar gates g ∈ [0,1] per input
• Multiplicatively weights each modality's contribution
• Enables adaptive fusion when certain assays are unreliable
• Complements Tensor Fusion by pre-filtering before outer product computation

Example: Down-weighting degraded RNA-seq samples while prioritizing high-quality proteomic data.

A regularization technique where entire data modalities are randomly zeroed out during training, forcing the fusion network to learn robust representations that handle missing clinical assays at inference time.

• Randomly drops modalities with probability p per batch
• Prevents over-reliance on any single omics layer
• Simulates real-world missing data patterns in clinical settings
• Works synergistically with Tensor Fusion: the outer product must remain informative even with absent modalities

Critical for deployment where not all assays are always available.

The task of computationally converting one data modality into another using encoder-decoder architectures, often leveraging shared latent representations learned through tensor fusion.

• Predicts chromatin accessibility from DNA sequence alone
• Infers proteomic abundance from transcriptomic data
• Uses cycle-consistency losses to ensure bidirectional mapping quality
• Tensor Fusion's interaction tensor can serve as a rich bottleneck for translation

Enables virtual assays when physical measurements are cost-prohibitive or sample-limited.

An alternative fusion paradigm using attention mechanisms to dynamically weigh modality importance, contrasting with Tensor Fusion's explicit multiplicative approach.

• Learns modality-wise attention scores for each prediction
• Prioritizes gene expression over methylation for expression QTL tasks
• Can be combined with Tensor Fusion as a hybrid architecture
• More interpretable but may miss fine-grained cross-modal interactions

Comparison: Attention selects modalities; Tensor Fusion explicitly models their interactions.