Temporal Disentanglement

The fundamental mechanism that partitions the latent representation z into two independent subspaces: z_s (static) and z_d (dynamic).

• Static factors capture time-invariant attributes like patient demographics, equipment specifications, or system constants
• Dynamic factors encode time-varying behaviors such as trends, seasonality, and transient events
• Enforced through architectural constraints like separate encoder pathways or mutual information minimization
• Enables isolated attribution: a forecast change can be traced to either a fixed property or a temporal event

A statistical regularization technique that ensures the static and dynamic latent codes contain no shared information.

• Minimizes I(z_s; z_d) — the mutual information between the two subspaces
• Implemented via adversarial training, contrastive losses, or KL-divergence penalties
• Prevents information leakage where dynamic patterns inadvertently encode static properties
• Critical for out-of-distribution generalization: static factors remain stable when temporal dynamics shift

Most temporal disentanglement architectures extend the VAE framework to handle sequences.

• The prior p(z_s, z_d) is factorized as p(z_s) * p(z_d | z_s), reflecting that dynamics may depend on static properties
• The posterior q(z_s, z_d | x_{1:T}) is learned via recurrent or transformer encoders
• Reconstruction loss and KL-divergence terms are modified to enforce disentanglement
• Example: a patient's baseline health (static) influences but does not determine their vital sign trajectory (dynamic)

Self-supervised objectives that push apart representations of sequences with different static attributes while pulling together sequences sharing the same static factors.

• Positive pairs: augmented versions of the same sequence or sequences from the same subject
• Negative pairs: sequences from different subjects or with different static labels
• The InfoNCE loss encourages the static encoder to ignore temporal variations
• Produces a static code that is invariant to time-warping, cropping, or noise perturbations

Once disentangled, the latent space enables counterfactual reasoning by manipulating individual factors.

• Static intervention: modify z_s to simulate a different subject or system configuration while keeping dynamics fixed
• Dynamic intervention: alter z_d at specific time steps to test how temporal patterns influence predictions
• Generates counterfactual trajectories showing what the forecast would be under altered conditions
• Used in finance to isolate the impact of a policy change (static) from market volatility (dynamic)

Theoretical conditions under which the true static and dynamic factors can be recovered up to permutation and scaling.

• Requires auxiliary variables like domain labels or time indices to break symmetry
• Nonlinear ICA theory proves identifiability when the data distribution is conditionally factorial
• Practical implementations use iVAE or TCL (Temporal Contrastive Learning) frameworks
• Without identifiability, disentangled representations may not correspond to real-world concepts

The foundational generative framework often used to enforce disentanglement. In temporal contexts, VAEs are modified to factorize the latent space by conditioning dynamic variables on previous time steps while keeping static variables constant across a sequence. The KL divergence term in the evidence lower bound (ELBO) is structured to penalize interdependence between latent factors, encouraging the model to learn statistically independent representations for time-invariant and time-varying attributes.

A class of generative models that combine state-space models with neural networks to capture temporal dependencies. Architectures like the Kalman Variational Autoencoder explicitly model a latent state that evolves via linear dynamics, naturally separating a global static latent vector from a local dynamic latent sequence. This structural inductive bias makes attribution trivial: the static latent code explains time-invariant behavior, while the dynamic transition matrix explains temporal evolution.

A core optimization objective that forces independence between latent factors. Techniques like Contrastive Log-ratio Upper Bound (CLUB) or adversarial training are employed to estimate and minimize the mutual information between static and dynamic latent codes. By driving I(z_static; z_dynamic) toward zero, the model guarantees that the static code captures only time-invariant attributes (e.g., patient ID, sensor calibration) and the dynamic code captures only temporal fluctuations (e.g., heart rate variability, vibration patterns).

A technique using a Gradient Reversal Layer (GRL) to purge time-variant information from static representations. A discriminator network is trained to predict the time step or sequence identity from the static latent code, while the encoder is adversarially trained to fool this discriminator. This minimax game ensures the static code becomes invariant to temporal position, isolating purely persistent attributes. This is critical for domain adaptation and fair representation learning in time-series.

A self-supervised approach that learns factorized representations by defining positive and negative pairs in time. The model is trained to maximize agreement between representations of temporally close segments (dynamic similarity) while ensuring a global summary vector remains invariant across distant segments (static consistency). Frameworks like CPC (Contrastive Predictive Coding) can be extended to disentangle 'slow' features (static) from 'fast' features (dynamic) using different timescale contrastive losses.

Specific neural network designs that structurally separate static and dynamic processing streams. An encoder might use a Transformer to generate a static summary token and an LSTM to produce a sequence of dynamic residuals. The decoder then reconstructs the input using both codes. This explicit architectural factorization provides a strong inductive bias, making the latent traversal interpretable: modifying the static token changes global identity, while modifying the dynamic sequence alters local temporal patterns.