Disentanglement

Each latent dimension corresponds to exactly one generative factor of variation. Changing a single latent code alters only one semantic attribute of the generated output, with no unintended side effects. This is the most stringent requirement and is often evaluated by traversing individual latent dimensions and observing the decoded output.

• A dimension controlling object rotation should not affect color or size
• Violated when a single dimension simultaneously changes lighting and texture
• Tested via latent traversal visualizations and qualitative inspection

A single generative factor is encoded by a small number of latent dimensions, ideally just one. Compactness ensures that all information about a semantic attribute is concentrated rather than diffused across the entire latent space. This property enables efficient manipulation and reduces the dimensionality required for downstream tasks.

• A smile attribute should be captured by one dimension, not distributed across dozens
• Measured by the mutual information gap between latent codes and ground-truth factors
• Promotes sparse, efficient representations for transfer learning

The mapping from latent code to generative factor is simple and monotonic, typically linear. A linear classifier or regressor trained on the latent representation should easily predict the corresponding ground-truth factor. Explicitness ensures that downstream models can readily access the encoded information without complex non-linear decoding.

• A linear probe on the latent space should achieve high accuracy for factor prediction
• Evaluated using the DCI (Disentanglement, Completeness, Informativeness) framework
• Non-linear relationships indicate residual entanglement or poor representation structure

Latent dimensions are mutually independent under the model's aggregate posterior distribution. This is typically enforced by encouraging a factorized prior, such as an isotropic Gaussian, and penalizing total correlation. Statistical independence prevents dimensions from encoding redundant or correlated information.

• Enforced via KL divergence with a factorized prior in β-VAE variants
• Measured by the Mutual Information Gap (MIG) metric
• A high MIG score indicates each factor is uniquely captured by a single latent code

Every generative factor present in the data is captured by at least one latent dimension. Completeness ensures no semantic attribute is lost or ignored during encoding. A representation that fails completeness will exhibit blind spots where certain data variations cannot be controlled or predicted from the latent space.

• Evaluated by training regressors from the latent space to all known ground-truth factors
• A low completeness score indicates the encoder is discarding relevant information
• Critical for applications requiring full semantic control over generated outputs

A truly disentangled representation supports compositional generalization. Because factors are encoded independently, the decoder should plausibly render novel combinations of attribute values never seen during training. This property is the primary practical motivation for learning disentangled representations.

• A model trained on red triangles and blue squares should generate a blue triangle
• Tested using held-out factor combinations during evaluation
• Indicates the model has learned the true causal structure rather than memorizing correlations