Representation Learning

A paradigm where a model generates its own supervisory signal from the structure of unlabeled data. This is achieved by defining a pretext task that forces the model to learn useful features.

• Example: In natural language processing, models like BERT are trained by predicting masked words in a sentence.
• Core Mechanism: The model learns by solving an auxiliary prediction problem, with the resulting representations proving highly effective for downstream tasks like classification.
• Benefit: It leverages vast amounts of unlabeled data, reducing dependency on expensive human annotations.

A self-supervised technique that learns representations by teaching a model to distinguish between similar (positive) and dissimilar (negative) data pairs.

• Core Objective: Minimize the distance between embeddings of positive pairs (e.g., two augmented views of the same image) while maximizing the distance from negative pairs.
• Key Framework: InfoNCE Loss (Noise-Contrastive Estimation) is a common objective function used.
• Application: Pioneered by models like SimCLR and MoCo for computer vision, it creates robust visual embeddings without class labels.

Techniques where a model learns the underlying probability distribution p(x) of the training data, enabling it to generate new, plausible samples.

• Variational Autoencoders (VAEs): Learn a latent space by encoding data into a distribution and decoding from it, optimized via the Evidence Lower Bound (ELBO).
• Generative Adversarial Networks (GANs): Use a generator and a discriminator in an adversarial game, where the generator learns to produce realistic data.
• Diffusion Models: Iteratively denoise data from pure noise, learning a complex data distribution through a Markov chain process.

The goal of encoding distinct, semantically meaningful factors of variation in the data into separate and independent dimensions of the latent space.

• Ideal Outcome: A single latent dimension controls one interpretable attribute (e.g., object size, rotation, color), while being invariant to others.
• Challenge: Achieving perfect disentanglement is an open research problem, often requiring specific inductive biases or regularization like the β-VAE objective.
• Utility: Enables controllable generation, improved interpretability, and more efficient downstream task learning.

In reinforcement learning, this involves an agent learning an explicit world model—a predictive representation of environment dynamics (transition function) and rewards.

• Function: The agent uses this internal model for planning (e.g., via Model Predictive Control or Monte Carlo Tree Search) to simulate trajectories and select optimal actions without costly real-world interaction.
• Architecture: Often implemented as a recurrent state-space model (RSSM) that learns a latent state representation to predict future observations and rewards.
• Benefit: Dramatically improves sample efficiency compared to model-free RL.

A paradigm where a model is encouraged to decompose a complex scene into a structured set of entities or 'slots,' each representing a distinct object or concept.

• Objective: To discover object-centric representations where each entity's latent code captures its properties (shape, color, position) independently.
• Methods: Include architectures like Slot Attention and MONet that use iterative attention to assign image features to slots.
• Significance: Mimics human perception, enabling compositionality, systematic generalization, and improved reasoning about object interactions and physics.

A latent space is a lower-dimensional, continuous vector space where learned representations of data reside. It captures the essential factors of variation from the high-dimensional raw input.

• Key Property: Enables semantic operations like interpolation (morphing between concepts) and arithmetic (e.g., 'king' - 'man' + 'woman' = 'queen').
• Foundation for Generation: Serves as the source distribution for generative models like VAEs and GANs to create new data samples.
• Example: In an image model, a single point in latent space might encode the concept of 'a red car at a 45-degree angle'.

Self-supervised learning is a paradigm where a model generates its own supervisory signal from the structure of unlabeled data, making it the dominant method for pre-training foundational representations.

• Core Mechanism: Creates a pretext task by masking, distorting, or predicting parts of the input (e.g., predicting a masked word in BERT, or the next frame in a video).
• Data Efficiency: Leverages vast amounts of unlabeled data, which is far more abundant than costly human-annotated data.
• Contrastive Learning: A popular self-supervised technique that learns by pulling similar data points closer and pushing dissimilar ones apart in the embedding space.

A disentangled representation is a latent space where distinct, semantically meaningful factors of variation in the data are encoded in separate, statistically independent dimensions.

• Goal: To isolate attributes like object shape, size, color, position, and lighting in an image, or sentiment and topic in text.
• Benefit: Enables precise, interpretable control over generated outputs and improves robustness to distribution shifts.
• Challenge: Achieving perfect disentanglement is an open research problem, often formalized using metrics like the β-VAE framework which penalizes the KL divergence between the latent distribution and a factorized prior.

Contrastive learning is a self-supervised technique that learns representations by training a model to distinguish between similar (positive) and dissimilar (negative) data pairs.

• Process: An encoder network produces embeddings. The loss function (e.g., InfoNCE) minimizes the distance between embeddings of augmented views of the same instance (positives) while maximizing distance to embeddings of other instances (negatives).
• Frameworks: Includes methods like SimCLR, MoCo, and CLIP (which contrasts images with text captions).
• Outcome: Creates a semantically structured embedding space where similarity in the space reflects semantic relatedness in the data.

A world model is an internal, learned representation within an agent that captures the dynamics and regularities of its environment, enabling prediction and planning.

• Function: It acts as a 'simulator in the mind,' allowing the agent to imagine consequences of actions without costly real-world interaction. This is central to model-based reinforcement learning.
• Implementation: Often built using recurrent neural networks (e.g., RSSM - Recurrent State-Space Model) that learn to encode a latent state and predict future latent states and rewards.
• Application: Critical for sample-efficient learning in robotics, autonomous driving, and game AI (e.g., DeepMind's Dreamer).

A POMDP is the formal mathematical framework for sequential decision-making under uncertainty, where the agent cannot directly observe the true state of the environment.

• Core Components: Includes states, actions, observations, transition probabilities, reward function, and an observation function. The agent maintains a belief state—a probability distribution over possible true states.
• Connection to Representation Learning: The agent's task is to learn a representation (the belief state) from its history of observations and actions. Recurrent world models are a practical, learned approximation for solving POMDPs.
• Use Case: The standard model for real-world robotics, dialogue systems, and medical diagnosis, where sensors provide incomplete information.