One-Shot Imitation Learning

The foundational paradigm enabling one-shot learning. The agent is trained on a distribution of tasks during a meta-training phase, allowing it to quickly adapt to a novel task presented during meta-testing. This is typically implemented via Model-Agnostic Meta-Learning (MAML), which learns a parameter initialization that is sensitive to gradient updates from few examples. The process involves:

• Support Set: The single demonstration for the new task.
• Query Set: The agent's attempt to execute the task, used to compute the adaptation loss.
• Inner Loop: A few gradient steps on the support set to adapt to the new task.
• Outer Loop: Optimization of the initial parameters across many tasks to enable fast adaptation.

A neural network module that compresses the single demonstration into a task embedding or context vector. This embedding serves as a compact, informative representation of the goal, which conditions the policy. Architectures include:

• Temporal Convolutional Networks (TCNs): Process the sequence of demonstration states/actions.
• Recurrent Networks (LSTMs/GRUs): Encode variable-length demonstration trajectories.
• Transformers: Use self-attention to weigh important segments of the demonstration. The encoder's output is concatenated with the agent's current observation, providing the policy with explicit information about what to imitate.

The core control module that generates actions. It takes two inputs: the current environmental observation (e.g., robot joint angles, camera image) and the task embedding from the encoder. The policy is conditioned on this embedding, allowing the same network parameters to produce different behaviors for different tasks. Common implementations are:

• Multi-Layer Perceptrons (MLPs): For low-dimensional state spaces.
• Convolutional Neural Networks (CNNs): For processing pixel-based observations.
• Recurrent Policies: To maintain temporal coherence over long horizons. The policy is trained not for one specific task, but to be conditionally generalizable across the task distribution.

The specific format of the single example provided to the agent. This defines the interface for human or expert instruction. Key representations include:

• State-Action Trajectories: A sequence of <state, action> pairs, often from kinesthetic teaching or teleoperation.
• Visual Demonstrations: A video of the task being performed, requiring the agent to infer states and actions.
• Language-Conditioned: A natural language instruction paired with a demonstration, combining one-shot learning with language grounding.
• Keyframes or Waypoints: A sparse set of critical states, reducing demonstration complexity. The choice of representation directly impacts the difficulty of the embedding and generalization problem.

The objective that drives learning across the meta-training task distribution. It is designed to evaluate performance after the one-shot adaptation. The standard approach uses a meta-loss: the policy's performance on the query set of a task, after being adapted on that task's support set (the single demo). Common losses include:

• Behavior Cloning Loss: Mean-squared error between the policy's predicted actions and the demonstration's actions (on the query trajectory).
• Task Success Metric: A sparse reward (1 for success, 0 for failure) for goal-oriented tasks.
• Adversarial Loss: Used in frameworks like one-shot GAIL, where a discriminator tries to distinguish adapted policy trajectories from expert ones. The system is optimized so that a single gradient step on the demonstration leads to low loss on the query.

Critical infrastructure for scalable meta-training. Acquiring massive, diverse datasets of physical demonstrations is impractical. Therefore, training occurs in physics simulators (e.g., MuJoCo, PyBullet, Isaac Sim) where thousands of task variations can be generated automatically. To bridge the sim-to-real gap, Domain Randomization is applied:

• Visual Randomization: Varying textures, lighting, colors, and camera angles.
• Dynamics Randomization: Altering friction, mass, and motor strength parameters.
• Task Parameter Randomization: Changing object sizes, shapes, and initial/goal positions. This forces the meta-learner to extract invariant task concepts, resulting in a policy robust enough to transfer to a single real-world demonstration.

One-Shot Imitation Learning is fundamentally a meta-learning problem. The agent is trained on a distribution of tasks during a meta-training phase, learning a generalizable policy or embedding space. When presented with a single new demonstration during meta-testing, the agent uses this prior experience to infer the task's intent and generate appropriate actions. This is often implemented via model-agnostic meta-learning (MAML) or prototypical networks, where the model learns to quickly adapt its parameters.

The core technical implementation involves a conditional neural network policy. This policy takes two inputs:

• The current observation (e.g., camera image, robot state).
• An embedded demonstration, which is the encoded single example. The network is trained to produce actions that, when conditioned on demonstration A, perform task A, and when conditioned on demonstration B, perform the different task B. The embedding is typically produced by a demonstration encoder, often a recurrent or transformer network that processes the sequence of demonstration states.

In industrial automation, this technique allows a robot to learn new assembly or kitting routines from one human demonstration. For example:

• A worker shows the robot how to insert a specific component into a circuit board once.
• The robot generalizes from prior experience with other insertion tasks to perform this new variant, adjusting for slight differences in part orientation or receptacle location. This reduces programming time for new products from hours to minutes, enabling flexible, low-volume manufacturing.

A key application is in domestic robots that must handle a vast array of objects and tasks. From a single demonstration of a new chore, such as:

• Placing dishes in a novel rack
• Arranging groceries in a specific cupboard The robot leverages its prior meta-training on hundreds of other placement and manipulation tasks. It uses the demonstration to ground the goal state (the final arrangement) and selects appropriate visuomotor skills from its repertoire to achieve it, despite never seeing that exact object or location before.

A major challenge is the embodiment gap—the difference between the demonstrator (e.g., a human hand) and the robot. One-shot methods address this by learning task-level intent rather than low-level motion mimicry. The demonstration encoder extracts the abstract goal (e.g., 'open the drawer'), and the robot's policy executes it using its own kinematics. This is often combined with domain-invariant feature learning during meta-training to filter out embodiment-specific details.

Modern systems combine one-shot visual imitation with language models for richer task specification. The process becomes:

1. A human provides a language command (e.g., 'Tidy the red blocks').
2. They give a single visual demonstration of the tidying concept on a different set of objects.
3. The LLM helps interpret the command and align it with the demonstrated skill's intent.
4. The one-shot policy generalizes the demonstrated skill to the new objects mentioned in the command. This creates a powerful vision-language-action pipeline for learning from minimal examples.

Often called "learning to learn," meta-learning is the overarching paradigm that enables one-shot imitation. A meta-learner is trained on a distribution of tasks so it can quickly adapt to a new, unseen task with minimal data.

• Model-Agnostic Meta-Learning (MAML) is a key algorithm that learns an initial set of model parameters that are highly sensitive to loss gradients from new tasks, allowing fast fine-tuning.
• In the context of one-shot imitation, the meta-training tasks are different robotic skills, and the single demonstration provides the context for rapid adaptation.

A broader subfield of machine learning focused on learning from a very small number of labeled examples per class. One-shot learning is its most extreme case (N=1).

• Key techniques include metric-based learning (e.g., Siamese Networks, Prototypical Networks) which learn a distance function in an embedding space to compare new examples to a small support set.
• While few-shot learning is often applied to image classification, in robotics it translates to learning new manipulation or navigation tasks from a handful of demonstrations.

The core architectural component in one-shot imitation. This is a neural network policy conditioned on both the current environment observation and a task context derived from the single demonstration.

• The context is typically encoded into a fixed-length vector using a separate network, often called a context encoder or demonstration encoder.
• The policy must learn to interpret this context vector to modulate its behavior, allowing the same network parameters to execute a wide variety of tasks based on the provided example.

A low-dimensional vector representation that captures the essence of a task. In one-shot imitation, the demonstration is processed to produce a task embedding, which serves as the context for the policy.

• The embedding space is learned during meta-training so that similar tasks (e.g., "push blue block" and "push red block") have nearby embeddings, while dissimilar tasks (e.g., "push" vs. "stack") are farther apart.
• This structured representation is crucial for the policy's ability to generalize to new tasks not seen during meta-training.

The challenge of applying knowledge learned in a source domain (e.g., simulation, a specific robot arm) to a different but related target domain (e.g., real world, a different robot). This is a critical concern for deploying one-shot learned policies.

• Techniques like Domain Randomization are used during meta-training in simulation to create a vast distribution of visual and physical conditions, forcing the policy to learn domain-invariant features.
• The embodiment gap—differences in physical form between the demonstrator and learner—is a specific domain adaptation problem addressed through kinematic or dynamic mapping.

An alternative symbolic approach to one-shot learning, where the goal is to infer a general program or policy from a single example of its execution. The agent must reason about the underlying intent and constraints of the demonstration.

• This often involves inverse planning or searching a space of possible programs (e.g., in a domain-specific language) that are consistent with the observed trajectory.
• It contrasts with the neural network-based, sub-symbolic approach of contextual policies, offering potential advantages in interpretability and compositional generalization.