Dynamic Movement Primitive (DMP)

At the heart of a DMP is a canonical dynamical system, typically a simple linear system like a phase oscillator. This system provides a phase variable that monotonically drives the transformation from the start to the goal state, ensuring temporal stability and robustness. It abstracts time, allowing the movement's progression to be defined by the phase rather than absolute time, which is crucial for temporal scaling and pausing.

The shape of the learned movement is encoded in a nonlinear forcing function. This function is typically represented as a weighted sum of radial basis functions (RBFs) centered along the phase of the canonical system. During learning from a demonstration, the weights of these basis functions are adjusted to reconstruct the demonstrated trajectory. During execution, this function perturbs the simple dynamical system to produce the desired complex motion.

DMPs guarantee convergence to a specified goal through globally stable attractor dynamics. The transformation system is formulated as a second-order damped spring model:

• Goal Attractor: The system is inherently drawn to the goal state, acting as a point attractor.
• Damping Term: Provides critical damping to ensure smooth, non-oscillatory convergence. This stability makes DMPs robust to perturbations; if the system is pushed off trajectory, it will naturally converge back to the goal.

A primary advantage of DMPs is the ease of spatial and temporal scaling without re-learning. The formulation cleanly separates movement shape from its execution parameters:

• Spatial Scaling: The target goal g can be changed, and the entire trajectory scales accordingly while preserving its shape.
• Temporal Scaling: The time constant τ of the canonical system can be adjusted to execute the movement faster or slower. This allows a single learned DMP, like a reaching motion, to be reused for different targets and speeds.

DMPs can be extended with coupling terms to react to external sensory feedback or to synchronize multiple DMPs. These terms modify the transformation or canonical systems in real-time:

• Obstacle Avoidance: A coupling term can deflect the trajectory based on proximity to obstacles.
• Synchronization: The phase of multiple DMPs (e.g., for a robot arm) can be coupled to maintain coordination. This enables the generation of adaptive, reactive behaviors beyond simple open-loop playback of demonstrations.

While standard DMPs represent a single trajectory, Probabilistic Movement Primitives (ProMPs) extend the framework to model a distribution over trajectories. This is critical for imitation learning as it:

• Captures Variability: Models the natural variance observed across multiple demonstrations of a skill.
• Enables Conditioning: The distribution can be conditioned on via-points (intermediate points) or final goals.
• Facilitates Blending: Multiple skills represented as ProMPs can be combined probabilistically. ProMPs provide a more flexible and powerful representation for learning from demonstration datasets.

A Probabilistic Movement Primitive (ProMP) is a trajectory representation that models a distribution over movements, capturing variability and correlation across degrees of freedom and time. Unlike a standard DMP, which encodes a single trajectory, a ProMP can represent a family of similar movements, enabling:

• Modulation for new goals via conditioning.
• Blending of multiple skills.
• Uncertainty quantification in the reproduced motion. It is particularly useful for learning from multiple, slightly varied demonstrations and for tasks requiring adaptability to perturbations.

Trajectory Optimization is a planning technique that computes a sequence of states and actions minimizing a cost function while satisfying system dynamics and constraints. In relation to DMPs:

• It can be used to generate optimal demonstrations that DMPs then learn to reproduce.
• DMPs can serve as a compact, stable parameterization for the variables being optimized, reducing the search space.
• Methods like iterative Linear Quadratic Regulator (iLQR) are common for refining DMP parameters for complex, contact-rich tasks where pure imitation may be insufficient.

Behavior Cloning (BC) is a supervised learning approach where a policy is trained to map observed states directly to expert actions. DMPs are a common policy architecture used within the BC framework.

• The DMP provides the mathematical structure (the dynamical system), while BC provides the learning algorithm (regression on demonstration data).
• This combination allows for learning smooth, goal-directed movements from kinesthetic teaching or motion capture.
• A key limitation both face is compounding error, where small mistakes lead the agent into unseen states.

Inverse Reinforcement Learning (IRL) is the problem of inferring a reward function from observed expert behavior. It contrasts with DMP-based imitation learning:

• DMP/BC learns how to act (the policy).
• IRL learns why the expert acted (the reward), then uses RL to derive a policy.
• IRL is more general and can outperform BC when demonstrations are suboptimal or the environment changes, but it is computationally more complex. DMPs can be the output policy of an IRL process that has inferred a reward for trajectory smoothness and goal achievement.

Policy Distillation is a knowledge transfer technique where a complex "teacher" policy (or ensemble) is used to train a simpler "student" policy. DMPs can act as an effective student architecture in this process.

• A high-capacity neural network policy (the teacher) can be trained via RL or from vast demonstration data.
• Its behavior is then distilled into a DMP, resulting in a more interpretable, stable, and computationally efficient policy for deployment on robotic hardware.
• This combines the representational power of deep learning with the robustness and modularity of dynamical systems.

A Mixture of Experts (MoE) policy uses a gating network to select or combine outputs from multiple specialized sub-policies ("experts"). DMPs are well-suited to serve as these expert modules.

• Each DMP expert can encode a distinct, reusable motor primitive (e.g., "reach," "grasp," "place").
• A higher-level planner or learned gating network activates and sequences these DMPs to perform long-horizon tasks.
• This creates a hierarchical imitation learning system where complex behavior emerges from the composition of simple, stable dynamical systems.