Model Predictive Control vs Reinforcement Learning for Bipedal Locomotion

[Model Predictive Control (MPC)] excels at real-time constraint satisfaction because it solves an open-loop optimization problem at every control cycle, explicitly respecting joint torque, friction cone, and kinematic limits. For example, in the DARPA Robotics Challenge, MPC-based controllers demonstrated a 99% step success rate on flat terrain by continuously replanning center of mass trajectories within a 1-2 second horizon, ensuring stability even under moderate external pushes.

[Reinforcement Learning (RL)] takes a fundamentally different approach by learning a closed-loop policy offline through trial and error in simulation. This results in policies that can achieve 15-30% greater energy efficiency than MPC baselines, as demonstrated by ETH Zurich's ANYmal research, where RL-trained policies learned to exploit natural system dynamics and passive ankle compliance in ways that hand-crafted MPC cost functions did not capture.

The key trade-off: If your priority is provable safety guarantees and constraint satisfaction for deployment in predictable, structured environments, choose MPC. If you prioritize agility, energy efficiency, and generalization to highly unstructured terrain where hand-engineering cost functions becomes intractable, choose RL. For many commercial humanoid platforms, a hybrid architecture using MPC as a safety filter around an RL nominal policy is emerging as the pragmatic standard.

Model Predictive Control (MPC) excels at real-time constraint satisfaction because it solves a finite-horizon optimization problem at each control cycle. For example, MIT's Cheetah 3 robot uses a convex MPC formulation running at 50 Hz to explicitly enforce friction cone constraints and torque limits, achieving a 99.8% success rate on stair climbing without visual perception. This mathematical guarantee of stability within the prediction horizon makes MPC the gold standard for safety-critical maneuvers where violating a joint limit or slipping is unacceptable.

Reinforcement Learning (RL) takes a fundamentally different approach by learning a policy offline through trial and error in simulation. This results in emergent behaviors that are difficult to hand-engineer. ETH Zurich's ANYmal robot, trained with RL in Isaac Gym, demonstrated a 25% improvement in energy efficiency and the ability to recover from pushes of up to 80 N—disturbances that would saturate a traditional MPC controller. However, these policies can produce unpredictable actions on out-of-distribution terrain, lacking the formal safety guarantees of optimization-based methods.

The key trade-off: If your priority is deterministic safety and real-time constraint enforcement for predictable environments, choose MPC. If you prioritize energy efficiency, dynamic maneuverability, and generalization to unstructured terrain, choose RL. For most commercial humanoids, the pragmatic path is a layered architecture: an RL policy generates a rich motion plan, while a low-level MPC controller acts as a safety filter, projecting actions onto a feasible set to prevent constraint violations. This combines the creativity of learning with the rigor of optimization.