Model Predictive Control (MPC) for Manipulation

This is the core mechanism of MPC. At each control time step, the controller:

• Solves a finite-horizon optimal control problem over a future time window (the prediction horizon).
• Applies only the first control input from the computed optimal sequence.
• Repeats the process at the next time step with updated state feedback, effectively 'receding' the horizon forward.

This creates a closed-loop feedback policy that is continuously re-optimized based on the latest sensor data, making it highly reactive to disturbances and model inaccuracies.

A defining strength of MPC is its ability to explicitly encode hard and soft constraints directly into the optimization problem. For robotic manipulation, critical constraints include:

• Joint Limits: Angular position, velocity, and torque limits for each actuator.
• Obstacle Avoidance: Ensuring the end-effector and arm links do not collide with the environment.
• Force Limits: Constraining contact forces to prevent damage to the robot or object.
• Workspace Boundaries: Keeping the end-effector within a defined operational volume.

Unlike reactive methods that handle constraints post-hoc, MPC plans trajectories that are feasible by construction, enhancing safety and reliability.

MPC's predictive capability is grounded in an internal dynamic model of the system. For manipulation, this model typically includes:

• Robot Dynamics: Equations of motion (inverse dynamics) relating joint torques to acceleration.
• Contact Models: Simplified models of interaction forces (e.g., spring-damper) for tasks like insertion or pushing.
• Object Dynamics: Models of how manipulated objects move in response to forces.

The fidelity of this model directly impacts performance. While complex non-linear models are most accurate, Linear Time-Varying (LTV) or Linear Parameter-Varying (LPV) approximations are often used to keep the optimization tractable for real-time control at kHz rates.

The MPC optimizer minimizes a scalar cost function that encodes competing task objectives. For a manipulation task, this cost is a weighted sum of terms such as:

• Tracking Error: Deviation from a desired end-effector trajectory or pose.
• Control Effort: Minimization of joint torques or energy consumption.
• Terminal Cost: A penalty on the predicted state at the end of the horizon to ensure stability.
• Smoothness: Penalties on jerk or high accelerations for graceful motion.

Engineers tune the weights to balance performance (speed/accuracy) against factors like energy use and wear on the hardware.

The practical deployment of MPC hinges on solving the optimization problem within the control loop's fixed period (often 1-10 ms). This is enabled by:

• Convex Formulations: Casting the problem as a Quadratic Program (QP) or Second-Order Cone Program (SOCP) for which fast, reliable solvers exist (e.g., OSQP, qpOASES).
• Warm Starting: Using the solution from the previous time step as an initial guess, drastically reducing solver iterations.
• Hardware Acceleration: Implementing solvers on FPGAs or GPUs for the most demanding applications.

The rise of efficient numerical solvers has been a key driver in bringing MPC from theory to real-world robotic applications.

MPC provides inherent robustness to uncertainties through its feedback mechanism and can be extended for explicit disturbance handling:

• Feedback Re-planning: The receding horizon approach constantly corrects for deviations caused by unmodeled dynamics, slipping objects, or external pushes.
• Robust MPC Variants: Advanced formulations like Tube MPC or Stochastic MPC explicitly account for bounded disturbances or noise within the optimization, computing policies that guarantee constraint satisfaction under uncertainty.
• State Estimation Integration: MPC is typically paired with a state estimator (e.g., a Kalman Filter) that provides a best-guess of the current state from noisy sensor data (joint encoders, vision, force/torque sensors).

MPC excels at intercepting moving objects, a task requiring precise prediction and rapid replanning. The controller uses a model of the object's ballistic trajectory and the robot's arm dynamics to solve for the optimal joint motions that will place the end-effector at the predicted interception point at the correct time.

• Key Challenge: Compensating for sensor latency and actuator dynamics.
• Real-World Example: A robotic arm catching a ball thrown in an arc, where the MPC continuously updates the catch point based on the ball's observed flight path.

This classic industrial task demonstrates MPC's ability to handle contact constraints. The controller's internal model includes both the robot's dynamics and a representation of the contact forces between the peg and the hole.

• How it works: The MPC solves for control inputs that minimize a cost function combining position error and contact force, allowing the peg to slide and tilt compliantly into place without jamming.
• Advantage over traditional methods: Proactively plans force-guided motions over a horizon, rather than reacting to force errors with simple feedback.

For robots like Boston Dynamics' Atlas, MPC coordinates whole-body motion. The controller must balance multiple, often competing, objectives: reaching for an object while maintaining balance, avoiding joint limits, and preventing self-collision.

• Hierarchical MPC: Often implemented with a high-level MPC for centroidal dynamics (balance) and a low-level one for full-body kinematics.
• Output: An optimized sequence of foothold placements, torso motions, and arm trajectories that satisfy all dynamic constraints over the next few seconds.

Manipulating fluids and granular materials requires controlling non-rigid dynamics. An MPC for pouring uses a simplified model of the liquid's flow (e.g., based on container tilt angle and time) to predict the poured volume.

• Application: Precisely filling a cup to a target volume. The MPC plans the wrist rotation trajectory, potentially pausing or adjusting the tilt rate based on its prediction to hit the volume target exactly.
• Complexity: The model is inherently approximate, making the feedback correction at each MPC step critical for accuracy.

Unlike pre-computed path planning, MPC enables real-time reaction to unexpected obstacles. The robot's dynamic model and a representation of the obstacle's position/velocity are used within the optimization.

• Process: At each control cycle, the MPC predicts the future path of both the robot and the moving obstacle. If a collision is predicted within the horizon, the optimizer finds a new control sequence that deviates minimally from the goal while avoiding the obstacle.
• Result: Smooth, anticipatory evasion maneuvers instead of jerky, last-moment stops.

Controlling two arms to manipulate a single large or flexible object (e.g., a panel, a hose) is a complex underactuated problem. A centralized MPC treats both arms and the object as a single coupled dynamic system.

• Model: Includes the kinematics/dynamics of both manipulators and a physical model of the internal forces within the grasped object.
• Optimization: Solves for the coordinated control inputs to both arms that move the object along a desired trajectory while maintaining a stable grasp and avoiding excessive internal forces that could damage the object.

Trajectory optimization is the mathematical process of finding a sequence of robot states and control inputs that minimizes a cost function (e.g., energy, jerk, time) while satisfying dynamic constraints and avoiding obstacles. It is the core numerical engine inside an MPC loop. For manipulation, this involves planning smooth, feasible paths for the end-effector and joints.

• Direct vs. Indirect Methods: Direct methods (like direct collocation) transcribe the continuous problem into a large, sparse nonlinear program (NLP) solved by optimizers like IPOPT or SNOPT.
• Application: Used offline for pre-computed motions or online within receding-horizon MPC to react to new sensor data.

Whole-Body Control (WBC) is a hierarchical control framework for complex robots (e.g., humanoids, mobile manipulators) that coordinates all degrees of freedom to execute multiple, potentially conflicting tasks simultaneously. For a manipulator on a mobile base, WBC might solve for joint torques that prioritize end-effector tracking while maintaining balance and avoiding joint limits.

• Task-Priority Inverse Dynamics: Formulates control as a quadratic program (QP) that solves for optimal joint accelerations/torques, strictly enforcing a hierarchy of tasks (e.g., balance > manipulation > posture).
• Relation to MPC: WBC is often used for local, high-frequency control (1kHz), while MPC provides the reference trajectory and higher-level plan over a longer horizon (100ms).

Task and Motion Planning (TAMP) is an integrated AI planning paradigm that combines discrete, symbolic task planning ("what to do") with continuous motion planning ("how to move"). For manipulation, this means reasoning about sequences of high-level actions (pick, place, push) while ensuring each action has a geometrically and dynamically feasible motion.

• Symbolic Grounding: A TAMP solver must ground abstract symbols (e.g., ON(A, B)) into concrete robot configurations and collision-free trajectories.
• Relation to MPC: MPC operates at the lower level of this stack. A TAMP system might generate a sequence of sub-goals (e.g., grasp poses, placement locations), which are then executed and stabilized in real-time by an MPC controller that handles dynamic uncertainty and disturbances.

Impedance Control and Admittance Control are two fundamental strategies for achieving compliant, force-sensitive interaction with the environment—a critical requirement for robust manipulation.

• Impedance Control: The controller modulates the robot's dynamic response (mass-spring-damper behavior) to external forces. It commands torque/force to achieve a desired relationship between position error and contact force.
• Admittance Control: The controller uses a force/torque sensor to measure contact and then generates a motion command (position or velocity) accordingly. It specifies how the robot should move in response to force.
• Integration with MPC: Modern MPC formulations can directly incorporate contact force constraints and hybrid dynamics, unifying trajectory optimization with impedance-like behavior. This is known as Model Predictive Impedance Control.

Reinforcement Learning (RL) is a machine learning paradigm where an agent learns a control policy through trial-and-error interaction to maximize a reward signal. In robotics, Deep RL uses neural networks to represent complex policies that map states to actions.

• Model-Based vs. Model-Free RL: Model-Based RL learns or uses an internal dynamics model (like MPC) for planning. Model-Free RL (e.g., PPO, SAC) learns a policy directly without an explicit model.
• Synergy with MPC: Learned models (neural networks) can be used as the predictive model within an MPC framework, capturing complex, non-linear dynamics that are difficult to model analytically. Conversely, MPC can provide expert demonstrations or a guidance cost to accelerate and stabilize RL training, an approach known as RL with MPC priors.

Sim-to-Real Transfer is the methodology of training robotic control policies (including MPC controllers) in high-fidelity physics simulation and then deploying them on physical hardware. This is essential for data-efficient and safe development of manipulation skills.

• Domain Randomization: A key technique where simulation parameters (e.g., friction, object mass, lighting, sensor noise) are varied during training. This forces the controller (or the model inside an MPC) to become robust to inaccuracies in the simulation model, bridging the reality gap.
• Application to MPC: The dynamics model used for prediction in MPC can be a physics engine (e.g., MuJoCo, Bullet) or a neural network trained on randomized simulation data. This allows the MPC to make accurate predictions despite real-world variations, enabling successful transfer without fine-tuning.