Model Predictive Control (MPC)

MPC's foundation is an explicit mathematical model of the system being controlled. This model predicts how the system's states will evolve over time in response to potential control inputs. Common model types include:

• Linear Time-Invariant (LTI) Models: Simplest form, using state-space or transfer function representations.
• Nonlinear Models: Necessary for systems with complex dynamics (e.g., robotics, chemical processes), though they increase computational cost.
• Data-Driven Models: Such as neural networks, used when first-principles models are unavailable. The accuracy of this model is paramount, as prediction errors directly degrade control performance, making system identification a critical precursor step.

At each control step, MPC solves a constrained optimization problem over a finite future window called the prediction horizon (N). The objective is to find a sequence of future control actions that minimizes a cost function, typically quantifying:

• Tracking Error: Deviation from a desired reference trajectory.
• Control Effort: Magnitude of actuator commands.
• Constraint Violations: Penalties for exceeding limits. The optimization must respect hard constraints (e.g., actuator saturation, safety limits) and soft constraints. This allows MPC to proactively 'plan' optimal actions, unlike reactive controllers.

MPC implements a receding (or moving) horizon strategy. After solving the optimization:

1. Only the first control action in the computed optimal sequence is applied to the real system.
2. The system state is measured (or estimated) again.
3. The horizon 'recedes' one step into the future, and a new optimization is solved from the new current state. This feedback mechanism provides inherent robustness to model inaccuracies and disturbances, as the controller constantly re-plans based on updated measurements, correcting for prediction errors.

A defining strength of MPC is its ability to explicitly and systematically handle constraints. These are directly incorporated into the online optimization problem. Key constraint types include:

• Input Constraints: Limits on actuator range (e.g., voltage, torque).
• State Constraints: Safety or operational limits on system variables (e.g., temperature, position).
• Output Constraints: Limits on measured outputs. By solving constrained optimization, MPC can operate systems at their very limits without violation, enabling performance maximization while guaranteeing safety, which is critical for sim-to-real transfer where real hardware has physical limits.

MPC naturally handles Multiple-Input, Multiple-Output (MIMO) systems. The optimization framework can simultaneously coordinate numerous actuators to control multiple interdependent states or outputs. This is a significant advantage over single-loop PID controllers, which struggle with cross-coupling between variables. For example, in a drone, MPC can jointly optimize rotor speeds to control position, attitude, and velocity in all three axes, accounting for the complex interactions between them through the system model.

The primary challenge of MPC is its computational intensity. Solving an optimization problem at every control time step (often at 10-1000 Hz) requires significant processing power. Its real-time viability depends on:

• Model Complexity: Linear Quadratic MPC is fast; Nonlinear MPC (NMPC) is much slower.
• Horizon Length: Longer horizons increase problem size.
• Solver Efficiency: Specialized Quadratic Programming (QP) solvers for linear MPC and gradient-based solvers for NMPC. Advances in embedded computing, code generation, and specialized hardware have made MPC feasible for applications from automotive engine control to fast robotic manipulators, closing the loop between complex optimization and physical actuation.

MPC acts as a powerful bridge in the sim-to-real pipeline. A common architecture involves:

1. Training a high-level neural network policy in simulation using Reinforcement Learning (RL).
2. Deploying this policy to output a reference trajectory or goal state for a real-time MPC controller on the physical robot.

• The MPC 'wrapper' provides crucial safety filtering and low-level robustness, compensating for the reality gap (dynamics mismatch, sensor noise) that the RL policy did not encounter in simulation.
• This hybrid approach combines the adaptability of learned policies with the constraint satisfaction and predictability of model-based MPC.

MPC scales to coordinate fleets of robots by solving a coupled optimization problem.

• In swarm robotics, a centralized or distributed MPC framework plans paths for all agents to achieve a collective goal (e.g., formation flying, area coverage) while avoiding inter-agent collisions.
• For heterogeneous fleets (e.g., AMRs and human-operated vehicles in a warehouse), MPC can predict the intentions of other agents and plan compliant, efficient paths.
• The optimization includes spatiotemporal constraints to deconflict trajectories, making it fundamental for automated logistics and large-scale autonomous systems.

A machine learning paradigm where an agent learns a policy—a mapping from environmental states to actions—by maximizing a cumulative reward signal through trial-and-error interaction. Unlike MPC's explicit online optimization, RL policies are typically neural networks trained offline to encode optimal behavior.

• Key Distinction from MPC: RL learns a reactive policy; MPC solves an optimization problem at each timestep.
• Synergy with MPC: RL can be used to learn the dynamics model or terminal cost function for an MPC controller, a method known as model-based RL.

The mathematical framework for designing control laws that drive a dynamical system to minimize a cost function (or maximize a reward) over time. MPC is a specific, receding-horizon implementation of optimal control.

• Core Problem: Find a control sequence u(t) that minimizes J = ∫ L(x(t), u(t)) dt subject to system dynamics dx/dt = f(x, u) and constraints.
• Open-Loop vs. Closed-Loop: Classical optimal control often computes a single trajectory. MPC is closed-loop optimal control because it re-solves the problem at each step with new feedback.

A foundational optimal control algorithm for linear systems with quadratic cost functions. LQR provides an analytical, closed-form solution for the optimal feedback control law u = -Kx, where K is a constant gain matrix.

• Comparison to MPC: LQR is computationally lightweight and globally optimal for its assumptions but cannot handle general non-linear dynamics or hard state and input constraints. MPC generalizes this concept to non-linear, constrained problems via numerical optimization.

The process of inferring the complete, internal state of a dynamical system (e.g., a robot's position, velocity, orientation) from partial, noisy sensor measurements. MPC requires an accurate state estimate as the initial condition for its prediction horizon.

• Common Algorithms: Kalman Filters (for linear systems), Extended Kalman Filters (EKF), and Particle Filters (for non-linear, non-Gaussian systems).
• Critical Role: Poor state estimation directly corrupts MPC's predictions, leading to suboptimal or unstable control actions.

The discipline of building or calibrating mathematical models of physical systems from observed input-output data. For MPC, this is the process of learning or refining the dynamics model f(x, u) used for prediction.

• Methods Range from classical linear regression for simple models to neural networks for capturing complex, non-linear dynamics.
• Sim-to-Real Context: System ID is used to minimize the reality gap by tuning simulation parameters to match real-world robot data, creating a more accurate model for MPC.

The computational engine at the heart of MPC. At each control step, MPC formulates a constrained optimization problem over future actions, which is solved by an NLP solver.

• Real-Time Requirement: Solvers must find a good solution within the controller's fixed sampling period (often milliseconds).
• Common Techniques: Interior-point methods, Sequential Quadratic Programming (SQP), and Augmented Lagrangian methods. Specialized solvers like ACADO, CasADi, and FORCES Pro are designed for real-time MPC.