Model Predictive Control (MPC)

MPC's core is an explicit mathematical model of the system's dynamics. This model, which can be linear, nonlinear, or data-driven, is used to predict the future trajectory of the system's states over a finite horizon based on current conditions and proposed control actions. The accuracy of this internal model directly determines the controller's performance.

• Types of Models: First-principles equations, linear state-space models, or learned models (e.g., neural networks).
• Purpose: Enables the controller to simulate 'what-if' scenarios before applying any real-world action.

At each control step, MPC solves a constrained optimization problem over a finite future window (the prediction horizon, N). It computes a sequence of future control actions that minimizes a cost function (e.g., tracking error, energy use) while respecting system constraints (e.g., actuator limits, safety bounds).

• Cost Function: Typically penalizes deviation from a setpoint and excessive control effort.
• Constraints: Hard limits on inputs/states are embedded directly into the optimization, a key advantage over PID control.

MPC implements a receding (or moving) horizon strategy. After solving the optimization, only the first control action in the computed sequence is applied to the real system. At the next time step, the horizon 'recedes' by one step, the system's new state is measured or estimated, and the optimization is solved again with updated information.

• Feedback Mechanism: This closed-loop feedback corrects for model inaccuracies and unmeasured disturbances.
• Real-time Requirement: The optimization must be solved within one sampling period.

A defining strength of MPC is its ability to systematically handle constraints on inputs (e.g., valve saturation), states (e.g., temperature limits), and outputs. Constraints are included directly as inequalities in the online optimization problem. This allows the controller to operate safely at the very limits of performance.

• Hard vs. Soft Constraints: Critical limits are enforced as hard constraints; less critical ones can be softened with penalty terms.
• Proactive Avoidance: The predictive model allows the controller to anticipate and avoid future constraint violations.

MPC naturally extends to Multi-Input, Multi-Output (MIMO) systems. The optimization framework simultaneously considers all manipulated variables and all controlled outputs, explicitly accounting for the couplings and interactions between them. This is a significant advantage over decentralized single-loop controllers which struggle with interaction.

• Example: In a chemical reactor, MPC can coordinate temperature, pressure, and flow rate adjustments to achieve multiple product quality targets.

The primary challenge of MPC is its computational intensity. Solving an optimization problem online, at every control step, requires significant processing power. Its viability depends on:

• Model Complexity: Linear Quadratic MPC (LQ-MPC) is fast; Nonlinear MPC (NMPC) is far more demanding.
• Hardware Advances: Deployment has been enabled by faster processors and specialized quadratic programming (QP) solvers.
• Approximation Techniques: Methods like explicit MPC pre-solve the optimization offline for online table lookup.

MPC is the dominant advanced control strategy in process industries like chemicals, oil refining, and pharmaceuticals. It optimizes continuous operations by:

• Predicting future states of reactors, distillation columns, and heat exchangers.
• Enforcing critical safety and operational constraints (e.g., temperature, pressure limits) within the optimization loop.
• Rejecting process disturbances and managing interactions between hundreds of coupled variables. Its ability to handle multi-input, multi-output (MIMO) systems with hard constraints makes it superior to traditional PID controllers for complex, economically significant processes.

Self-driving cars use MPC for real-time motion planning and control. The controller:

• Uses a kinematic or dynamic vehicle model to predict the car's path over a short horizon (1-3 seconds).
• Optimizes steering, throttle, and brake inputs to follow a reference path while maximizing passenger comfort and safety.
• Continuously re-plans at high frequency (10-100 Hz) to react to other vehicles, pedestrians, and road conditions. This application highlights MPC's strength in constrained optimization under uncertainty, where the 'model' is the physics of the vehicle and the 'constraints' are lane boundaries and collision avoidance.

In robotics, MPC enables dynamic, torque-controlled manipulation for tasks requiring contact and force interaction.

• Quadruped and Bipedal Locomotion: Robots like Boston Dynamics' Atlas use variants of MPC to compute optimal foot placements and body motions for dynamic walking and running, maintaining balance under external pushes.
• High-Speed Manipulation: Robotic arms in manufacturing use MPC for pick-and-place and assembly tasks, optimizing trajectories to minimize time while respecting joint torque and acceleration limits.
• Human-Robot Interaction: MPC can incorporate predictive models of human motion to enable safe and fluid collaboration, anticipating human movements to avoid collisions.

MPC is critical for managing complex, interconnected energy systems with intermittent renewable sources.

• Microgrid Control: Optimizes the dispatch of solar panels, wind turbines, and battery storage to meet demand, minimizing cost and carbon footprint while maintaining grid stability.
• Building Climate Control: Manages HVAC systems across large buildings by predicting thermal dynamics and occupancy patterns, reducing energy consumption by 20-40% compared to rule-based systems.
• Power Electronics: Controls voltage and frequency in inverters for renewable integration, using fast MPC implementations to handle switching constraints in real-time. These applications leverage MPC's ability to perform economic optimization (minimizing cost) while satisfying physical and contractual constraints.

MPC provides robust control for aircraft, spacecraft, and drones operating in challenging conditions.

• Quadrotor Drones: Uses a relatively simple dynamic model to achieve aggressive, acrobatic flight and precise hovering, even in windy conditions. The receding horizon allows for real-time obstacle avoidance.
• Rocket Landing: SpaceX's Falcon 9 uses MPC-like algorithms for propulsive landing. The controller solves a constrained optimization problem to guide the rocket to the landing pad while managing fuel (mass) dynamics, thrust limits, and attitude.
• Missile Guidance: Implements model predictive guidance where the model includes target motion prediction, leading to higher interception probabilities against maneuvering targets. This domain emphasizes MPC's performance in highly nonlinear, safety-critical systems.

MPC personalizes treatment by modeling patient physiology as a dynamic system to be controlled.

• Artificial Pancreas for Diabetes: A premier example. The controller uses a glucose-insulin model of the patient to predict future blood sugar levels. It then computes optimal insulin pump delivery rates, balancing the need to lower glucose with the risk of hypoglycemia. It operates as a closed-loop system, automating what was a manual decision process.
• Anesthesia Delivery: Regulates the infusion of anesthetic drugs (e.g., propofol) during surgery to maintain a target depth of anesthesia, modeled from measured brain signals (like BIS index), while accounting for patient-specific pharmacokinetics and dynamics.
• Cardiac Assist Devices: Optimizes the operation of ventricular assist devices (VADs) to support heart function, adapting to the patient's changing circulatory demands. These applications are distinguished by their use of stochastic, individualized models and extremely high safety-critical constraints.

A POMDP is the foundational mathematical framework for sequential decision-making under uncertainty where the agent cannot directly observe the true environment state. It formalizes the core challenge MPC addresses:

• Belief State: The agent maintains a probability distribution over possible states.
• Observation Model: Defines the relationship between the true state and noisy sensor data.
• Planning Horizon: The agent optimizes actions over a future time window, balancing immediate and long-term rewards. MPC is often viewed as an online, receding-horizon approximation for solving POMDPs, using an explicit dynamics model for prediction.

Model-Based Reinforcement Learning is a paradigm where an agent learns an explicit model of the environment's transition dynamics and reward function, then uses this model for planning. MPC is a specific, widely used control strategy within MBRL.

Key distinctions:

• MPC typically uses a known or identified model (e.g., physics-based) and focuses on short-horizon, online optimal control.
• MBRL often emphasizes learning the model from interaction data and may use the model for longer-horizon policy optimization. Both share the core loop: use a model to simulate futures, optimize actions, execute, and re-plan.

Optimal Control is the broader field of engineering and mathematics concerned with finding a control law for a dynamical system that minimizes a cost function over time. MPC is a subfield of optimal control characterized by its receding horizon approach.

Classical optimal control methods (e.g., Linear Quadratic Regulator - LQR) provide a single, fixed policy for all time. In contrast:

• MPC solves a finite-horizon optimal control problem repeatedly at each time step.
• This allows MPC to handle state and input constraints explicitly and adapt to changing references or disturbances in real-time, making it more flexible for complex, constrained systems.

Receding Horizon Control is the defining operational principle of MPC. It describes the iterative process where only the first control action from a finite-horizon optimization is executed before the horizon 'recedes' forward and the process repeats.

Mechanism:

1. At time t, measure/estimate the current state.
2. Solve an optimization to find the best control sequence for the window [t, t+H].
3. Apply only the first control action u(t) to the system.
4. At time t+1, repeat from step 1 with updated measurements. This feedback mechanism provides robustness to model inaccuracies and external disturbances that are not captured in the prediction.

The Linear Quadratic Regulator is a foundational optimal control solution for linear systems with quadratic cost functions. It provides a closed-form, optimal feedback control law u = -Kx.

LQR vs. MPC:

• LQR offers an infinite-horizon, globally optimal solution for unconstrained linear systems. It is computationally very cheap to execute.
• MPC can handle nonlinear dynamics, explicit constraints (e.g., actuator limits), and time-varying references, but requires solving an optimization problem online at each step. For linear, unconstrained problems with a long horizon, the MPC solution often converges to the LQR gain. LQR is frequently used as a terminal cost stabilizer within an MPC formulation.

The Cost Function (or objective function) and Constraints are the two pillars of the optimization problem solved at the heart of every MPC iteration.

• Cost Function: Typically penalizes:
  • Deviation from a desired reference trajectory (tracking error).
  • Magnitude of control actions (control effort).
  • Rate of change of controls (smoothness).
• Constraints: Hard or soft limits explicitly enforced during optimization, including:
  • Input Constraints: Physical actuator limits (e.g., max torque, voltage).
  • State Constraints: Safety or operational limits (e.g., temperature, position bounds).
  • Output Constraints: Limits on measured system outputs. The explicit handling of constraints is a primary advantage of MPC over other control methods.