Model-Predictive Control (MPC)

This is the defining mechanism of MPC. At each control step, the algorithm solves a finite-horizon optimal control problem over a future time window (the prediction horizon). Only the first control action from the optimized sequence is applied to the system. The horizon then 'recedes' forward by one step, and the process repeats with new state measurements. This provides continuous feedback and adaptability.

• Core Loop: Predict → Optimize → Execute first step → Repeat.
• Benefit: Constantly corrects for model inaccuracies and disturbances.

A major advantage of MPC is its ability to directly incorporate hard and soft constraints into the online optimization problem. This is a key differentiator from traditional control methods like PID, which handle constraints indirectly or heuristically.

• State Constraints: e.g., joint limits, safe temperature ranges.
• Input Constraints: e.g., actuator torque/speed limits, valve saturation.
• Output Constraints: e.g., keeping a robot's end-effector within a safe workspace.

Constraints are formulated mathematically (e.g., x_min ≤ x ≤ x_max) within the optimization, ensuring the computed control actions are feasible and safe by design.

MPC's performance is fundamentally tied to the accuracy of its internal dynamics model. This model, which predicts how states evolve given actions, can range from simple linear equations to complex non-linear neural networks.

• White-Box Models: First-principles physics (e.g., Newton-Euler equations for a robot arm). High interpretability, but can be complex to derive.
• Grey-Box Models: Hybrid models combining physics with learned parameters.
• Black-Box Models: Fully learned models, such as neural network dynamics models. Offer flexibility but raise concerns about generalization and safety verification.

Model error directly leads to prediction error, degrading control performance.

MPC does not just find a feasible control action; it finds the optimal one according to a defined cost function. This function, minimized over the prediction horizon, encodes the control objectives.

• Typical Cost Terms:
  • Reference Tracking: Minimize error from a desired state trajectory.
  • Control Effort: Minimize energy consumption or actuator wear.
  • Terminal Cost: Encourage the system to reach a desirable region by the horizon's end.

By balancing these competing objectives (e.g., fast response vs. energy use), MPC provides a principled, tunable approach to high-performance control.

The need to solve an optimization problem online at every control step is MPC's primary practical challenge. The algorithm must complete its prediction and optimization within one sampling period (often milliseconds).

• Bottleneck: Solving the (often non-convex) optimization problem.
• Enabling Techniques:
  • Using fast, specialized quadratic programming (QP) solvers for linear MPC.
  • Approximate MPC using explicit solution pre-computation or neural network policy approximation.
  • Leveraging model simplification (e.g., linearization) to speed up computation.

The trade-off between model fidelity, horizon length, and solve time is a central engineering decision.

MPC naturally combines feedforward and feedback control. The optimization uses the model to plan a future action sequence (feedforward). However, because it re-solves at each step using the latest measured state, it continuously corrects for deviations caused by model error or disturbances, providing robust feedback.

• Feedback: Comes from the receding horizon mechanism itself.
• Disturbance Rejection: The controller can react to unmeasured disturbances by observing their effect on the state in the next optimization cycle.
• Comparison: This is more integrated than classic control, where feedforward and feedback paths are often separate add-ons.

MPC is the industry standard for large-scale chemical plants, refineries, and pharmaceutical manufacturing. It excels here due to:

• Multivariable coordination: Simultaneously controlling interconnected variables like temperature, pressure, and flow rates.
• Hard constraint handling: Enforcing critical safety limits (e.g., maximum reactor temperature) and operational limits (e.g., valve saturation) directly within the optimization.
• Economic optimization: The cost function can be tuned not just for setpoint tracking but for minimizing energy consumption or maximizing yield, directly impacting the bottom line.

MPC provides the motion planning and control backbone for self-driving cars, drones, and autonomous mobile robots (AMRs).

• Path tracking & obstacle avoidance: The optimizer can plan smooth trajectories that respect vehicle dynamics (kinematics) while avoiding static and dynamic obstacles, encoded as constraints.
• Stability control: For road vehicles, MPC can unify adaptive cruise control, lane-keeping, and electronic stability control into a single optimization problem.
• Energy efficiency: For electric vehicles, the cost function can minimize battery usage over the prediction horizon.

Aircraft and spacecraft leverage MPC for its ability to manage highly nonlinear dynamics and strict safety envelopes.

• Attitude control: For satellites and rockets, MPC calculates optimal thruster firings to achieve desired orientation while minimizing fuel (a limited resource).
• Load alleviation: In commercial aircraft, MPC can adjust control surfaces to reduce structural stress from turbulence.
• Formation flying: For drone swarms or satellite constellations, distributed MPC algorithms enable coordinated maneuvers while maintaining safe relative distances.

In robotics, MPC enables reactive, contact-rich manipulation and whole-body dynamic control.

• Legged locomotion: Robots like Boston Dynamics' systems use variants of MPC to maintain balance, walk, and run by solving for optimal ground reaction forces and joint torques in real-time.
• Manipulator control: For arms performing tasks like wiping a surface or inserting a peg, MPC can optimize contact forces and trajectories while respecting torque limits.
• Humanoid robotics: MPC coordinates complex, high-degree-of-freedom bodies to perform dynamic tasks like catching an object or recovering from a push.

MPC is critical for managing the volatility and complexity of modern energy grids with renewable sources.

• Building climate control: Optimizes heating, ventilation, and air conditioning (HVAC) schedules to maintain comfort while minimizing energy costs, using predictions of weather and occupancy.
• Microgrid dispatch: Coordinates distributed energy resources (solar panels, batteries, generators) to meet demand, using forecasts for renewable generation and electricity prices.
• Power electronics: Controls voltage and frequency in inverters for wind turbines and solar farms to ensure grid stability.

MPC enables precise, adaptive automation in life-critical medical systems and wearable robotics.

• Artificial pancreas: For Type 1 diabetes, MPC uses a model of the patient's glucose-insulin dynamics to automatically regulate insulin pump delivery, predicting and preventing dangerous highs and lows.
• Anesthesia control: Automatically adjusts the delivery of anesthetic agents to maintain a target depth of sedation based on physiological measurements.
• Robotic prosthetics & exoskeletons: Provides smooth, intuitive assistive force by predicting the user's intended motion (e.g., walking gait) and optimizing actuator output to reduce metabolic cost.

Model-Based Reinforcement Learning (MBRL) is a paradigm where an agent explicitly learns a model of the environment's transition dynamics and reward function, then uses this model for planning or policy optimization. Unlike MPC's typically hand-specified or identified model, MBRL focuses on learning the model from interaction data.

• Core Distinction: MPC is an online control algorithm that uses a model; MBRL is a learning framework for acquiring that model.
• Synergy: Learned models from MBRL (e.g., neural networks) are increasingly used as the predictive model within an MPC loop, creating adaptive, learning-based controllers.
• Example: The Dreamer agent uses a learned Recurrent State-Space Model (RSSM) as its world model and plans actions within it, blurring the line between advanced MBRL and MPC.

A Markov Decision Process (MDP) is the foundational mathematical framework for sequential decision-making. It formally defines the problem MPC solves online. An MDP is a tuple (S, A, P, R, γ) where:

• S is a set of states.
• A is a set of actions.
• P(s' | s, a) is the transition dynamics (the model MPC uses).
• R(s, a, s') is the reward function (encoded in MPC's cost function).
• γ is a discount factor.

MPC approximates the solution to the finite-horizon, undiscounted version of the MDP problem at each control step. The 'Markov' property—that the future depends only on the present state—is a core assumption for the state representation used in MPC's predictions.

State Estimation is the process of inferring the true, often hidden, state of a dynamical system from a sequence of noisy sensor observations and known control inputs. It is a critical prerequisite for MPC, as the controller's predictions are only as good as its initial state.

• Kalman Filter: The optimal estimator for linear systems with Gaussian noise. Often used in linear MPC pipelines.
• Extended Kalman Filter (EKF) & Unscented Kalman Filter (UKF): Non-linear extensions for systems like robotics.
• Particle Filter: A sequential Monte Carlo method for complex, non-Gaussian estimation problems.

In practice, an MPC controller is typically fed the output of a state estimator (e.g., fused IMU and camera data for a drone's pose), not raw sensor readings. This falls under the broader Partially Observable MDP (POMDP) framework, where the belief state from the estimator is the input to MPC.

Optimal Control is the broader field of mathematics and engineering concerned with finding a control law that minimizes a cost function over time for a dynamical system. MPC is a specific, online implementation of optimal control principles.

• Classical Methods: Include Linear Quadratic Regulator (LQR) and its non-linear variant Linear Quadratic Gaussian (LQG). These provide closed-form, offline solutions for specific problem classes.
• MPC's Contribution: It handles constraints (e.g., joint limits, obstacle avoidance) explicitly and naturally, which is analytically intractable for many classical methods.
• Shooting Methods: MPC is closely related to direct multiple shooting, a numerical optimal control technique where the control trajectory is parameterized and the problem is transcribed into a Non-Linear Program (NLP) solved at each step.

Monte Carlo Tree Search (MCTS) is a heuristic search algorithm for decision processes that, like MPC, builds a look-ahead tree of possible future states and actions. It uses random sampling (Monte Carlo) to estimate the value of nodes.

• Comparison with MPC:
  • MPC uses a deterministic or stochastic model to predict exact next states and solves a continuous optimization problem (often gradient-based).
  • MCTS uses randomized simulations (rollouts) to estimate state values and is inherently discrete, though adaptations exist.
  • Use Case: MCTS famously powered AlphaGo for discrete board games. MPC dominates continuous control domains like process plants and autonomous vehicles.
• Hybrid Approaches: Some advanced planners use MCTS to explore high-level discrete options while MPC executes low-level continuous control, a method relevant to Task and Motion Planning (TAMP).

The Sim2Real Gap refers to the performance degradation experienced when a policy or controller (like an MPC scheme) trained or tuned in simulation is deployed on a physical system. This gap arises due to inaccuracies in the simulation's model of dynamics, sensors, or actuators.

• Impact on MPC: The predictive model used by MPC is never perfect. The sim2real gap is a manifestation of model error or model mismatch.
• Mitigation Strategies:
  • System Identification: Carefully identifying parameters of the simulation model from real data.
  • Domain Randomization: Training or tuning with randomized simulation parameters to create a robust controller.
  • Online Adaptation: Using real-world data to continuously update the MPC's internal model (Model Adaptation).
  • Robust MPC: Designing the MPC optimization to account for bounded model uncertainty.

Bridging this gap is essential for deploying data-efficient, simulation-tuned MPC in the real world.