Linear Quadratic Regulator (LQR)

LQR is an optimal feedback controller. It does not compute an open-loop trajectory; instead, it calculates a state-feedback gain matrix (K). This matrix is applied as u = -Kx, where u is the control input and x is the system state. This policy is optimal for the infinite-horizon problem, meaning the control law is constant and stabilizes the system from any initial condition while minimizing the defined cost.

The 'Quadratic' in LQR refers to its cost function: J = ∫ (xᵀQx + uᵀRu) dt. This function penalizes two things:

• State Deviation (xᵀQx): The matrix Q (positive semi-definite) penalizes the system for being away from the desired equilibrium (typically zero).
• Control Effort (uᵀRu): The matrix R (positive definite) penalizes the use of large control inputs. The engineer's primary design task is to tune the Q and R matrices to trade off between performance (fast convergence) and actuator effort/energy consumption.

LQR's theoretical guarantees and solution method are strictly valid for linear time-invariant (LTI) systems described by: ẋ = Ax + Bu. Here, A is the state matrix and B is the input matrix. This is a core limitation. In practice, LQR is often applied to linearized models of nonlinear systems (e.g., a robot arm linearized around an upright pose). The controller's performance is optimal for the linear model and is typically only locally effective for the original nonlinear system.

The optimal feedback gain matrix K is not found through iterative optimization for each new state. Instead, it is computed offline by solving the Continuous-Time Algebraic Riccati Equation (CARE): AᵀP + PA - PBR⁻¹BᵀP + Q = 0. The solution is a constant matrix P (positive definite), from which the gain is calculated as K = R⁻¹BᵀP. This provides a closed-form solution, making the online control computation (just a matrix multiplication) extremely fast and deterministic, suitable for real-time control loops.

The standard LQR formulation regulates a system to zero. For trajectory tracking—following a time-varying reference—the problem is transformed. A time-varying error state is defined as the deviation from the desired trajectory. If the trajectory is generated by a linear model, the error dynamics are also linear, and an LQR controller can be designed for these error dynamics. This results in a Linear Quadratic Tracker, which is a foundational component in hierarchical control: a planner generates a trajectory, and an LQR-based tracker stabilizes the system to it.

LQR's core assumptions lead to key limitations, which have inspired major extensions:

• Nonlinear Systems: Linearization is required, limiting validity to regions near the linearization point.
• No Constraints: The standard LQR solution ignores physical limits on state or control (e.g., torque limits). This is addressed by Model Predictive Control (MPC), which solves a constrained finite-horizon optimization online.
• Stochastic Noise: The basic LQR assumes a perfect, noise-free model. The Linear Quadratic Gaussian (LQG) controller combines an LQR with a Kalman filter to handle state estimation in the presence of noise.

LQR is the workhorse controller for robotic manipulators performing precise tasks like welding, assembly, or pick-and-place. It is often used as the tracking controller for a pre-planned trajectory.

• Core Function: The controller calculates the optimal joint torques to follow a desired end-effector path while minimizing a cost function that penalizes tracking error and control effort.
• Implementation: The robot's nonlinear dynamics are linearized around the reference trajectory. LQR gains are then computed for this linear time-varying (LTV) system, a method known as Linear Quadratic Tracking (LQT) or as the feedback component of a Linear-Quadratic-Gaussian (LQG) controller when combined with a Kalman filter for state estimation.
• Advantage: Provides a mathematically rigorous, stable controller that optimally balances performance (speed/accuracy) against energy consumption and actuator wear.

The classic benchmark for modern control theory, the inverted pendulum (and its variant, the cart-pole system) is inherently unstable. LQR is the standard method for balancing it.

• Control Objective: Stabilize the pendulum in its upright equilibrium point using force applied to the cart.
• Process: The nonlinear equations of motion are linearized around the unstable upright position (θ = 0). The LQR algorithm then computes the optimal state feedback gains (K) that map the measured state [cart position, cart velocity, pendulum angle, pendulum angular velocity] to a corrective force.
• Significance: This application demonstrates LQR's ability to stabilize unstable systems, a principle extended to bipedal robot balance, rocket guidance, and high-performance aircraft control.

LQR forms the inner-loop attitude controller for multirotor drones and fixed-wing UAVs, responsible for stable, agile flight.

• Architecture: Flight control is typically hierarchical. LQR operates on the attitude dynamics (pitch, roll, yaw), which are approximately linear for small angles. A higher-level planner or outer loop generates desired attitude commands for the LQR controller to track.
• Cost Function Design: The quadratic cost function is tuned to heavily penalize angular errors and angular rates, ensuring rapid disturbance rejection (e.g., from wind gusts) while limiting aggressive, power-hungry control actions.
• Result: This provides the drone with the inherent stability required for autonomous navigation, hovering, and trajectory following, often serving as the baseline against which more complex nonlinear controllers are compared.

LQR is applied in mechatronic systems to isolate sensitive components from vibrations or to improve ride quality by actively controlling suspension elements.

• System Model: The vehicle or platform is modeled as a multi-mass spring-damper system. The states typically represent displacements and velocities of the chassis, wheels, and/or payload.
• Optimization Goal: The cost function is designed to minimize a weighted sum of:
  • Payload acceleration (for ride comfort or instrument protection).
  • Suspension travel (to avoid hitting mechanical limits).
  • Control force (actuator energy).
• Applications: High-performance automotive suspensions, precision manufacturing stages, satellite instrument isolation, and seismic damping systems for buildings.

In electrical engineering, LQR is used for wide-area damping control in power grids and for voltage/frequency regulation in microgrids and spacecraft power systems.

• Challenge: Power networks are large-scale, interconnected linear systems susceptible to low-frequency oscillations following disturbances like a fault or load change.
• LQR Solution: A centralized or decentralized LQR controller uses measurements from phasor measurement units (PMUs) to compute optimal damping signals for generator exciters or FACTS devices. The cost function penalizes deviations in rotor speeds and angles across the network.
• Outcome: Enhances the transient stability and dynamic performance of the grid, preventing cascading failures. Similarly, LQR regulates the output of DC-DC converters in satellite power systems.

In industrial process control, LQR manages continuous stirred-tank reactors (CSTRs), distillation columns, and other unit operations by regulating temperatures, pressures, and concentrations.

• Linearization: The highly nonlinear chemical process dynamics are linearized around a desired operating point (e.g., a specific temperature and reactant concentration for optimal yield).
• Multivariable Control: LQR excels here as a Multi-Input Multi-Output (MIMO) controller. It can simultaneously adjust heater power, coolant flow, and inlet valve positions to maintain all process variables at their setpoints while minimizing energy use.
• Economic Optimization: The quadratic cost can be tuned to reflect operating economics, penalizing expensive control actions (like steam consumption) more heavily than cheaper ones, leading to optimal steady-state operation.

The cornerstone equation solved to compute the LQR optimal feedback gain. It is a nonlinear matrix equation of the form: AᵀP + PA - PBR⁻¹BᵀP + Q = 0

Where:

• A, B are the system matrices.
• Q, R are the state and control cost matrices from the quadratic objective.
• P is the unknown symmetric, positive-definite matrix whose solution yields the optimal cost matrix and the feedback gain K = R⁻¹BᵀP.

Solving the ARE is the central computational step in LQR design.

The broader mathematical framework for finding optimal state and control sequences. LQR is a specific, linear case within this family.

• General Trajectory Optimization solves for a sequence over a finite horizon, often with nonlinear dynamics and constraints, using methods like Direct Collocation or Sequential Quadratic Programming (SQP).
• LQR provides a closed-form, infinite-horizon solution for linear systems with quadratic costs.
• Iterative LQR (iLQR) is a variant that linearizes nonlinear dynamics around a nominal trajectory and solves a sequence of LQR subproblems to converge to a locally optimal solution for the nonlinear system.