Linear Quadratic Regulator (LQR)

A critical property of the infinite-horizon LQR is its inherent robustness to gain and phase variations. For single-input systems, the LQR guarantees a gain margin of (−6 dB, ∞) and a phase margin of ±60°. This means the system remains stable even if the actuator output is halved or the input phase shifts significantly. This robustness arises from the return difference inequality of the optimal Kalman gain K, making LQR superior to pole placement for physical systems with unmodeled actuator dynamics.

LQR solves the optimization problem of minimizing a quadratic cost function J = ∫(xᵀQx + uᵀRu) dt. The state weighting matrix Q penalizes deviations from the desired trajectory, while the control weighting matrix R penalizes actuator effort. The solution yields a linear state-feedback law u = −Kx, where the optimal gain matrix K = R⁻¹BᵀP. The matrix P is found by solving the Continuous-time Algebraic Riccati Equation (CARE): AᵀP + PA − PBR⁻¹BᵀP + Q = 0.

The core computational step in LQR is solving the Algebraic Riccati Equation (ARE). For high-dimensional systems, direct numerical solvers are used:

• Hamiltonian matrix method: Construct a 2n×2n matrix and compute its stable invariant subspace
• Schur decomposition: Numerically stable approach that avoids matrix inversion errors
• Iterative methods: Kleinman's algorithm solves a sequence of Lyapunov equations Modern libraries like SLICOT and ControlSystems.jl provide optimized ARE solvers for systems with hundreds of states.

For tracking a finite-horizon trajectory, the time-varying LQR solves the Differential Riccati Equation backward in time from a terminal condition P(t_f) = Q_f. This produces a time-dependent gain matrix K(t) that optimally balances tracking error and control effort at each instant. This is essential for robotic tasks like pick-and-place operations where the final pose accuracy is critical. The backward integration naturally yields a feedforward term that compensates for the reference dynamics.

LQR requires the pair (A, B) to be stabilizable to guarantee a finite solution to the Riccati equation. Full controllability ensures the Riccati solution P is positive definite. Key conditions:

• Controllability matrix [B, AB, A²B, ..., Aⁿ⁻¹B] must have full row rank
• Uncontrollable modes must be stable (have negative real parts)
• Q must be positive semi-definite and R must be positive definite If these conditions fail, the ARE may have no stabilizing solution, and the closed-loop system will be unstable.

LQR excels at stabilizing a robot around a nominal trajectory. Given a pre-planned path from a sampling-based planner like RRT*, LQR computes time-varying feedback gains that correct deviations caused by disturbances or modeling errors.

• Linearizes nonlinear dynamics around the reference trajectory at each timestep
• Produces a time-varying LQR (TVLQR) controller that minimizes a running cost on state error and control effort
• Commonly used in quadrotor aggressive maneuvering and autonomous vehicle lane-keeping
• The Q matrix penalizes state error (e.g., position offset), while R penalizes control effort (e.g., motor torque)

LQR is the classical solution for stabilizing inherently unstable systems like the cart-pole, Segway-style robots, and bipedal walking robots. The controller continuously computes corrective forces to keep the system upright.

• The linearized dynamics of an inverted pendulum fit perfectly into the LQR framework
• The cost function balances deviation from vertical (Q) against motor effort (R)
• Extends naturally to Linear Quadratic Gaussian (LQG) when sensor noise is significant
• Used in the MIT Cheetah and Boston Dynamics Atlas for dynamic balance during locomotion

For robotic arms performing high-precision tasks like assembly, welding, or surgical manipulation, LQR provides smooth, optimal convergence to a target pose without overshoot.

• Formulated in operational space using the end-effector Jacobian to map joint torques to Cartesian forces
• The Q matrix heavily penalizes Tool Center Point (TCP) position error
• Outperforms simple PID controllers when dynamic coupling between joints is significant
• Often combined with impedance control to manage contact forces during insertion tasks

LQR-based controllers are widely deployed in autonomous ground vehicles for smooth, stable path following. The controller minimizes both lateral offset from the reference path and steering effort.

• Uses a dynamic bicycle model linearized around the reference path
• The cost function penalizes cross-track error, heading error, and steering rate
• Provides a mathematically optimal alternative to geometric controllers like Pure Pursuit
• Implemented in the Apollo autonomous driving framework and numerous DARPA Grand Challenge entries

Modern humanoid robots use LQR within a whole-body control (WBC) hierarchy. LQR computes optimal ground reaction forces and joint torques to track a desired center of mass trajectory while maintaining balance.

• The linear inverted pendulum model (LIPM) provides the simplified dynamics for LQR design
• Q penalizes Center of Mass (CoM) position and velocity errors
• R penalizes Center of Pressure (CoP) movement, encouraging smooth weight shifts
• Integrated with Model Predictive Control (MPC) for handling future preview information and constraints

While both are optimal control methods, LQR and Model Predictive Control (MPC) serve different roles in the robotics control stack.

• LQR: Computes an explicit, closed-form feedback policy. Best for unconstrained problems where computational speed is critical and the system is well-approximated as linear
• MPC: Solves a numerical optimization at each timestep. Handles explicit constraints on joint limits, torque saturation, and obstacle avoidance
• A common architecture uses MPC for high-level planning with constraints, and LQR for low-level stabilization around the MPC-generated trajectory
• LQR provides the infinite-horizon cost-to-go used as the terminal cost in MPC formulations

The state weighting matrix Q penalizes deviations from desired states, while the input weighting matrix R penalizes control effort. Tuning these matrices defines the performance-aggressiveness trade-off:

• Large Q, small R: Aggressive error correction, high-gain control
• Small Q, large R: Conservative, energy-efficient motions
• Diagonal structure: Common for independent joint penalties
• Bryson's rule: A systematic method for initial Q/R selection based on acceptable state and input ranges

An extension of standard LQR for systems with time-varying dynamics A(t) and B(t). Instead of a single steady-state gain, LTV-LQR computes a time-varying gain schedule K(t) by integrating the Differential Riccati Equation backward from a terminal cost. This is critical for trajectory tracking in robotics, where the linearization of nonlinear dynamics around a reference trajectory produces time-varying Jacobians at each point along the path.

An iterative trajectory optimization algorithm that generalizes LQR to nonlinear systems. DDP performs a backward pass computing local LQR solutions along a nominal trajectory, then a forward pass with a line search. The backward pass solves a time-varying LQR problem at each iteration, making DDP a direct nonlinear extension of LTV-LQR. Widely used in legged locomotion and manipulation for computing locally optimal trajectories with full dynamics.