Lyapunov Function

A Lyapunov function, denoted as V(x), must be positive definite around the equilibrium point (typically the origin, x=0). This means:

• V(0) = 0 (zero at the equilibrium).
• V(x) > 0 for all x ≠ 0 in a region around the origin.

This property ensures the function acts like a measure of 'energy' or 'distance' from the equilibrium, where zero represents the desired stable state.

The core stability proof lies in the time derivative of V(x) along the system's trajectories. For asymptotic stability, the derivative V̇(x) must be negative definite (or negative semi-definite with additional conditions). This means:

• V̇(0) = 0.
• V̇(x) < 0 for all x ≠ 0.

This property guarantees that the 'energy' V(x) is strictly decreasing over time, forcing the system state x(t) to converge to the equilibrium.

To prove global asymptotic stability, a Lyapunov function must be radially unbounded. This means:

• ||x|| → ∞ implies V(x) → ∞.

This property ensures that as the system state moves infinitely far from the origin, the 'energy' measure becomes infinitely large. It guarantees that no state can diverge to infinity while still decreasing V(x), confining all trajectories to eventually converge to the equilibrium from any initial condition.

When V̇(x) is only negative semi-definite (V̇(x) ≤ 0), asymptotic stability can still be proven using LaSalle's Invariance Principle. It states that the system will converge to the largest invariant set contained within the set where V̇(x) = 0.

This is critical for systems with conserved quantities (like mechanical energy) where the Lyapunov function derivative is zero not only at the equilibrium but along certain motions. The principle shows convergence to a set, not just a point.

Finding a valid Lyapunov function is often non-trivial. Common construction methods include:

• Quadratic Forms: For linear systems, V(x) = xᵀPx, where P is a positive definite matrix. Stability is checked by solving the Lyapunov equation AᵀP + PA = -Q.
• Sum of Squares (SOS) Programming: For polynomial systems, uses semidefinite programming to search for a polynomial Lyapunov function.
• Energy-Based Methods: For mechanical/electrical systems, using total energy (kinetic + potential) as a starting candidate.
• Computational Lyapunov Functions: Using machine learning (e.g., neural networks) to approximate Lyapunov functions for complex systems.

In robotics and autonomous systems, a Lyapunov function provides a formal certificate of asymptotic stability for an equilibrium point (e.g., a robot standing upright or a drone hovering). Engineers design controllers and then construct a candidate Lyapunov function V(x) to prove that the closed-loop system's state x will converge to the desired equilibrium. This is foundational for nonlinear control and adaptive control systems, ensuring that learned or designed policies do not lead to unstable, divergent behavior.

Lyapunov functions are used to formulate safety constraints within Constrained Markov Decision Processes (CMDPs). A Lyapunov-based safety critic can be learned or designed to estimate the risk of leaving a safe set. The core idea is to restrict the RL agent's policy to actions that guarantee the Lyapunov function is non-increasing, thereby ensuring the system state remains within a region of attraction or a predefined safe region. This provides a mathematically rigorous alternative to heuristic penalty methods.

While Lyapunov functions certify stability, Control Barrier Functions (CBFs) are closely related constructs that certify safety (forward invariance of a safe set). They are often used in tandem. A CBF h(x) defines a safe region (where h(x) >= 0). The controller is synthesized to ensure the derivative of h(x) satisfies a condition that keeps the state safe. This framework is central to provably safe motion planning for autonomous vehicles and robots, enabling them to avoid obstacles while maintaining stability.

Before deploying a simulation-trained policy to physical hardware, a Lyapunov analysis can verify its robustness to the reality gap. By analyzing the policy's behavior in simulation with a Lyapunov function, engineers can bound the expected performance degradation due to model mismatch or unmodeled dynamics. If the function remains valid under bounded disturbances, it increases confidence that the policy will remain stable in the real world, a key step in safety-critical validation.

When control policies are parameterized by deep neural networks (e.g., Neural Lyapunov Functions, Learning-based MPC), verifying their stability is challenging. Techniques like Lyapunov-based training jointly learn a policy and a candidate Lyapunov function. Post-training, tools from formal verification and sum-of-squares programming can be used to check the Lyapunov conditions over a continuous region of the state space, providing certificates for neural network-controlled systems.

In systems subject to actuator failures or sensor faults, a family of Lyapunov functions can be used to design switched controllers that maintain stability. Each controller, paired with its Lyapunov function, is valid for a specific system mode (e.g., a drone with a failed motor). A supervisory system detects the fault and switches to the appropriate controller, ensuring graceful degradation rather than catastrophic failure. This is a core method in fault-tolerant control design for aerospace and industrial systems.

A Control Barrier Function (CBF) is a mathematical construct used to formally guarantee that a dynamical system's state remains within a predefined safe set. Unlike a Lyapunov function, which certifies stability, a CBF is used to synthesize a safe control input that actively keeps the system away from unsafe states. It is a key tool for enforcing safety constraints in real-time control.

• Core Mechanism: Defines a function h(x) where h(x) >= 0 denotes the safe set. The controller is designed to ensure the derivative of h(x) satisfies a condition that keeps the state safe.
• Application: Critical for robotic safety where a system must avoid obstacles or stay within operational limits.

A barrier function is a Lyapunov-like scalar function used to certify set invariance—ensuring a system's trajectory remains inside a desired region. It becomes infinite as the system's state approaches the boundary of the safe set, creating a repulsive "barrier."

• Relation to Lyapunov: While a Lyapunov function proves convergence to an equilibrium, a barrier function proves the system will never leave a safe region. They are often used together for stability and safety guarantees.
• Types: Includes reciprocal barrier functions and zeroing barrier functions, each with different properties for control synthesis.

Safe Reinforcement Learning (Safe RL) is a subfield focused on developing algorithms that learn to maximize performance while satisfying safety constraints during training and deployment. It formalizes the problem using frameworks like Constrained Markov Decision Processes (CMDPs).

• Lyapunov's Role: Lyapunov functions can be used to learn stability-guaranteed policies, where the agent's policy is constrained to ensure the underlying system dynamics remain stable.
• Key Techniques: Includes constrained policy optimization, risk-sensitive RL using metrics like CVaR, and shielded learning where an external monitor overrides unsafe actions.

A Constrained Markov Decision Process (CMDP) extends the standard MDP by incorporating constraints on expected cumulative costs. It is the primary mathematical framework for Safe Reinforcement Learning.

• Formulation: Defined by states, actions, transition probabilities, a reward function, and one or more cost functions with associated cost limits.
• Objective: The agent must find a policy that maximizes expected cumulative reward while ensuring the expected cumulative cost remains below a specified threshold. This directly encodes safety or resource constraints into the learning problem.

Formal verification is the process of using rigorous mathematical methods to prove or disprove the correctness of a system's design with respect to a formal specification. In control and AI safety, it is used to prove properties like stability, safety, and liveness.

• Connection to Lyapunov: Finding a valid Lyapunov function for a dynamical system constitutes a formal proof of its asymptotic stability. Techniques like sum-of-squares programming automate the search for Lyapunov functions, bridging control theory and formal methods.
• Application: Used to verify neural network controllers, hybrid systems, and cyber-physical systems before deployment.

Runtime monitoring is a safety technique that involves continuously observing a system's execution to detect violations of specified safety properties or constraints in real-time. It acts as a last line of defense.

• Lyapunov-based Monitoring: A Lyapunov function can serve as a runtime certificate. If the computed value of the function or its derivative violates stability conditions, the monitor can trigger a fail-safe intervention or switch to a recovery policy.
• Use Case: Essential for adaptive systems and learning-based controllers where formal guarantees may not hold under all conditions, enabling graceful degradation.