State Estimation

The Kalman Filter is an optimal recursive algorithm for estimating the state of a linear dynamical system with Gaussian noise. It operates in a two-step predict-update cycle:

• Prediction: Projects the current state estimate forward using the system's dynamics model.
• Update (Correction): Fuses the prediction with a new, noisy measurement to produce a refined state estimate.

It is the workhorse for applications like GPS navigation and inertial measurement unit (IMU) fusion, providing minimum mean square error estimates.

The Extended Kalman Filter is the non-linear extension of the standard Kalman Filter. It linearizes the system's non-linear dynamics and measurement models around the current state estimate using a first-order Taylor expansion (the Jacobian).

While sub-optimal, it is widely used for state estimation in robotics (e.g., robot localization with lidar) and aerospace. Its primary limitation is divergence if the linearization is poor for highly non-linear systems.

The Unscented Kalman Filter addresses the linearization errors of the EKF by using a deterministic sampling technique called the unscented transform. It propagates a carefully chosen set of sample points (sigma points) through the true non-linear functions and then computes the mean and covariance of the transformed points.

This provides more accurate estimation for strongly non-linear systems than the EKF, with similar computational cost, making it popular for attitude estimation and vehicle tracking.

The Particle Filter is a sequential Monte Carlo method for state estimation in non-linear, non-Gaussian systems. It represents the belief state (the probability distribution over states) as a set of discrete samples called particles, each with an associated weight.

The algorithm iterates through:

• Prediction: Particles are propagated through the dynamics model with added noise.
• Update: Particle weights are updated based on the likelihood of the new observation.
• Resampling: Particles with low weights are replaced by copies of high-weight particles to avoid degeneracy.

It is essential for problems like robotic Simultaneous Localization and Mapping (SLAM) in cluttered environments.

Moving Horizon Estimation is an optimization-based approach that formulates state estimation as solving a constrained least-squares problem over a sliding window of the most recent measurements and control inputs. It explicitly handles state and measurement constraints (e.g., physical limits).

By re-solving this optimization problem online at each time step, MHE provides highly accurate estimates for complex, constrained systems but is computationally more intensive than recursive filters. It is commonly used in chemical process control and advanced automotive applications.

The Complementary Filter is a simple, efficient frequency-domain sensor fusion technique. It combines two or more sensors with complementary noise characteristics: one accurate in the long term (low-frequency) but noisy in the short term (e.g., accelerometer for tilt), and one stable in the short term but prone to drift (e.g., gyroscope).

A high-pass filter is applied to the drift-prone sensor and a low-pass filter to the noisy sensor; their outputs are summed. This provides robust attitude estimation for drones and wearable devices with minimal computational overhead compared to Kalman-based approaches.

Simultaneous Localization and Mapping (SLAM) is a canonical application where a robot must estimate its own pose (position and orientation) while concurrently building a map of an unknown environment. Algorithms like the Extended Kalman Filter (EKF) or Particle Filter fuse data from inertial measurement units (IMUs), wheel odometry, LiDAR, and cameras to maintain a probabilistic belief over the robot's state. This enables autonomous vehicles, drones, and warehouse robots to navigate reliably without GPS.

State estimation is mission-critical in aerospace for attitude and orbit determination. The Kalman filter, originally developed for the Apollo program, processes gyroscope, star tracker, and GPS measurements to estimate a spacecraft's orientation, position, and velocity with extreme precision. Modern aircraft use similar techniques for sensor fusion in flight control systems, integrating air data, inertial navigation, and radar altimeter readings to provide a unified, reliable state estimate for autopilot functions.

In chemical plants, refineries, and power grids, state estimators monitor hidden internal variables (e.g., reactant concentrations, catalyst activity, turbine health) that cannot be measured directly with sensors. By combining a first-principles physics model with readings from pressure, temperature, and flow sensors, algorithms like the Unscented Kalman Filter provide real-time estimates. These estimates feed into Model-Predictive Control (MPC) systems to optimize production, ensure safety, and predict maintenance needs.

State estimation techniques decode physiological signals to infer hidden health states. Key applications include:

• Brain-Computer Interfaces (BCIs): Decoding neural activity from EEG to estimate intended limb movement.
• Cardiac Monitoring: Using particle filtering to estimate the electrical state of the heart from noisy ECG signals, enabling detection of arrhythmias.
• Glucose Prediction: In artificial pancreas systems, Kalman filters predict future blood glucose levels from continuous monitor data to inform insulin dosing.

In quantitative finance, state-space models treat economic variables (like true asset volatility or a company's latent financial health) as hidden states observed through noisy market prices. The Kalman filter is used for:

• Volatility Estimation: Filtering noisy price data to infer the true, time-varying volatility of an asset.
• Term Structure Modeling: Estimating the latent factors driving interest rate curves.
• Algorithmic Trading: Providing smoothed estimates of trend and momentum signals from high-frequency tick data, reducing the impact of market microstructure noise.

Defense and surveillance systems rely on state estimation to track multiple moving targets (e.g., aircraft, ships, ground vehicles) using networks of radars, sonars, or cameras. This involves multi-sensor data fusion and multi-object tracking algorithms like the Probability Hypothesis Density (PHD) filter. The system maintains a probabilistic belief over each target's kinematic state (position, velocity, acceleration), resolving ambiguities from clutter, missed detections, and occlusions to provide a coherent tactical picture.

A Partially Observable Markov Decision Process (POMDP) is the formal mathematical framework for state estimation. It extends the standard MDP by acknowledging the agent cannot directly observe the true state. Instead, it must maintain a belief state—a probability distribution over all possible states—which is updated via a belief update rule using observations and actions.

• Core Challenge: Optimal planning under uncertainty.
• Key Components: State space, action space, observation model, transition model, reward function.
• Applications: Robot navigation in cluttered environments, medical treatment planning, dialogue systems.

The Kalman filter is the optimal recursive algorithm for state estimation in linear Gaussian systems. It operates in a predict-update cycle: it predicts the next state using a dynamics model, then corrects this prediction by incorporating a new, noisy observation. The filter maintains estimates of both the state mean and its covariance (uncertainty).

• Assumptions: Linear dynamics, Gaussian noise.
• Variants: Extended Kalman Filter (EKF) for nonlinear systems, Unscented Kalman Filter (UKF).
• Ubiquitous Use: GPS navigation, aircraft tracking, financial econometrics.

A particle filter (or Sequential Monte Carlo method) is a non-parametric state estimation algorithm for nonlinear, non-Gaussian systems. It represents the belief state as a set of discrete samples (particles), each with a weight. The algorithm propagates these particles through the dynamics model and reweights them based on the likelihood of new observations.

• Key Strength: Handles complex, multi-modal distributions (e.g., global robot localization).
• Challenge: Particle degeneracy, often addressed by resampling.
• Applications: Visual object tracking, simultaneous localization and mapping (SLAM), financial volatility modeling.

A belief state is the internal representation of an agent's knowledge about the world in a POMDP. Formally, it is a probability distribution, b(s), over the set of possible true states S. It is a sufficient statistic for the history of actions and observations, meaning the optimal policy can be defined as a function of the belief state alone.

• Update Rule: Governed by Bayes' theorem.
• Representation: Can be parametric (e.g., Gaussian for a Kalman filter) or non-parametric (e.g., particle set).
• Role: The core output of state estimation, serving as the input to planning and policy modules.

Simultaneous Localization and Mapping (SLAM) is the canonical robotics problem that combines state estimation with mapping. A robot must concurrently build a map of an unknown environment and estimate its own position within that map. It is a high-dimensional, online state estimation problem where the state includes both the robot's pose and the locations of landmarks.

• Front-End: Data association (matching observations to landmarks).
• Back-End: Optimization (e.g., Graph-based SLAM) or filtering (e.g., EKF-SLAM) to correct pose and map estimates.
• Modern Approach: Use factor graphs and nonlinear optimization (e.g., Google Cartographer).

A Recurrent State-Space Model (RSSM) is a deep learning-based world model architecture that learns a compact latent state representation for planning from high-dimensional observations (like pixels). It combines a deterministic recurrent neural network (to model temporal dependencies) with a stochastic latent variable model (to capture uncertainty).

• Key Innovation: Separates deterministic and stochastic components of the state.
• Famous Use Case: The Dreamer series of model-based RL agents.
• Function: Acts as a learned, differentiable filter and dynamics model, enabling latent-space imagination for planning.