Sensor Model

The observation model, or measurement model, is the mathematical function that maps the true state of the world to the expected sensor reading. It defines what the sensor should measure in an ideal, noise-free scenario.

• For a camera: This is the projective geometry (pinhole model) that maps 3D world points to 2D pixel coordinates.
• For a lidar: This is the expected range and bearing to a point given the sensor's pose and the target's position.
• For an IMU: This is the expected specific force and angular rate given the platform's acceleration and rotation.

This model is essential for the update step in Bayesian filters like the Kalman filter, where predicted measurements are compared against actual observations.

This component quantifies the stochastic errors inherent in all physical sensors. It defines the statistical properties of the discrepancy between the ideal observation model and real-world, noisy measurements.

• Key Parameter: The covariance matrix (R) of the measurement noise. This matrix encodes the variance (uncertainty) of each measurement dimension and the correlations between them.
• Common Distributions: Sensor noise is often modeled as zero-mean Gaussian (white noise), but models can include bias (non-zero mean), colored noise (time-correlated), or non-Gaussian distributions for robust estimation.
• Example: A GPS sensor model specifies not just the expected position, but the Circular Error Probable (CEP) or the covariance ellipse that defines the probable region of the true location.

This defines the spatial and operational envelope within which the sensor functions. It specifies the conditions under which the observation model and noise characteristics are valid.

• Spatial Limits: Field of View (FoV) (angular coverage for cameras/lidar), maximum/minimum range, and blind spots.
• Physical Limits: Saturation points (e.g., a camera's over-exposed pixels), minimum detectable signal, and dynamic range.
• Detection Probability (Pd): For object-detecting sensors like radar, the model includes the probability of detecting a target of a given size at a specific range, often formalized in a Receiver Operating Characteristic (ROC) curve. The complementary False Alarm Rate (FAR) is also modeled.

This component models the time-dependent behavior of the sensor, accounting for delays and the sensor's own internal state dynamics.

• Fixed Latency: The constant delay from the physical event to the measurement being timestamped and available for processing (e.g., camera readout time, filter settling time).
• Variable Latency/Jitter: Fluctuations in delay caused by network transmission or operating system scheduling, which must be characterized for precise sensor synchronization.
• Dynamic Response: For sensors like accelerometers, this includes the frequency response and bandwidth, defining how well the sensor can track rapidly changing signals. A first-order low-pass filter is a common simple model for this dynamic.

These are the fixed, calibratable parameters that complete the mathematical description of the sensor's geometry and placement.

• Intrinsic Parameters: Define the sensor's internal geometry. For a camera, these are focal length, principal point, and lens distortion coefficients (e.g., radial, tangential). Intrinsic calibration is required to determine these.
• Extrinsic Parameters: Define the sensor's pose (position and orientation) relative to a common reference frame, such as the vehicle body frame. This is a rigid transformation (rotation matrix and translation vector) obtained through extrinsic calibration. Accurate extrinsics are critical for sensor fusion to correctly align data from multiple sensors (e.g., lidar to camera).

A comprehensive sensor model includes characterizations of how the sensor can fail or degrade, enabling the system to perform Fault Detection and Isolation (FDI).

• Soft Failures: Gradual degradation, such as increasing noise covariance or a slowly drifting bias.
• Hard Failures: Catastrophic loss of data, returning null values, or constant nonsense readings.
• Fault Models: These describe the signature of specific failures. For example, a stuck-at-value fault where a radar's range reading becomes constant, or a scale factor fault in an IMU where its output is incorrectly scaled. Modeling these allows the fusion system to identify anomalous sensors and down-weight or exclude their data via algorithms like Chi-squared gating or dedicated FDI modules.

Sensor fusion is the overarching process of combining data from multiple physical sensors to produce a more accurate, complete, and reliable estimate of the state of an environment or system than is possible with any single sensor. It is the primary engineering goal that a sensor model serves.

• Purpose: Mitigate individual sensor limitations (noise, blind spots, failure).
• Architectures: Includes centralized, decentralized, and distributed fusion.
• Applications: Autonomous vehicles, robotics, aerospace navigation, and industrial monitoring.

State estimation is the process of inferring the unknown or hidden variables (the state) of a dynamic system from a sequence of noisy sensor observations. A sensor model provides the critical observation model that links the true state to the sensor's measurements.

• Core Problem: Determining position, velocity, orientation, or other latent properties.
• Relation to Sensor Model: The sensor model defines z = h(x) + v, where z is the measurement, x is the state, h() is the observation function, and v is noise.
• Algorithms: Encompasses all filtering and smoothing techniques, including Kalman and particle filters.

Bayesian filtering is the general probabilistic framework for recursively estimating the state of a dynamic system from noisy sensor data. It provides the mathematical foundation for implementing a sensor model within an estimation pipeline.

• Principle: Recursively applies Bayes' rule: P(state | measurements) ∝ P(measurement | state) * P(state).
• Key Component: The likelihood function P(measurement | state) is derived directly from the sensor model.
• Implementations: The Kalman filter (for linear/Gaussian), particle filter (for nonlinear/non-Gaussian), and their variants are specific realizations of this framework.

A process model (or motion model) is a mathematical representation that predicts how the state of a system evolves over time, based on known dynamics or control inputs. It works in tandem with the sensor model within a filtering cycle.

• Role: Used in the prediction step of estimators (e.g., Kalman filter) to propagate the state forward: x_k = f(x_{k-1}, u_k) + w.
• Contrast with Sensor Model: The process model predicts the state; the sensor model describes how that predicted state is observed.
• Examples: Constant velocity model for tracking, inertial navigation equations, robot kinematic models.

Calibration is the process of determining the precise parameters of a sensor model. Intrinsic calibration finds internal parameters (e.g., focal length, lens distortion), while extrinsic calibration finds the sensor's position and orientation relative to a reference frame or other sensors.

• Intrinsic: Essential for accurate measurement generation (e.g., correcting a camera's radial distortion).
• Extrinsic: Critical for sensor fusion, as it defines the rigid transformation needed to align data from different sensors into a common coordinate system.
• Methods: Often involves solving optimization problems using known calibration targets or patterns.

A covariance matrix is a square matrix that represents the uncertainty of a state vector and the correlations between its different components. In sensor modeling, it quantifies the measurement noise (R matrix) and is central to probabilistic fusion.

• Sensor Noise: The sensor model's noise characteristics are typically defined by a covariance matrix R.
• Role in Fusion: The Kalman filter uses the measurement covariance R and state covariance P to compute the optimal Kalman gain, determining how much to trust the new sensor reading versus the prediction.
• Related Metric: The Mahalanobis distance, which uses the covariance, is a standard method for gating and outlier rejection of sensor data.