Outlier Rejection

The Mahalanobis distance is the statistically principled metric for outlier detection within Gaussian estimation frameworks like the Kalman filter. It measures the distance between a new sensor measurement and the filter's predicted measurement, normalized by the innovation covariance.

• Mechanism: A measurement is flagged as an outlier if its squared Mahalanobis distance exceeds a threshold from a chi-squared distribution (e.g., p=0.95).
• Use Case: Standard in Kalman Filters, EKF, and UKF for validating incoming measurements from LiDAR, radar, or GPS before the update step.
• Key Insight: It accounts for the estimated uncertainty and correlations between state variables, making it more robust than simple Euclidean distance.

RANSAC is an iterative, hypothesis-and-test algorithm designed to estimate model parameters from a dataset containing a large proportion of outliers.

• Mechanism: It randomly selects minimal subsets of data to form candidate models, then evaluates consensus by counting inliers within a tolerance threshold. The model with the most inliers is selected.
• Use Case: Fundamental in computer vision for tasks like estimating essential matrices in visual odometry or plane fitting in LiDAR point clouds, where many data points may belong to irrelevant objects.
• Limitation: Performance degrades when the inlier ratio is very low (<50%), and it is computationally intensive for large datasets.

These are simple, non-parametric methods that identify outliers based on the distribution of a single scalar measurement stream.

• Z-Score: Assumes a Gaussian distribution. Data points exceeding a threshold (e.g., |Z| > 3) are considered outliers. Sensitive to non-Gaussian data.
• Interquartile Range (IQR): A robust, distribution-agnostic method. Outliers are points below Q1 - 1.5IQR or above Q3 + 1.5IQR, where Q1 and Q3 are the 25th and 75th percentiles.
• Use Case: Pre-processing for individual sensor channels (e.g., filtering obviously erroneous temperature readings from a thermistor or spike noise from a single-axis accelerometer) before fusion.

DCS is a robust kernel-based method integrated directly into the Kalman filter update. Instead of binary rejection, it gracefully down-weights the influence of potential outliers.

• Mechanism: It scales the measurement covariance matrix inversely based on the Mahalanobis distance. A large innovation leads to an inflated covariance, reducing the Kalman gain and thus the impact of that measurement.
• Advantage over Gating: Avoids the brittle 'accept/reject' decision. A single extreme outlier is heavily discounted without being completely ignored, which can be safer for system stability.
• Use Case: Preferred in safety-critical inertial navigation systems and visual-inertial odometry (VIO) where continuous, smooth correction is required.

In geometric registration algorithms like Iterative Closest Point (ICP), outlier rejection is built into the correspondence matching step.

• Mechanism: Methods like Trimmed ICP or Threshold Rejection discard point correspondences based on distance. Only the best 80-90% of matches (or those below a distance threshold) are used to compute the alignment transform in each iteration.
• Use Case: Essential for LiDAR scan matching and point cloud registration in dynamic environments, where moving objects (people, cars) generate many spurious point correspondences that would derail alignment.

This method leverages the physical constraints of the system over time to identify implausible measurements.

• Mechanism: It checks if a new measurement violates limits on physically possible rates of change (e.g., maximum acceleration, jerk, or angular velocity). A LiDAR scan placing an object 10 meters away when it was 1 meter away 0.01 seconds prior is physically impossible.
• Implementation: Often uses a simple process model (constant velocity/acceleration) to predict a measurement envelope. Measurements outside this envelope are rejected.
• Use Case: Critical for high-frequency sensors (IMU, radar) and in object tracking pipelines to reject sensor ghosts, multipath reflections, and sporadic noise bursts.

The core statistical gating mechanism within recursive Bayesian filters like the Kalman filter. The test compares the measurement residual (innovation) against its predicted statistical distribution.

Process:

1. Predict the next state and its covariance.
2. Predict what the measurement should be and its covariance (S).
3. When a real measurement arrives, compute the innovation z - Hx.
4. Compute the squared Mahalanobis distance of the innovation.
5. If the distance exceeds a threshold (e.g., 95% percentile of the χ² distribution), the measurement is rejected as an outlier.

The critical preceding step to outlier rejection in many perception tasks. Before a measurement can be tested, it must be assigned to a known landmark, object, or track. Incorrect association is a primary source of outliers.

Methods include:

• Nearest Neighbor with gating.
• Joint Probabilistic Data Association (JPDA) which considers multiple hypotheses.
• Global Nearest Neighbor (GNN) via assignment algorithms.

Failed or low-confidence data association often results in a measurement being treated as a potential outlier or used to initialize a new track.