Mahalanobis Distance

The core property of the Mahalanobis distance is its incorporation of the covariance matrix of the dataset. Unlike Euclidean distance, which treats all dimensions as equally important and independent, Mahalanobis distance scales the space. It accounts for:

• Correlations between variables: It shrinks distances along axes of high correlation and stretches them along axes of low correlation.
• Variance of each feature: Dimensions with high variance are given less weight, preventing them from dominating the distance calculation. This results in a unitless, scale-invariant metric where one unit of distance corresponds to one standard deviation along the principal component axes of the data.

Points at a constant Mahalanobis distance from the mean of a multivariate Gaussian distribution form an ellipsoid (or ellipse in 2D). This is a direct geometric consequence of using the covariance matrix. Key implications:

• Natural cluster shape: It defines the shape of the data's distribution, allowing for more accurate membership testing.
• Gating in tracking: In multi-object tracking, a validation gate is often defined as an ellipsoidal region where the Mahalanobis distance between a predicted track and a new measurement is below a threshold. This is more efficient than a circular gate, reducing false associations.
• Anomaly detection: Observations falling outside a predefined ellipsoidal boundary (e.g., distance > 3 standard deviations) can be flagged as statistical outliers.

For a point x drawn from a p-dimensional multivariate normal distribution with mean μ and covariance matrix Σ, the squared Mahalanobis distance follows a chi-square distribution with p degrees of freedom.

• Formula: ( D^2_M = (x - \mu)^T \Sigma^{-1} (x - \mu) \sim \chi^2_p )
• Statistical testing: This property allows for rigorous probabilistic interpretation. You can calculate the p-value that a point belongs to the distribution. For example, a squared distance corresponding to the 95th percentile of the ( \chi^2_p ) distribution defines a 95% confidence ellipsoid.
• Robustness in practice: This theoretical foundation is why Mahalanobis distance is preferred over heuristic distances for statistical filtering, such as in the innovation gate of a Kalman filter.

The Mahalanobis distance is highly sensitive to the accuracy of the estimated mean and, especially, the covariance matrix. This leads to important practical considerations:

• Requires sufficient data: The covariance matrix must be estimated reliably, which typically requires many more samples than the number of dimensions (n >> p) to avoid a singular or ill-conditioned matrix.
• Robustness to outliers: Standard sample mean and covariance are not robust. A single outlier can severely distort the covariance estimate, making the distance metric unreliable. Solutions include using robust covariance estimators (e.g., Minimum Covariance Determinant).
• Use in high dimensions: In very high-dimensional spaces (common in deep learning features), the covariance matrix becomes enormous and singular. Here, diagonal approximations (assuming independence) or regularization (adding a small value to the diagonal) are often applied, reverting it toward a scaled Euclidean distance.

In robotics and embedded systems, Mahalanobis distance is a workhorse for low-latency, probabilistic data association and state estimation.

• Sensor Fusion Gating: In a Kalman Filter or Extended Kalman Filter, the innovation (difference between predicted and actual measurement) is evaluated using its Mahalanobis distance to determine if a sensor reading is a valid match for a track.
• Multi-Object Tracking (MOT): Used in the Hungarian algorithm for optimal assignment, where the cost matrix is often populated with Mahalanobis distances between track predictions and new detections.
• Anomaly Detection for Sensor Health: Sudden changes in the Mahalanobis distance of incoming IMU or LiDAR feature vectors can indicate sensor failure or unexpected environmental conditions.
• Efficiency: The computation involves a matrix inversion, which can be pre-computed for a known distribution, and a few matrix multiplications, making it suitable for real-time systems.

Mahalanobis distance generalizes simpler metrics by considering the data's internal structure.

• Euclidean Distance: Assumes a spherical distribution where all axes are uncorrelated and have unit variance. It is a special case of Mahalanobis distance where the covariance matrix is the identity matrix.
• Manhattan Distance: Measures distance along axes at right angles (L1 norm). It does not account for correlations or scaling.

When to use which:

• Use Mahalanobis when your features are correlated and have different scales, and you have a reliable model of the "normal" data distribution (e.g., for anomaly detection within a known operational envelope).
• Use Euclidean for raw, normalized, or PCA-transformed features where correlations have been removed.
• Use Manhattan for robustness to outliers in low-dimensional, grid-like data (e.g., some image processing tasks).

The Kalman filter is an optimal recursive algorithm for estimating the state of a linear dynamic system from a series of noisy measurements. It operates in a two-step cycle:

• Prediction: Projects the current state estimate forward in time using the system's dynamic model.
• Update (Correction): Fuses the prediction with a new measurement, weighting each by their respective covariances.

The Mahalanobis distance is used within the update step for innovation gating, to determine if a new measurement is statistically probable given the current state prediction, thereby rejecting outliers.

Multi-Object Tracking (MOT) is the task of detecting and maintaining the identities of multiple objects over time in a video or sensor stream. A critical sub-problem is data association—determining which new detection corresponds to which existing track.

Mahalanobis distance is a standard metric in global nearest neighbor (GNN) and joint probabilistic data association (JPDA) frameworks. It measures the distance between a predicted track position (with its covariance) and a new detection, enabling the matching of detections to the most statistically likely track.

The covariance matrix is a square matrix that encapsulates the variances and correlations between multiple random variables. For a multivariate distribution, it defines the shape and orientation of the distribution's ellipsoidal contours.

Mahalanobis distance is fundamentally defined using the inverse of the covariance matrix: D² = (x - μ)ᵀ Σ⁻¹ (x - μ). This inverse operation:

• Normalizes each dimension by its variance.
• De-correlates the variables, transforming the ellipsoidal distribution into a unit sphere.
• This allows for a true standardized distance measure that accounts for scale and correlation.

Euclidean distance is the "ordinary" straight-line distance between two points in Euclidean space, calculated as the square root of the sum of squared differences. It is a special case of Mahalanobis distance.

Key Differences:

• Assumptions: Euclidean distance assumes all variables are uncorrelated and have equal variance (unit covariance matrix, Σ = I).
• Sensitivity: It is sensitive to the scale of measurements. A variable with a large variance will dominate the distance.
• Use Case: Use Euclidean distance for physical distances in a normalized, isotropic space. Use Mahalanobis distance for statistical distances in correlated, anisotropic feature spaces common in sensor data and machine learning.

The chi-squared (χ²) distribution is the probability distribution of a sum of the squares of independent standard normal variables. Its degrees of freedom equal the number of variables summed.

Under the assumption that the data point x comes from the multivariate normal distribution N(μ, Σ), the squared Mahalanobis distance follows a chi-squared distribution.

This property is crucial for statistical gating:

• The value D² (the squared Mahalanobis distance) can be compared to a threshold from the χ² distribution.
• For example, with 2 degrees of freedom (e.g., x-y position), a threshold of χ²(0.95, 2) ≈ 5.991 defines a 95% confidence ellipsoid. Points with D² > 5.991 are considered statistical outliers.

Outlier detection identifies data points that deviate significantly from the majority of a dataset. Mahalanobis distance is a powerful, multivariate method for this task.

Process:

1. Estimate the mean (μ) and covariance matrix (Σ) from a (presumed clean) dataset.
2. For a new point, compute its Mahalanobis distance to the estimated distribution.
3. Flag the point as an outlier if the distance exceeds a threshold derived from the χ² distribution.

Advantages over univariate methods:

• Considers correlations between features (e.g., a slightly high temperature and a very low pressure might be normal, but not vice versa).
• Automatically accounts for different scales and variances of features. It is widely used in fraud detection, sensor fault diagnosis, and cleaning training data for machine learning models.