Mahalanobis Distance Score

The method operates by fitting a class-conditional Gaussian distribution to the feature representations of the training data for each known class. It assumes that the deep features of each class can be modeled by a multivariate normal distribution, defined by a mean vector and a covariance matrix. This parametric approach allows for a compact, closed-form representation of the in-distribution data manifold.

Unlike Euclidean distance, which treats all feature dimensions independently, the Mahalanobis distance uses the inverse of the covariance matrix to scale and rotate the feature space. This accounts for correlations between features, effectively measuring distance in units of standard deviation along the principal components of the class distribution. A point far along a low-variance direction is penalized more heavily than one far along a high-variance direction.

The score is computed by extracting the feature vector from a penultimate layer of a pre-trained neural network, not the final logit layer. For a test sample, the distance to the nearest class-conditional Gaussian is calculated. A high score indicates the sample is far from all known class manifolds and is therefore out-of-distribution. This leverages the rich, discriminative representations learned by the classifier.

At inference time, the method requires only a single forward pass to extract features, followed by a simple matrix-vector multiplication against the pre-computed precision matrix (inverse covariance). This makes it significantly faster than ensemble or generative model-based OOD detectors. The class means and covariance matrices are calculated once and stored, enabling low-latency deployment in production systems.

Performance is highly sensitive to the input data distribution. The method is often paired with input pre-processing techniques, such as adding a small, controlled perturbation to the input that is designed to increase the softmax score for in-distribution data. This widens the separation gap between the Mahalanobis scores of in-distribution and OOD samples, significantly improving detection accuracy.

The core limitation is the estimation of a reliable covariance matrix in high-dimensional feature spaces. With limited training samples per class, the empirical covariance matrix can become singular or ill-conditioned, making its inverse unstable. This necessitates dimensionality reduction techniques or shrinkage estimators to regularize the covariance, adding complexity to the training pipeline.

The core task of identifying inputs that are semantically or statistically different from a model's training data. The Mahalanobis distance score is a specific parametric technique used to solve this problem by modeling class-conditional Gaussian distributions in feature space.

• Prevents unpredictable predictions on unknown concepts
• Critical for safety-critical AI systems
• Relies on feature representations from pre-trained networks

A baseline OOD detection method that uses the highest softmax output score as a confidence measure. Unlike the Mahalanobis distance, MSP ignores the covariance structure of features, making it less effective at detecting inputs that lie far from the training manifold but still trigger high confidence.

• Simple to implement with no extra computation
• Often overconfident on OOD inputs
• Serves as a common benchmark for comparison

A probabilistic framework that assigns low energy values to in-distribution data and high energy to OOD data. The Helmholtz free energy function acts as a discriminative score. While Mahalanobis uses class-conditional Gaussians, EBMs learn an energy landscape directly from data.

• Aligns with physical interpretations of probability
• Can be trained jointly with classifiers
• Provides a unified view of generative and discriminative models

A method that trains multiple independent models with different initializations and averages their predictions. The disagreement between ensemble members provides robust uncertainty estimates. When combined with Mahalanobis distance, ensemble feature spaces create richer representations for density estimation.

• Improves both accuracy and OOD detection
• Computationally expensive at training time
• Provides epistemic uncertainty quantification

A classification framework requiring models to accurately classify known classes while simultaneously rejecting samples from unknown classes. The Mahalanobis distance score is a natural fit for this task, as it thresholds the distance to the nearest class-conditional Gaussian to decide rejection.

• Extends closed-world classification assumptions
• Requires explicit 'unknown' modeling
• Critical for real-world deployment scenarios

A training strategy that leverages an auxiliary dataset of outliers to teach the model heuristics for detecting unknown inputs. When used with Mahalanobis distance, outlier exposure can refine the feature space to better separate in-distribution clusters from potential OOD regions.

• Significantly improves generalization to unseen OOD distributions
• Requires careful selection of auxiliary data
• Can be combined with contrastive learning objectives