Out-of-Distribution Detection

The foundational baseline method that uses the maximum softmax probability as a confidence score. In-distribution samples typically yield high maximum probabilities, while OOD inputs produce lower, more uniform distributions. A threshold is set on the maximum predicted probability; inputs falling below it are flagged. This method is computationally cheap but suffers from overconfidence—deep neural networks can assign high probabilities to nonsensical inputs due to the softmax function's forced normalization.

A theoretically grounded alternative to softmax that uses the Helmholtz free energy of a model's logits. The energy score is defined as E(x) = -T * log(Σ exp(f_i(x)/T)), where T is a temperature parameter. Lower energy indicates in-distribution, higher energy indicates OOD. This method aligns with Gibbs distributions and avoids the overconfidence pitfall of softmax by not forcing probability normalization. It consistently outperforms softmax baselines on standard benchmarks.

Rather than operating on output logits, these methods analyze the penultimate layer embeddings of a neural network. The core assumption is that in-distribution features cluster tightly in a learned manifold, while OOD features map to distant regions. Common approaches include:

• Mahalanobis distance: Measures distance from class-conditional Gaussian distributions fitted to training embeddings
• Cosine similarity: Compares angular distance to class prototypes
• K-Nearest Neighbors: Flags inputs whose embedding neighbors in the training set are too distant

A class of methods that analyzes the gradient information of a loss function with respect to model parameters for a given input. The intuition is that in-distribution samples produce small, consistent gradients because the model has already learned their patterns. OOD inputs, being unfamiliar, induce large, erratic gradient norms. GradNorm is a prominent example, computing the L1 norm of gradients from the KL divergence between softmax output and a uniform distribution. High gradient norms indicate OOD.

A critical distinction in deployment strategy:

• OOD Detection is a binary task: is this input from the training distribution or not? It rejects everything unfamiliar.
• Open-Set Recognition is more nuanced: it must correctly classify known classes while also identifying inputs from unknown classes that were absent during training. This requires the model to maintain a compact decision boundary for known classes while reserving open space for novelty. Open-set methods often use reciprocal point learning or generative negative mining.

A post-hoc calibration technique that smooths the softmax distribution using a learned temperature parameter T > 1. The logits are divided by T before softmax: σ(f(x)/T). This reduces overconfidence by softening peaked distributions, making OOD detection via thresholding more reliable. The optimal temperature is found by minimizing the negative log-likelihood on a held-out validation set. Critically, temperature scaling does not change the argmax prediction—it only adjusts confidence scores.

A closely related but distinct task where the model is trained only on in-distribution (ID) data and must identify any input that deviates from this learned manifold. Unlike open-set recognition, novelty detection does not require classifying the known classes; it simply flags the unfamiliar. Common techniques include One-Class SVM and Deep Support Vector Data Description (Deep SVDD) , which learn a minimal hypersphere enclosing the normal data in the feature space.

A formal framework that requires a classifier to simultaneously accurately classify known classes and reject unknown classes not seen during training. This moves beyond simple anomaly detection by demanding dual competence. Key algorithms like OpenMax replace the standard Softmax layer with a recalibrated probability distribution that estimates the likelihood of an input belonging to an unknown class, preventing overconfident misclassifications on novel defect types.

A parametric statistical method for OOD detection that measures the distance of a test sample's feature representation from the class-conditional Gaussian distribution of the training data. Unlike Euclidean distance, it accounts for the covariance structure of the data. A high Mahalanobis score indicates the sample lies far from the training manifold. This method is computationally efficient and provides a principled confidence score derived from the feature space of pre-trained neural networks.

A framework that assigns a scalar energy value to each input configuration, where in-distribution data is assigned low energy and OOD data high energy. The Energy Score for OOD detection uses the log-sum-exp of the model's logits, avoiding the overconfidence issues of the Softmax probability. This approach is architecturally agnostic and can be applied to any pre-trained classifier without retraining, making it a practical, plug-and-play solution for production systems.

A method that leverages the vector norm of the gradient of the Kullback-Leibler divergence between the model's output and a uniform distribution. The intuition is that OOD inputs cause much larger gradient magnitudes when the model is forced to align with a uniform prediction. This technique is hyperparameter-free and does not require access to OOD samples for tuning, making it highly practical for industrial deployment where novel failure modes are unknown a priori.

A practical method that combines temperature scaling and input pre-processing to amplify the separability between ID and OOD samples. By slightly perturbing the input in the direction of the gradient of the Softmax score, ID samples become more confident while OOD samples remain diffuse. This technique requires no architectural changes and can be applied to any pre-trained network, making it a popular baseline for industrial visual inspection systems.