Out-of-distribution Detection (OOD Detection)

The foundational approach that models the probability distribution of training data and flags inputs in low-density regions as OOD.

• Parametric methods fit a known distribution (e.g., Gaussian mixture models) to learned features
• Non-parametric methods use kernel density estimation or nearest-neighbor distances
• Deep generative models like normalizing flows and variational autoencoders learn explicit likelihood functions

A critical limitation: deep generative models can paradoxically assign higher likelihood to OOD inputs than in-distribution data, a phenomenon known as the "OOD detection failure mode" identified by Nalisnick et al. (2019).

Classifies inputs based on their distance from training samples in a learned feature space, operating on the assumption that OOD inputs map to distant regions.

• Mahalanobis distance computes class-conditional distances in feature space with a covariance-aware metric
• k-Nearest Neighbors (k-NN) uses the distance to the k-th nearest training sample as an OOD score
• Deep Nearest Neighbors applies k-NN on penultimate layer representations of a pre-trained classifier

Distance-based methods are computationally efficient at inference but require storing or indexing the entire training set, creating memory trade-offs for large-scale deployment.

Derives OOD scores directly from a classifier's output probabilities or logits without modifying the model architecture.

• Maximum Softmax Probability (MSP) uses the highest predicted probability as a confidence score; OOD inputs typically produce lower maxima
• Energy-based scoring computes the free energy function from logits, providing better separation than MSP
• ODIN applies temperature scaling and input perturbations to amplify the gap between in-distribution and OOD softmax scores

These methods are post-hoc and require no retraining, making them the most practical for existing deployed systems. Energy-based scores consistently outperform MSP in benchmark evaluations.

Exploits the gradient signal of a neural network with respect to its inputs to identify distributional shifts.

• GradNorm measures the magnitude of gradients of the KL divergence between softmax outputs and a uniform distribution
• OOD inputs typically produce larger gradient norms than in-distribution samples under this formulation
• The method leverages the fact that a well-trained network is in a "flatter" minimum for familiar inputs

Gradient-based approaches require backpropagation at inference time, increasing computational overhead compared to output-based methods but often yielding superior detection AUROC.

Examines the geometric and topological properties of learned representations to separate in-distribution from OOD samples.

• Residual-based methods measure reconstruction error when projecting features onto learned principal components
• Virtual-logit matching constructs a proxy OOD class by matching features to the nearest in-distribution class boundary
• Contrastive learning pre-training naturally creates more separable feature spaces for OOD detection

Feature space methods are particularly effective when combined with self-supervised pre-training, as the representations capture richer semantic structure than those from supervised learning alone.

Modifies the training procedure to explicitly prepare the model for OOD detection rather than relying on post-hoc scoring.

• Outlier exposure fine-tunes models on auxiliary outlier datasets to learn a decision boundary between in-distribution and OOD
• Confidence loss penalizes high-confidence predictions on OOD examples, encouraging uniform output distributions
• LogitNorm constrains logit magnitudes during training, producing more calibrated confidence scores

These methods require access to auxiliary OOD data during training, which may not represent all possible deployment-time distribution shifts. The choice of auxiliary data significantly impacts generalization.

The process of estimating and characterizing the total predictive uncertainty of a model's output, decomposed into aleatoric uncertainty (inherent data noise) and epistemic uncertainty (model ignorance). In OOD detection, high epistemic uncertainty signals that an input lies far from the training distribution.

• Bayesian neural networks and Monte Carlo Dropout are common approaches
• Critical for distinguishing between ambiguous in-distribution cases and truly novel inputs
• Provides a continuous risk score rather than a binary in/out decision

A model-agnostic framework that produces prediction sets with a rigorous, finite-sample guarantee of marginal coverage without assuming a specific data distribution. For OOD detection, conformal methods can flag inputs where the prediction set is empty or abnormally large.

• Provides distribution-free statistical guarantees
• Works as a wrapper around any pre-trained classifier
• Enables regulatory-friendly uncertainty reporting with provable error rates

The ability of a machine learning model to maintain accurate predictions when presented with intentionally perturbed inputs designed to cause misclassification. Robustness and OOD detection are deeply linked: adversarially trained models often develop better representations of the training manifold.

• Adversarial examples are a specific, worst-case form of distribution shift
• Gradient-based attacks like FGSM and PGD test model stability
• Robust features learned during adversarial training improve OOD separability

A scalar summary statistic that measures the discrepancy between a model's predicted confidence scores and its empirical accuracy. A well-calibrated model should be uncertain on OOD inputs, but modern neural networks are often overconfident on far-from-distribution samples.

• Computed by binning predictions and comparing average confidence to accuracy
• Temperature scaling is a common post-hoc calibration method
• Poor calibration undermines the reliability of confidence-based OOD detection

A regulatory mechanism allowing manufacturers of AI/ML-enabled medical devices to pre-specify planned modifications without requiring a new marketing submission. OOD detection is a key component of PCCPs, as it defines the operational envelope where the device is validated.

• Enables continuous learning within a controlled, pre-authorized scope
• Requires explicit monitoring protocols for distribution shift
• Links OOD detection directly to post-market surveillance obligations

A set of best practices and standards for developing, validating, and monitoring AI/ML-enabled medical devices to ensure their safety and effectiveness. GMLP explicitly addresses data representativeness and performance monitoring under distribution shift.

• Principle 7: Focus on the performance of the human-AI team
• Principle 9: Emphasizes statistically sound testing on clinically relevant subgroups
• OOD detection operationalizes GMLP's mandate for monitoring real-world performance drift