Out-of-Distribution Detection

The fundamental capability to detect when deployment data diverges from the training distribution. This involves measuring the statistical distance between input features and the model's learned manifold.

• Covariate shift: Input feature distribution changes while the conditional relationship remains stable
• Concept drift: The relationship between inputs and targets itself changes over time
• Semantic shift: Entirely new classes or categories appear that were absent during training

Effective OOD detectors must distinguish between epistemic uncertainty (lack of knowledge) and aleatoric uncertainty (inherent noise).

Probabilistic approaches that model the training data distribution and flag low-likelihood inputs as OOD. These methods compute a probability density score for each input.

• Gaussian Mixture Models fit a parametric distribution to learned feature representations
• Normalizing Flows transform complex distributions into simple base distributions for exact likelihood computation
• Energy-based models assign low energy to in-distribution samples and high energy to OOD inputs

Density methods struggle with high-dimensional data where likelihood can be counterintuitively high for OOD samples.

Techniques that measure the distance in embedding space between a test sample and the training data. Samples far from any training cluster are classified as OOD.

• Mahalanobis distance computes class-conditional distances using feature covariance
• k-Nearest Neighbors thresholds based on distance to the k-th nearest training sample
• Prototype learning maintains class centroids and rejects samples exceeding a radius threshold

These methods are computationally efficient but sensitive to the quality of learned representations.

Methods that analyze the model's own predictions to identify uncertain or anomalous inputs without modifying the architecture.

• Maximum Softmax Probability uses the highest predicted probability as a confidence score
• Entropy-based detection flags inputs with high predictive entropy across classes
• ODIN applies temperature scaling and input perturbations to widen the gap between in-distribution and OOD scores
• Energy score derives a scalar from the logit outputs that separates distributions more effectively than softmax

These post-hoc methods require no retraining and integrate easily with existing deployed models.

The operational response triggered when an input is classified as OOD. Detection without appropriate action provides no safety benefit.

• Abstention: The agent refuses to act and returns a controlled uncertainty signal
• Human escalation: The input is routed to a human operator for manual review
• Conservative policy: The agent defaults to a minimal-risk action or safe state
• Graceful degradation: The system reduces autonomy level while maintaining partial functionality

Fallback design must balance safety against operational continuity to avoid excessive false-positive interruptions.

A formal framework extending classification to handle unknown classes at test time. Unlike closed-set classification, open-set systems must simultaneously classify known classes and reject unknown ones.

• OpenMax replaces the softmax layer with a Weibull-calibrated rejection mechanism
• Generative open-set methods synthesize unknown-class examples for training
• Reciprocal point learning models the complement of the known feature space

Open-set recognition is essential for agents deployed in dynamic environments where novel inputs are inevitable.

The root cause that OOD detection addresses. A distributional shift occurs when the statistical properties of deployment data diverge from training data. Types include:

• Covariate shift: Change in input distribution P(X)
• Label shift: Change in output distribution P(Y)
• Concept drift: Change in the relationship P(Y|X) OOD detectors act as the first line of defense, triggering a safe fallback when a shift is detected before the model produces an erroneous prediction.

A broader field encompassing OOD detection. Anomaly detection identifies individual samples that deviate from expected patterns, which may include OOD inputs but also in-distribution outliers. Key distinctions:

• OOD detection focuses on semantic novelty (new classes, domains)
• Anomaly detection catches statistical outliers (rare events, defects)
• Both use density estimation, reconstruction error, and one-class classification In agentic systems, anomaly detection monitors behavioral patterns while OOD detection guards input integrity.

The classification paradigm that formalizes OOD detection. Open Set Recognition requires a model to:

• Correctly classify known classes seen during training
• Reject unknown classes as "none of the above" This contrasts with closed-set classification, which forces every input into a known category. OSR is essential for autonomous agents operating in unbounded environments where encountering novel objects, commands, or states is inevitable.

A decision framework where a model can abstain from making a prediction when confidence is low. Components include:

• A predictor that generates outputs
• A selection function that decides whether to output or abstain
• A coverage vs. risk trade-off curve OOD detection serves as the selection function, enabling agents to gracefully decline unsafe actions rather than executing with high uncertainty. This is critical for corrigibility—the ability to accept human override.

The type of uncertainty that OOD detection aims to quantify. Epistemic uncertainty arises from lack of knowledge—the model doesn't know because it hasn't seen similar data. This differs from:

• Aleatoric uncertainty: Inherent noise in the data (e.g., sensor error)
• Epistemic uncertainty is reducible with more training data
• Aleatoric uncertainty is irreducible Bayesian neural networks and deep ensembles explicitly model epistemic uncertainty, providing a principled signal for OOD rejection thresholds.

The reinforcement learning paradigm that constrains agent behavior to known-safe regions. Safe exploration relies on OOD detection to:

• Identify novel states where the policy is untested
• Trigger conservative fallback policies or human consultation
• Build uncertainty-aware world models that flag risky transitions Without OOD detection, an RL agent may confidently execute catastrophic actions in unfamiliar environments—a failure mode known as optimism bias in the face of novelty.