Out-of-Distribution Detection

A baseline method that uses the maximum softmax probability as a proxy for epistemic certainty. High confidence does not imply correctness—deep neural networks are notoriously overconfident on OOD inputs. This approach establishes a rejection threshold: if the highest predicted probability falls below a calibrated value, the input is flagged for human review. Modern implementations often replace raw softmax with temperature-scaled probabilities or energy-based scores to improve separability between in-distribution and OOD samples.

A parametric method that models the class-conditional feature distributions as multivariate Gaussians in the penultimate layer. The Mahalanobis distance measures how many standard deviations a test point's feature representation lies from the closest class mean, accounting for feature covariance. Key advantages:

• Provides a calibrated confidence interval with formal statistical grounding
• Captures inter-feature correlations that softmax ignores
• Requires only a single forward pass plus a lightweight distance computation
• Effective for detecting both semantic shift and covariate shift

Leverages the magnitude of backpropagated gradients as an OOD indicator. When a model processes an unfamiliar input, the KL divergence between the output distribution and a uniform target produces larger gradient norms compared to in-distribution samples. This technique exploits the fact that OOD inputs require more substantial parameter updates to fit. The GradNorm method computes the L1 norm of gradients flowing from the final classification layer, providing a computationally efficient score that correlates strongly with epistemic uncertainty.

Uses invertible neural networks to learn the exact likelihood of the training distribution. Normalizing flows transform complex data into a simple latent space where density is tractable. OOD detection occurs by thresholding the log-likelihood under the learned distribution. Critical considerations:

• Flow-based models can paradoxically assign higher likelihood to OOD inputs due to background statistics
• Modern variants use likelihood ratios comparing semantic vs. background models
• Computationally expensive but provides the most principled probabilistic framework

Operates on the principle that OOD samples map to low-density regions of the learned feature manifold. Techniques include:

• k-Nearest Neighbors: Computing the distance to the k-th nearest training sample in embedding space
• Deep Nearest Neighbors: Using normalized penultimate features with cosine similarity
• Local Outlier Factor: Identifying samples whose local density deviates significantly from their neighbors These non-parametric approaches require storing a representative subset of training embeddings but offer strong empirical performance without distributional assumptions.

Exploits the variance across multiple independently trained models or Monte Carlo Dropout samples. In-distribution inputs produce consistent predictions across the ensemble, while OOD inputs cause high predictive disagreement. Metrics include:

• Mutual Information between model outputs and the ensemble
• Jensen-Shannon Divergence across predictive distributions
• Variance of predicted probabilities This Bayesian-inspired approach naturally captures both aleatoric and epistemic uncertainty, making it robust against adversarial OOD examples designed to fool single-model detectors.

Programmatic constraints and filters applied to AI inputs and outputs to enforce safety policies, prevent toxicity, and maintain topical relevance. OOD detection serves as a critical input guardrail, rejecting data that falls outside the model's known competency.

• Input Guardrails: OOD detectors, semantic filters, and format validators
• Output Guardrails: Factuality checks, toxicity classifiers, and schema validators
• Layered Defense: OOD detection is the first line of defense before the model processes data

A stability pattern that automatically stops requests to a failing AI service to prevent cascading failures. When an OOD detector signals a sustained anomaly rate above a threshold, a circuit breaker can halt inference entirely.

• Closed State: Normal operation, requests flow freely
• Open State: Requests are immediately rejected without processing
• Half-Open State: A limited number of requests are allowed to test if the system has recovered
• Integration: OOD rate spikes can trigger circuit breaker transitions

A deployment pattern where a new model runs in parallel with the production model, receiving live traffic and logging predictions without affecting the user-facing response. Shadow mode is essential for calibrating OOD detectors before they go live.

• Silent Validation: Compare OOD scores against production outcomes without risk
• Threshold Tuning: Establish rejection thresholds based on real traffic patterns
• A/B Comparison: Evaluate if the new detector catches anomalies the current system misses

The immutable audit trail that records the exact model version, input data, and parameters that produced a specific automated decision. When an OOD detector flags an input for human review, decision provenance ensures complete traceability.

• Artifact Linking: Connects the rejected input to the OOD score and detector version
• Regulatory Compliance: Supports the right to explanation under GDPR and the EU AI Act
• Forensic Analysis: Enables post-hoc investigation of why an input was flagged

A design principle ensuring that when an AI component fails, the system continues to operate with reduced functionality rather than failing completely. OOD detection enables graceful degradation by routing uncertain inputs to fallback logic.

• Fallback Strategies: Rule-based systems, human review queues, or conservative defaults
• User Experience: Instead of a hard error, users receive a qualified response with lower confidence
• Resilience Engineering: OOD detection is a key enabler of non-binary failure modes