Out-of-Distribution (OOD) Detection

The most basic OOD technique uses the maximum softmax probability (MSP) as a proxy for epistemic certainty. In-distribution inputs typically produce high-confidence predictions, while OOD inputs yield lower, more uniform probability distributions across classes.

• Mechanism: A threshold is set on the maximum class probability; inputs below this threshold are rejected.
• Limitation: Modern neural networks can produce overconfident softmax outputs for OOD data, making this baseline unreliable in isolation.
• Example: A CIFAR-10 classifier might assign 99% confidence to a random noise image, demonstrating why raw softmax is insufficient.

Energy-based models map inputs to a scalar energy value, where in-distribution data occupies low-energy basins and OOD data maps to higher energy states. The energy function is derived directly from the logits of a discriminative classifier without architectural changes.

• Formula: E(x) = -T · log Σᵢ exp(fᵢ(x)/T), where T is a temperature parameter.
• Advantage: Energy scores are theoretically aligned with the model's probability density, making them less susceptible to overconfidence than softmax.
• Use Case: Rejecting adversarial inputs that lie far from the training manifold in the logit space.

This method fits class-conditional Gaussian distributions to the penultimate layer embeddings of a trained model. The Mahalanobis distance measures how many standard deviations an input's embedding is from the nearest class centroid.

• Process: Compute the mean and covariance matrix for each class's embeddings, then calculate the minimum distance for a new input.
• Strength: Leverages the structured, semantically meaningful representations learned by deep networks, providing a more geometrically principled OOD score.
• Variation: A small amount of input noise can be added to sharpen the separation between in-distribution and OOD distance scores.

OOD inputs can be identified by analyzing the magnitude and consistency of gradients during backpropagation. In-distribution samples produce coherent, low-magnitude gradients, while unfamiliar inputs generate noisy, high-magnitude gradient signals.

• GradNorm: A simple metric that uses the vector norm of gradients with respect to a KL-divergence loss.
• Behavior: OOD data forces the model to make large, conflicting parameter updates to accommodate the anomalous input.
• Application: Particularly effective in continual learning scenarios where the model must detect distributional shift without access to the original training data.

Normalizing flows learn an explicit, tractable probability density of the training data by transforming a simple base distribution through a series of invertible mappings. The negative log-likelihood under this learned density serves as a direct OOD score.

• Mechanism: p(x) is computed exactly via the change-of-variables formula, unlike VAEs which provide a lower bound.
• Failure Mode: Surprisingly, flows trained on images can assign higher likelihood to constant-value or noise images than to real images, a phenomenon known as the "typicality" problem.
• Mitigation: Use likelihood ratios or complexity-adjusted scores to correct for this counter-intuitive behavior.

A non-parametric, tree-based method that isolates observations by randomly selecting a feature and a split value. Anomalous points require fewer random partitions to be isolated, resulting in shorter average path lengths.

• Integration: Applied to the embedding vectors extracted from a frozen pre-trained model, decoupling the OOD detector from the classifier's training.
• Efficiency: Linear time complexity with low memory requirements, making it suitable for high-throughput agent output validation pipelines.
• Advantage: Does not assume any underlying distribution of the embeddings, making it robust to non-Gaussian feature geometries.

A set of statistical methods that measure a model's confidence in its predictions, enabling systems to abstain from acting on low-certainty outputs. Unlike OOD detection, which identifies unfamiliar input distributions, uncertainty quantification estimates the model's own epistemic or aleatoric doubt.

• Conformal prediction provides distribution-free coverage guarantees
• Monte Carlo Dropout approximates Bayesian inference at test time
• Deep Ensembles measure disagreement across independently trained models

A minimum probability score that an agent's output or classification must exceed to be considered valid. Outputs falling below this threshold are rejected or flagged for review. This acts as a downstream gate that leverages OOD detection signals.

• Softmax probabilities are often poorly calibrated in modern neural networks
• Temperature scaling can improve calibration without changing accuracy
• Thresholds should be tuned per-task using a held-out validation set

A metric that measures the uncertainty of an LLM's output by clustering semantically equivalent generations and calculating entropy across meaning clusters, rather than just token sequences. This addresses the limitation that token-level entropy fails to capture when an LLM is confidently wrong.

• Uses natural language inference models to cluster equivalent meanings
• Detects confabulation where token probabilities are high but meaning is inconsistent
• More computationally expensive than naive token entropy

Input perturbations that cause misclassification or erroneous actions in multimodal and embodied agent systems. Adversarial examples exploit blind spots in the model's learned decision boundaries, often appearing in-distribution to naive detectors.

• FGSM and PGD attacks craft imperceptible perturbations
• Adversarial training augments training data with attacked examples
• OOD detectors must be robust to adaptive adversaries who optimize inputs to evade detection

A resilience pattern that automatically halts an agent's operation or tool access when a predefined failure threshold or anomaly rate is exceeded. OOD detection scores often serve as the trigger signal for circuit breakers in production agentic systems.

• Three states: closed (normal), open (halted), half-open (testing recovery)
• Prevents cascading failures from a single anomalous input
• Should be combined with graceful degradation rather than hard crashes

A security mechanism that runs an agent's generated code or tool calls inside an isolated, ephemeral environment to prevent access to the host system. When OOD detection flags an input as anomalous, sandboxing provides a secondary containment layer.

• gVisor and Firecracker provide lightweight VM-level isolation
• Docker containers offer process-level sandboxing with namespace isolation
• Ephemeral environments are destroyed after each execution, limiting blast radius