Out-of-Distribution Detection

OOD detection fundamentally relies on modeling the probability density function of the training distribution. Inputs falling below a calibrated likelihood threshold are flagged as anomalous.

• Explicit Density Models: Normalizing flows and PixelCNN++ directly compute log-likelihoods, though they can paradoxically assign high likelihood to OOD inputs due to background statistics.
• Energy-Based Models (EBM): Use the Helmholtz free energy function E(x) = -T log ∑ᵢ e^{fᵢ(x)/T} as a scoring metric, consistently outperforming softmax confidence for OOD separation.
• Typicality Test: Addresses the failure mode of standard likelihood models by evaluating if an input resides in the typical set rather than just high-density regions.

Post-hoc methods repurpose a pre-trained classifier's output layer to derive OOD scores without modifying the original training pipeline, enabling immediate deployment on existing models.

• Maximum Softmax Probability (MSP): The baseline approach that uses maxₖ p(y=k|x) as a confidence proxy, though it suffers from overconfidence on OOD inputs.
• ODIN: Applies temperature scaling and input pre-processing perturbations to amplify the separability between in-distribution and OOD softmax scores.
• GradNorm: Exploits the observation that the L1 norm of gradients w.r.t. the predicted class is consistently smaller for OOD inputs, providing a hyperparameter-free detection signal.

Proactive methods alter the training objective to explicitly enforce a margin between in-distribution confidence and OOD uncertainty, building the detector directly into the model's inductive bias.

• Outlier Exposure (OE): Augments training with a diverse auxiliary OOD dataset and penalizes the model for producing high-confidence predictions on these outliers.
• Confidence Loss: Trains a branch to predict a confidence score c(x) and applies a loss that rewards low confidence on OOD samples while maintaining accuracy on in-distribution data.
• VOS (Virtual Outlier Synthesis): Synthesizes virtual outliers at the class-conditional boundary in feature space during training, eliminating the need for real OOD data.

The operational deployment of OOD detection translates detection scores into actionable abstention decisions, directly throttling the API surface available to membership inference attackers.

• Risk-Coverage Trade-off: Defines a policy that rejects the fraction of inputs with the lowest OOD scores, balancing model utility against the privacy risk of responding to probes.
• Conformal OOD: Wraps OOD scoring in a conformal prediction framework to provide finite-sample, distribution-free guarantees on the false detection rate.
• Graduated Response: Instead of binary reject/accept, returns degraded predictions (e.g., coarse-grained labels or delayed responses) for borderline OOD scores, masking the precise confidence boundary.

A distribution-free statistical framework that wraps any pre-trained model to produce prediction sets with a finite-sample coverage guarantee. Instead of a single point prediction, it outputs a set of plausible labels. For OOD detection, if the prediction set is empty or abnormally large, the input is flagged as non-conforming to the training distribution. This provides a rigorous, assumption-light alternative to softmax thresholding.

A decision-making paradigm where the model has the option to abstain from predicting on inputs it deems uncertain. This is implemented via a rejection rule, often based on maximum softmax probability or entropy. By refusing to answer on OOD inputs, selective classifiers deny attackers the high-confidence predictions required for membership inference probes, effectively throttling the information leakage channel.

The discipline of estimating a model's confidence in its predictions, decomposed into two distinct types:

• Aleatoric Uncertainty: Inherent noise in the data (e.g., sensor noise). Irreducible.
• Epistemic Uncertainty: Model ignorance due to lack of data (e.g., an OOD input). Reducible with more data. OOD detection primarily exploits high epistemic uncertainty, often measured via Bayesian neural networks or deep ensembles.

A scalar measure of the disorder in a model's output probability vector, calculated as -Σ p(y_i) * log p(y_i). High entropy indicates a confused, flat distribution across classes—a strong signal of an OOD input. Low entropy on a wrong class signals overconfident failure. This metric is a lightweight, post-hoc OOD score requiring no model modification.

A post-hoc calibration technique that smooths a model's output probabilities by dividing the logits by a learned scalar parameter T > 0. When T > 1, the softmax distribution is softened, reducing the confidence gap between in-distribution and OOD samples. While primarily used for calibration, it can be tuned to mask the overconfident signals that membership inference attacks exploit.

A parametric OOD detection method that models the feature representations of each class as a multivariate Gaussian distribution. For a new input, it computes the distance to the nearest class-conditional Gaussian. A large distance indicates the input lies far from the training manifold. This method operates on intermediate layer activations and is more robust than softmax-based scores for detecting semantically anomalous inputs.