Likelihood Regret

Standard deep generative models often assign higher likelihoods to OOD data than in-distribution data due to the dominance of background statistics. Likelihood Regret solves this by computing the log-ratio between a domain-specific model and a background model trained on general, agnostic data. This subtraction cancels out common complexity factors like local pixel correlations, isolating the semantic novelty signal. The final score is defined as LLR(x) = log p_θ(x) − log p_θ₀(x), where p_θ₀ is the background reference.

This metric distinguishes between two types of novelty:

• Statistical Novelty: Detected by raw likelihood, often fooled by simple backgrounds.
• Semantic Novelty: Detected by the regret ratio, isolating high-level features. By leveraging a background model that captures universal low-level statistics, Likelihood Regret ensures that only inputs with semantically foreign concepts are flagged as OOD. This prevents false positives on high-contrast or blurry in-distribution images.

Likelihood Regret is not tied to a specific generative architecture. It functions as a post-hoc scoring wrapper applicable to:

• Autoregressive models (PixelCNN, Transformers)
• Flow-based models (RealNVP, Glow)
• Variational Autoencoders (VAEs) As long as the model provides a tractable log-likelihood, the regret can be computed. This flexibility allows security engineers to integrate it into existing ML pipelines without retraining the primary classifier.

The effectiveness of Likelihood Regret is highly sensitive to the design of the background model. If the background model is too expressive or trained on data too similar to the in-distribution set, it will learn semantic features rather than just low-level statistics. This causes regret collapse, where the subtraction cancels out the semantic signal, making OOD inputs indistinguishable from in-distribution data. Optimal background models are typically shallow or trained on a highly diverse, generic dataset.

Computing Likelihood Regret requires two full forward passes per input: one through the domain model and one through the background model. For large generative models, this effectively doubles inference latency. Optimization strategies include:

• Shared feature extraction: Using a common backbone for both models.
• Batch processing: Evaluating the background model on aggregated inputs. In latency-critical security applications, this overhead must be weighed against the improved detection accuracy compared to single-model methods like MSP.

Likelihood Regret is closely related to the concept of typicality. Standard likelihood tests fail because OOD inputs can fall into high-density regions of the model that are not 'typical' for the data manifold. Regret implicitly enforces a typicality constraint by requiring the input to be specifically likely under the domain model relative to the background. This aligns the metric with the information-theoretic definition of a typical set, rejecting inputs that are merely high-probability noise.

A statistical evaluation that directly addresses the core failure mode of raw likelihood scores. While a generative model might assign a high likelihood to a nonsensical input, the typicality test rejects it for failing to reside in the typical set of the model's learned distribution. Likelihood Regret implicitly corrects for this by comparing against a background model, but explicit typicality tests use information-theoretic metrics to determine if a sample is a true member of the high-probability mass.

A probabilistic framework that assigns low energy values to in-distribution data and high energy to OOD data. The Helmholtz free energy serves as a discriminative score that is theoretically aligned with likelihood but is often more robust in practice. Likelihood Regret operates on a similar principle—using a corrected likelihood ratio—while EBMs directly learn an energy landscape that separates known from unknown inputs without requiring a background model for normalization.

A training-time strategy that leverages an auxiliary dataset of outliers to teach the model heuristics for detecting unknown inputs. Unlike Likelihood Regret, which is a post-hoc metric requiring no retraining, Outlier Exposure explicitly fine-tunes the model to push OOD samples into low-confidence regions. This approach significantly improves generalization to unseen OOD distributions but requires careful curation of the outlier dataset to avoid biasing the model against specific anomaly types.

An open set recognition algorithm that replaces the standard softmax layer with a Weibull-calibrated activation vector, explicitly modeling the probability of an 'unknown' class. It fits an Extreme Value Theory distribution to the logits of known classes and recalibrates scores to include an explicit rejection option. Likelihood Regret addresses the same open-set problem but from a generative perspective, using density ratios rather than discriminative logit recalibration to flag unfamiliar inputs.

A parametric OOD detection method that computes the distance of a feature representation to the nearest class-conditional Gaussian distribution. By capturing the covariance structure of the training data, it provides a more nuanced confidence measure than simple Euclidean distance. Likelihood Regret similarly accounts for input complexity through its background model comparison, but the Mahalanobis approach operates in feature space rather than probability space, making it computationally lighter for deployed classifiers.

A generative modeling approach that transforms a simple base distribution into a complex one through a sequence of invertible mappings, enabling exact likelihood computation. Normalizing Flows provide a tractable density that can be used directly for OOD detection. Likelihood Regret is often computed using such exact-likelihood models, where the flow serves as the trained model and a simpler distribution acts as the background model to correct for complexity bias in the raw likelihood scores.