Out-of-Distribution Detection

The simplest baseline method. Uses the maximum softmax probability as a proxy for model certainty. In-distribution inputs typically yield high confidence, while OOD inputs produce lower, more uniform probability distributions.

• Mechanism: score = max(softmax(logits))
• Limitation: Modern neural networks can be overconfident on OOD inputs, assigning high softmax scores to nonsensical data.
• Variants: Temperature scaling and input perturbation can sharpen the confidence gap.

Uses the Helmholtz free energy function to derive an OOD score directly from logits, without requiring a softmax operation. Lower energy indicates higher compatibility with the training distribution.

• Formula: E(x) = -T * log(sum(exp(f_i(x)/T)))
• Advantage: Outperforms softmax confidence by leveraging the full logit vector, not just the maximum.
• Tuning: The temperature parameter T controls the sensitivity of the score.

Fits class-conditional Gaussian distributions to the embeddings of a pre-trained model's penultimate layer. OOD detection is performed by computing the Mahalanobis distance to the nearest class centroid.

• Process: Compute (z - μ_c)^T Σ^{-1} (z - μ_c) for each class c.
• Strength: Accounts for feature covariance, unlike Euclidean distance.
• Requirement: Needs a small set of in-distribution data to estimate the mean μ_c and covariance Σ.

Enhances softmax-based detection with two post-processing steps: temperature scaling and input pre-processing via gradient-based perturbation.

• Step 1: Scale logits by a high temperature T to soften the softmax distribution.
• Step 2: Add a small perturbation -ε * sign(gradient) to the input, pushing it toward the predicted class.
• Effect: Amplifies the confidence gap between in-distribution and OOD inputs, making them more separable.

Trains a neural network to map normal data into a compact hypersphere in a latent space. Anomalies and OOD samples fall outside this learned boundary.

• Deep SVDD: Minimizes the volume of a hypersphere enclosing normal embeddings.
• Loss Function: min(||φ(x) - c||²) where c is the hypersphere center.
• Application: Effective for fraud when only legitimate transaction data is available for training.

Leverages the gradient information of a model's loss with respect to its parameters. OOD inputs produce distinct gradient patterns compared to in-distribution data.

• GradNorm: Uses the vector norm of gradients from the KL divergence between softmax and a uniform distribution.
• Insight: The magnitude and direction of parameter gradients encode distributional membership.
• Cost: Requires a backward pass, adding computational overhead during inference.

The semi-supervised sibling of OOD detection. A model is trained exclusively on clean, normal data to learn a tight decision boundary. The goal is to identify if new, unseen data points deviate from this learned normality.

• Key Difference: Assumes no outliers in training data.
• Use Case: Training an autoencoder on legitimate credit card swipes to flag physically impossible terminal locations.
• Core Mechanism: Often uses One-Class SVM or Deep SVDD to encapsulate the normal class.

A classification paradigm where the model must not only classify known classes but also reject unknown classes not seen during training. It balances closed-set accuracy with open-set rejection capability.

• Mechanism: Replaces the standard Softmax layer with a probabilistic model like Extreme Value Theory (EVT) to calibrate the "unknown" likelihood.
• Finance Application: A merchant category classifier that says "I don't know" instead of forcing a new, fraudulent storefront into a known retail bucket.

A multi-dimensional statistical measure that accounts for the covariance structure of the training data. Unlike Euclidean distance, it rescales axes to be unit-variance, making it sensitive to correlations.

• OOD Scoring: Compute the distance of a test point to the class-conditional Gaussian distribution of the training data.
• Advantage: Detects multivariate outliers that look normal on individual feature histograms but break the correlation (e.g., high amount + low frequency).

A framework that assigns a single scalar energy to each input configuration. Normal data is assigned low energy, while OOD data receives high energy. The Helmholtz Free Energy of a neural network's logits serves as a powerful, non-probabilistic OOD score.

• Advantage: Avoids the overconfidence issue of Softmax probabilities.
• Implementation: Often uses Langevin dynamics for sampling or contrastive divergence for training.

A probabilistic model assuming data is generated from a mixture of finite Gaussian distributions. OOD detection is performed by thresholding the log-likelihood of a sample under the fitted density.

• DAGMM Integration: Deep Autoencoding Gaussian Mixture Models jointly train an autoencoder and a GMM to estimate density in the latent space.
• Strength: Captures multi-modal normality (e.g., distinct behavioral clusters for retail vs. travel spending).

A self-supervised paradigm that pulls positive pairs (augmentations of the same data) closer and pushes negative pairs apart in an embedding space. This creates a highly structured latent space where OOD samples naturally fall into low-density regions.

• Method: SimCLR or SupCon loss functions.
• Benefit: Learns semantic invariances without labels, making it robust to novel fraud patterns that lack historical labels.