Energy-Based Model (EBM)

The core of an EBM is a scalar function E(x) that maps input data points to a single energy value. The model is trained to minimize energy for observed training data while shaping the energy landscape to be high elsewhere. Unlike standard classifiers that only learn decision boundaries, EBMs learn the full data density. This allows them to explicitly identify inputs that fall into high-energy, low-probability regions as OOD. The energy is unnormalized, meaning it does not need to integrate to one, removing the computational bottleneck of calculating a partition function during inference.

For discriminative EBMs, the Helmholtz free energy is used as a direct OOD detection score. Given a classifier with logits, the free energy is computed as the negative log-sum-exp of the logits: E(x) = -T log Σ exp(f_i(x)/T). This elegantly combines the model's confidence across all classes into a single density measure. A high free energy indicates the input is far from all known class manifolds. This is mathematically aligned with the model's likelihood and provides a theoretically grounded alternative to brittle softmax thresholding.

EBMs are often trained using Contrastive Divergence (CD) or Stochastic Gradient Langevin Dynamics (SGLD). The training loop involves two phases:

• Positive Phase: Lower the energy of real, in-distribution data samples.
• Negative Phase: Raise the energy of generated or sampled points from the model's current distribution. This contrastive process carves the energy landscape, creating low-energy valleys around the training data manifold. The resulting energy surface naturally repels OOD inputs, which have not been explicitly pushed down during training.

Unlike autoregressive models or normalizing flows, EBMs provide an implicit density representation. They do not require a tractable likelihood or a constrained architecture. The model only needs to output a scalar energy. This flexibility allows EBMs to use powerful, unconstrained neural network backbones like modern vision transformers or convolutional networks without worrying about invertibility or auto-regressive factorization. The implicit nature makes them highly scalable to high-dimensional data like images and sensor telemetry, where explicit density models often struggle.

A unique property of EBMs is the ability to refine predictions via gradient descent on the input space. If an input is ambiguous, one can compute the gradient of the energy with respect to the input and move the sample towards a lower energy state. For OOD detection, this can be used to check if an input can be 'pulled' into a low-energy manifold. OOD samples typically require large, destructive changes to reach low energy, while in-distribution samples are refined with minimal, semantic adjustments. This provides a dynamic, interpretable verification mechanism.

EBMs naturally separate aleatoric and epistemic uncertainty. The energy value itself represents the model's density estimation—a low energy indicates high confidence that the input is in-distribution. By ensembling multiple EBMs or using Langevin dynamics to explore the energy landscape, one can measure the variance in energy assignments. High variance indicates epistemic uncertainty (model disagreement), while consistently high energy across samples indicates the input is fundamentally OOD. This decomposition is critical for safety-critical systems that must distinguish between noisy known data and truly unknown threats.