Entropic Open-Set Loss

The core mechanism penalizes the network when it produces a confident, low-entropy prediction for any sample that does not belong to a known class. By maximizing the entropy of the output distribution for unknown inputs, the loss function forces the model to express maximum uncertainty—a uniform probability mass across all known classes. This creates a clear, measurable separation: known classes yield sharp, peaked SoftMax outputs, while unknown classes produce flat, high-entropy vectors that can be easily thresholded for rejection.

The loss function combines two competing objectives in a single training framework:

• Known Sample Objective: Standard categorical cross-entropy loss drives the network to produce low-entropy, high-confidence predictions for correctly labeled training samples.
• Unknown Sample Objective: An entropic regularizer penalizes confident predictions on auxiliary outlier data or synthetically generated background samples, pushing their output distributions toward uniformity. This dual pressure shapes a feature space where known classes form compact, high-density clusters while the space between them is explicitly modeled as a high-entropy rejection zone.

Entropic Open-Set Loss enables a simple, statistically grounded rejection rule. During inference, the entropy of the predicted probability vector is computed as:

H(p) = -Σ p_i log(p_i)

• Low entropy (< threshold): The model is confident; the input is classified as the argmax known class.
• High entropy (≥ threshold): The model is uncertain; the input is rejected as unknown or novel. The threshold is typically calibrated on a held-out validation set to achieve a desired balance between open-set rejection rate and closed-set accuracy, often using the Area Under the ROC Curve (AUROC) as the selection metric.

Entropic Open-Set Loss is frequently paired with Outlier Exposure techniques to provide the model with explicit examples of what it should reject. During training, an auxiliary dataset of diverse outlier samples—drawn from unrelated modulation schemes, noise distributions, or synthetically generated signals—is fed through the network. The entropic loss term is applied to these outliers, teaching the network to map them to uniform distributions. This explicit exposure prevents feature collapse, where unknown inputs inadvertently map to known class centroids, and significantly improves generalization to truly novel modulation types unseen during both training and outlier exposure.

The entropy of the SoftMax output serves as a proxy for epistemic uncertainty—the model's uncertainty due to lack of knowledge about a given input. Entropic Open-Set Loss directly optimizes this signal:

• Unlike post-hoc methods like Temperature Scaling or ODIN, which recalibrate an already-trained model, entropic loss shapes the feature representations during training.
• It contrasts with Evidence Deep Learning, which places a Dirichlet distribution over predictions, by operating directly on the SoftMax output rather than on second-order probability distributions.
• The resulting entropy score is computationally cheap, requiring no additional forward passes or ensemble members, making it suitable for real-time spectrum classification on edge hardware.

Both Entropic Open-Set Loss and Objectosphere Loss aim to separate known and unknown samples, but they operate on different properties of the feature representation:

• Entropic Open-Set Loss works in the probability space, enforcing uniform SoftMax outputs for unknowns.
• Objectosphere Loss works in the feature magnitude space, maximizing the L2 norm of embeddings for known samples while minimizing it for unknowns. Entropic loss has the advantage of producing a naturally interpretable rejection score (entropy in bits), while Objectosphere loss requires careful tuning of a magnitude threshold. In practice, the two can be combined to create a joint penalty that enforces both low feature magnitude and high output entropy for unknown modulation schemes.

A complementary loss function that creates a magnitude-based separation in the feature embedding space. It explicitly maximizes the feature vector norm for known classes while minimizing it for unknown samples. This creates a thresholdable rejection space where a simple norm cutoff can discriminate between in-distribution and out-of-distribution inputs. Unlike entropic loss which operates on probability distributions, Objectosphere operates directly on the penultimate layer activations, making it highly effective when combined with entropy-based objectives in a multi-task training setup.

The statistical framework that underpins many open-set loss calibrations, including OpenMax. EVT models the tail behavior of distance distributions—specifically, the distances between correctly classified samples and their class means. By fitting a Weibull distribution to these tail distances, the model can estimate the cumulative distribution function of extreme events and compute a calibrated probability that a query sample belongs to an unknown class. This provides a principled alternative to heuristic entropy thresholds.

A class of models that learn an energy function mapping inputs to a scalar energy value. In-distribution data is assigned low energy, while out-of-distribution data receives high energy. The Helmholtz free energy can be derived from any classifier's logits and used as a discriminative score for novelty detection. This approach aligns naturally with entropic open-set loss because high-entropy predictions correspond to high free energy, creating a unified theoretical bridge between probabilistic and energy-based rejection.

A regularization technique that directly improves open-set performance by training the model with an auxiliary dataset of diverse outlier examples. During training, the model is forced to produce high-entropy, uniform predictions on these synthetic or real outlier samples. This teaches the network to tighten its decision boundary around known classes. When combined with entropic open-set loss, outlier exposure provides explicit negative examples that reinforce the uniform-distribution objective for unknown regions of the feature space.

A framework that replaces standard SoftMax outputs with Dirichlet distributions over class probabilities, treating predictions as subjective opinions with quantifiable uncertainty. The model outputs evidence for each class, and the total evidence inversely relates to predictive uncertainty. Unknown samples naturally accumulate low total evidence, producing high epistemic uncertainty. This provides a principled alternative to entropic loss by directly modeling the second-order probability distribution rather than forcing uniform first-order probabilities.

The theoretical foundation motivating entropic open-set loss. Open space risk quantifies the relative measure of feature space that is far from any known training data yet still classified as known. Entropic loss directly addresses this risk by forcing the network to map unknown regions to high-entropy, low-confidence probability distributions. This effectively pushes the decision boundary inward around known classes, minimizing the volume of open space that could be incorrectly labeled as a known modulation scheme.