Open Set Recognition

Traditional classifiers operate under a closed-world assumption, forcing every input into one of K known classes. OSR introduces a (K+1)-th class representing 'unknown.'

• Closed-Set: A model trained on BPSK, QPSK, and 16QAM will classify an 8PSK signal as one of those three, creating a silent misclassification.
• Open-Set: The same model outputs 'unknown' for 8PSK, triggering an alert for further analysis.
• This distinction is vital in electronic warfare where encountering novel threat emitters is the primary operational concern.

OSR mathematically balances empirical classification risk on known classes against open space risk—the risk of labeling an unknown input as known.

• Open Space Risk is defined over the infinite space far from known training data.
• A robust OSR model minimizes this risk by tightly bounding the decision regions around known classes.
• This prevents the model from being overconfident in regions of the feature space where no training data exists, a common failure mode of SoftMax-based classifiers.

OSR often leverages Extreme Value Theory to model the statistical behavior of the tails of class distributions, enabling calibrated rejection decisions.

• EVT fits a Weibull, Fréchet, or Gumbel distribution to the distances between known samples and their class means.
• This provides a probabilistic framework for determining if a new sample is too far from any known distribution to be considered in-distribution.
• The result is a statistically principled threshold rather than an arbitrary heuristic, critical for spectrum enforcement applications requiring defensible decisions.

OSR can be implemented through two distinct architectural philosophies, each with trade-offs for RF applications.

• Discriminative models (e.g., OpenMax) replace the SoftMax layer with a mechanism that estimates the probability of an input belonging to an unknown class.
• Generative models (e.g., GANs, VAEs) learn the probability distribution of known classes and reject samples with low likelihood.
• In spectrum monitoring, generative approaches excel at detecting rogue emitters because they model 'normal' RF background activity and flag statistical deviations.

OpenMax is a foundational OSR algorithm that augments a neural network's penultimate layer with a calibrated unknown class.

• It fits a Weibull distribution to the activation vectors of correctly classified training samples for each known class.
• At inference, it computes the distance of a new sample's activation vector from each class's mean activation vector (MAV).
• These distances are used to redistribute SoftMax probability mass, explicitly allocating probability to an 'unknown' pseudo-class when the input is far from all known MAVs.

In dynamic spectrum environments, OSR provides a principled framework for separating known interferers from genuinely novel threats.

• A system can be trained on known signal types: Wi-Fi, LTE, radar, and known jammers.
• When a new LPI waveform or a previously unseen drone control signal appears, OSR correctly identifies it as unknown rather than misclassifying it.
• This capability is essential for spectrum enforcement agencies tracking unauthorized transmissions and for defense systems requiring tactical situational awareness of novel emitters.

Regulatory bodies deploy OSR models to automatically detect unauthorized transmissions in licensed bands. The model identifies known cellular and broadcast waveforms while flagging rogue emitters as unknown anomalies, triggering enforcement workflows. This is critical for resolving interference complaints and protecting primary users from harmful intrusion without requiring a pre-existing signature for every possible illegal transmitter.

In contested electromagnetic environments, adversaries deploy Low Probability of Intercept (LPI) waveforms designed to evade conventional detection. OSR models trained on known friendly and commercial signals can isolate novel threat emitters as out-of-distribution events. This enables real-time threat library updates, where an unknown signal is flagged, geolocated, and escalated for expert analysis without prior exposure to that specific adversarial waveform.

Satellite operators use OSR to monitor transponder health and detect interference or hijacking attempts. The system learns the nominal spectral signature of legitimate carriers. Any unexpected signal—whether accidental cross-polarization interference or a deliberate satellite piracy attempt—is immediately flagged as an anomaly. This prevents service degradation without maintaining an exhaustive database of every possible interference pattern.

Factory floors rely on deterministic wireless protocols. OSR models monitor the ISM band to distinguish known automation traffic from anomalous emissions. When a malfunctioning sensor generates spurious transmissions or a jamming device enters the facility, the system detects the deviation from learned normality. This prevents production line stoppages by identifying the root cause as an unknown signal source rather than a protocol error.

Future dynamic spectrum access networks require secondary users to vacate channels when primary users return. OSR enables a cognitive radio to classify known primary user waveforms while treating any other detected energy as a potential incumbent. This conservative approach prevents harmful interference to legacy systems that may use proprietary or undocumented waveforms, ensuring regulatory compliance in shared spectrum bands.

Power grids and water utilities rely on licensed telemetry links. OSR models continuously monitor these bands, learning the normal communication patterns of SCADA and teleprotection equipment. The sudden appearance of an unknown signal—potentially a precursor to a coordinated physical or cyber attack—is detected as an anomaly. This provides early warning of reconnaissance activity without needing a signature for novel attack tools.

A neural one-class classification method that learns to map normal spectrum data into a minimal hypersphere in feature space. During inference, samples falling outside this learned boundary are classified as anomalies.

• Trains exclusively on known, authorized signal classes
• Minimizes sphere volume while maximizing enclosed normal samples
• Anomaly score: Euclidean distance from sphere center in embedding space
• Effective when anomaly samples are unavailable or highly heterogeneous

Deep SVDD provides a principled geometric approach to defining the boundary between known signal types and the open set of potential unknown emitters.

A detection paradigm where autoencoders trained on normal I/Q samples learn to compress and reconstruct known signal patterns. High reconstruction error on novel signal types serves as the anomaly indicator.

• LSTM Autoencoders: Capture temporal dependencies in sequential spectrum data
• Convolutional Autoencoders: Exploit spectral structure in frequency-domain representations
• Variational Autoencoders (VAEs): Provide probabilistic reconstruction with uncertainty estimates

The key insight: models cannot accurately reconstruct patterns they have never seen, making reconstruction error a powerful open-set discriminator.

A training paradigm where models learn rich representations from unlabeled spectrum data by solving auxiliary tasks, enabling anomaly detection without explicit labels for every signal type.

• Contrastive learning: Pull augmented views of the same signal together, push different signals apart
• Masked signal modeling: Predict masked portions of spectrograms or I/Q sequences
• Rotation prediction: Classify applied transformations to learn structural features

These representations create a feature space where known signals cluster tightly, leaving open space for unknown classes to appear as outliers during deployment.

A replacement for the standard SoftMax classification layer that explicitly models the probability of an input belonging to an unknown class. OpenMax recalibrates activation vectors using distances to class prototypes.

• Fits Weibull distributions to distances between correct class activations and mean activation vectors
• Recalibrates scores to include an explicit unknown class probability
• Rejects inputs when the unknown probability exceeds known class probabilities
• Enables standard deep classifiers to operate in open-world settings without retraining

OpenMax bridges the gap between high-accuracy closed-set classifiers and the practical need to detect novel, unauthorized signal types.

A statistical framework for modeling the tails of score distributions, enabling rigorous threshold setting for open set recognition. EVT provides theoretical guarantees on false positive rates when declaring signals as unknown.

• Fits Generalized Pareto Distribution to extreme anomaly scores
• Models the probability of observing scores beyond current maxima
• Enables calibrated rejection with specified confidence levels
• Particularly effective for rare, high-impact anomaly events in spectrum monitoring

EVT transforms heuristic anomaly thresholds into statistically principled decision boundaries, critical for mission-critical spectrum enforcement applications.