Open Set Recognition

The fundamental challenge of OSR is formally bounding open space risk—the measurable danger of labeling an unknown input as known. A robust OSR model must define a compact abatement probability for each known class, ensuring that feature space far from any training sample is labeled 'unknown' rather than incorrectly assigned to the nearest known emitter. This is achieved through techniques like extreme value theory to model the boundary between known and unknown space.

OSR systems rely on learning a latent representation where known classes form tight, separable clusters with significant inter-class margins. The empty space between clusters becomes the unknown rejection zone. Key techniques include:

• Center loss to minimize intra-class variance
• Angular margin penalties to maximize inter-class separation
• Contrastive learning to explicitly push unknown-like samples away from known centroids

Rather than relying solely on the maximum softmax probability—which is notoriously unreliable for unknown inputs—modern OSR systems apply calibrated thresholds to activation statistics. Common approaches include:

• Energy-based scores derived from logit magnitudes
• Mahalanobis distance from class-conditional Gaussian fits in feature space
• OpenMax, which recalibrates softmax scores using a Weibull distribution fitted to extreme distances per class

While related, these paradigms differ critically:

• Novelty Detection: Assumes only normal data during training; detects any deviation as novel. No multi-class discrimination.
• Open Set Recognition: Trains on multiple known classes and must simultaneously perform fine-grained classification of knowns while rejecting unknowns. This dual objective makes OSR significantly more challenging and directly applicable to RF emitter identification where many legitimate devices must be distinguished from rogue transmitters.

Closed-set accuracy is dangerously misleading for OSR. Proper evaluation requires:

• Open Set Classification Rate (OSCR) curves that plot correct known classification against false positive rate as threshold varies
• Area Under the ROC Curve (AUROC) for binary known-vs-unknown detection
• F1-score at a fixed, low false positive rate relevant to security applications
• Closed-set accuracy on known classes only, reported separately from rejection performance

A powerful training strategy involves synthesizing proxy unknowns to teach the model where the open space boundary should lie. Techniques include:

• Generative Adversarial Networks (GANs) trained to produce samples at the fringes of known class distributions
• Manifold sampling that generates points on the learned data manifold but far from any class centroid
• Mixup and CutMix variants that create ambiguous samples between known classes, forcing the model to assign low confidence to boundary regions

The core process that OSR protects. SEI uniquely identifies a physical radio transmitter by analyzing distinct, unintentional features in its waveform. While a closed-set SEI classifier assumes every signal belongs to a known device, an open set SEI system adds the critical capability to reject signals from transmitters never seen during training, preventing spoofing by cloned or rogue hardware.

A deep learning training methodology essential for building discriminative OSR embedding spaces. The model learns to pull representations of signals from the same device closer together while pushing representations from different devices apart. This creates a structured feature space where unknown emitters naturally fall into low-density regions far from known clusters, making them easier to detect and reject.

A critical preprocessing step for OSR systems using algorithms like Principal Component Analysis (PCA) or t-SNE. By projecting high-dimensional RF fingerprint vectors into a lower-dimensional space, redundancy and noise are removed while preserving the discriminative structure between known classes. This compact representation makes the subsequent open-set decision boundary more robust and computationally efficient to calculate.

A deliberate, often imperceptible perturbation crafted by an adversary to fool a deep learning classifier. In an OSR context, attacks can aim to cause a known emitter to be rejected as unknown (denial-of-service) or, more dangerously, cause an unknown rogue emitter to be accepted as a legitimate, enrolled device. Robust OSR models must be hardened against these evasion attacks.

An adaptive mechanism that updates a device's stored fingerprint model over time to account for gradual, environmentally-induced changes. Without drift compensation, a legitimate device experiencing temperature variation or component aging may see its signature drift outside its known class boundary, causing an OSR system to falsely reject it as an unknown. This maintains the stability of the open-set decision threshold.

A zero-trust security model where a device's physical-layer fingerprint is verified persistently throughout a session. OSR is fundamental here: the system must continuously distinguish between the legitimate enrolled device and any unknown transmitter that might attempt session hijacking. A single successful rejection of an unknown emitter during a session prevents a complete security breach.