Variational Autoencoder (VAE)

Unlike standard autoencoders that map inputs to a single point, a VAE learns a probability distribution over the latent space. For RF analysis, this means the model captures the mean and variance of normal signal features. This allows the VAE to not just reconstruct signals, but to sample new, plausible signal variations, creating a rich, continuous model of normality against which new samples are scored.

VAEs use reconstruction probability, not just reconstruction error, for anomaly detection. This metric combines the likelihood of a sample under the learned latent distribution with its reconstruction fidelity. A signal with a low reconstruction probability is a statistical outlier. This is more robust than simple error thresholds because it accounts for the model's inherent uncertainty about a given input, reducing false positives from noisy but normal signals.

The VAE loss function includes a Kullback-Leibler (KL) divergence term that forces the learned latent distribution to be close to a prior, typically a standard Gaussian. This acts as a regularizer, preventing the model from memorizing training data and encouraging a smooth, continuous latent space. For RF applications, this ensures that small, legitimate variations in a signal's parameters do not trigger an anomaly alert, as they map to nearby points in the latent space.

A trained VAE is a generative model. By sampling from the latent space, it can synthesize new, realistic I/Q samples or spectrograms of normal RF signals. This is invaluable for training downstream classifiers or other anomaly detectors when real-world anomalous data is scarce. It can also be used to simulate rare but normal signal conditions to harden the anomaly detection system against false alarms.

The VAE's probabilistic framework is well-suited for modeling the complex, multi-modal distributions of real-world spectrum data. It can learn a single, unified latent space that captures the characteristics of multiple normal signal types (e.g., Wi-Fi, LTE, and radar) simultaneously. An anomaly is then any signal that falls into a low-probability region of this joint distribution, enabling effective open-set recognition without needing a pre-defined catalog of all possible anomalies.

A VAE is trained exclusively on the I/Q samples of authorized transmitters within a monitored band. The model learns a compressed, probabilistic latent space representing normal signal morphology. During inference, an unauthorized or rogue emitter's signal will map to a low-probability region of this latent space, yielding a high reconstruction error and a low log-likelihood score, triggering an immediate alert for spectrum enforcement agencies.

LPI signals, such as those using direct-sequence spread spectrum, are engineered to hide below the noise floor, making traditional energy detection useless. A VAE learns the subtle, high-order statistical structure of normal noise and known signals. An LPI transmission introduces a non-random statistical anomaly. The VAE's Kullback-Leibler divergence term and reconstruction probability score can flag this deviation, exposing a hidden transmitter.

Every transmitter has unique, microscopic hardware imperfections (e.g., I/Q imbalance, oscillator phase noise) that manifest as subtle, consistent distortions on the modulated waveform. A VAE can be trained on the raw I/Q constellation of a specific device to learn its unique 'fingerprint' as a probability distribution. A signal claiming to be from that device but generated by a different one will be a statistical outlier in the VAE's latent space, enabling physical-layer authentication.

Beyond simple detection, a conditional VAE (CVAE) can be trained on a labeled dataset of common interference types (e.g., co-channel, adjacent-channel, intermodulation). The model learns a structured latent space where different interference morphologies are separated. An unknown signal can be encoded, and its latent vector's proximity to known interference clusters provides a probabilistic classification, enabling automated and specific mitigation strategies.

A VAE can model the complex temporal dynamics of spectrum usage. By training on sequences of spectrograms representing normal daily/weekly usage patterns, the VAE learns a generative model of spectrum occupancy. The model can then be used to predict future occupancy states and, crucially, to detect anomalies where observed usage deviates significantly from the predicted high-probability pattern, indicating an unexpected transmission or a change in emitter behavior.

In electronic warfare, new and unknown signal types appear constantly. A VAE trained on all known modulation schemes (e.g., QPSK, 16-QAM) learns a manifold of 'known' signals. A genuinely novel signal type will not lie on this manifold. By thresholding the reconstruction probability, the system can perform open set recognition, cleanly separating known signals from unknown, novel ones that require further analysis by a SIGINT specialist.