Kullback-Leibler Divergence

The KL divergence is always greater than or equal to zero, a property formally known as Gibbs' inequality. A value of zero is achieved if and only if the two distributions, P and Q, are identical almost everywhere. This provides a mathematically sound baseline for anomaly scoring: a perfect match to the normal signal model yields a divergence of zero, while any deviation results in a positive score.

KL divergence is not a true distance metric because it is asymmetric: D_KL(P || Q) ≠ D_KL(Q || P). This has profound practical implications for anomaly detection:

• Forward KL (D_KL(P || Q)): "Mean-seeking" behavior. Minimizing this forces the model Q to cover all modes of the true distribution P, which is useful for ensuring a detector covers all normal signal variations.
• Reverse KL (D_KL(Q || P)): "Mode-seeking" behavior. Minimizing this forces Q to fit a single high-probability mode of P, useful for precisely modeling the most common signal type.

KL divergence measures the expected excess surprise from using distribution Q as a model when the true distribution is P. It is the difference between the cross-entropy H(P, Q) and the entropy H(P): D_KL(P || Q) = H(P, Q) - H(P). In spectrum monitoring, this represents the extra bits needed to encode an observed signal segment using a model of normal behavior instead of the true anomalous distribution.

For detecting anomalies, KL divergence is intimately connected to the log-likelihood ratio test, the most powerful statistical test for simple hypotheses. The divergence D_KL(P || Q) is the expected value of the log-likelihood ratio log(P(x)/Q(x)) under the true distribution P. A high divergence for a signal segment directly implies a low likelihood of it belonging to the normal class Q, making it a theoretically optimal anomaly score.

KL divergence is invariant under invertible transformations of the random variable. If Y = f(X) where f is a bijective function, then D_KL(P_X || Q_X) = D_KL(P_Y || Q_Y). This is a critical advantage over simpler metrics like Euclidean distance in feature space, as the anomaly score remains consistent whether you are analyzing raw I/Q samples, power spectral density, or a learned feature embedding.

The KL divergence D_KL(P || Q) is jointly convex in the pair (P, Q). This means that for any two pairs of distributions, the divergence of a weighted average is less than or equal to the weighted average of the divergences. This property is crucial for optimization, guaranteeing that algorithms like Expectation-Maximization (EM), often used to fit Gaussian Mixture Models for normal signal baselines, will converge to a global minimum.

KL Divergence measures the information loss when a reference distribution (learned from normal spectrum data) is used to approximate a test distribution (current observation). In anomaly detection, a high divergence score indicates the current signal segment is statistically distinct from expected behavior.

• Reference distribution (P): A histogram or Gaussian Mixture Model of normal power spectral density
• Test distribution (Q): The empirical distribution of the incoming signal segment
• Anomaly threshold: Set empirically using a clean validation set to minimize false positives

In operational spectrum monitoring, KL Divergence is computed over sliding windows of I/Q samples or FFT bins to generate a continuous anomaly score. A sudden spike in divergence often correlates with the appearance of a rogue emitter, jamming signal, or unauthorized transmission.

• Sliding window: Typical window sizes range from 1ms to 100ms depending on signal bandwidth
• Computational efficiency: Pre-computed bin edges and vectorized operations enable microsecond scoring
• Multi-band monitoring: Independent divergence scores are maintained for each sub-band of interest

Beyond raw power spectra, KL Divergence is applied to learned feature embeddings from autoencoders or convolutional networks. By monitoring the divergence between the training embedding distribution and live feature vectors, systems detect subtle anomalies that may not manifest as obvious power changes.

• Embedding space: Low-dimensional representations capture semantic signal properties
• Drift detection: A gradual increase in KL Divergence over time signals concept drift in the RF environment
• Retraining trigger: Sustained high divergence can automatically initiate model fine-tuning

KL Divergence scores from multiple signal representations—power spectral density, cyclostationary features, and constellation diagrams—are combined into a unified anomaly metric. This fusion approach reduces false alarms by requiring consensus across independent statistical views of the signal.

• PSD divergence: Detects broadband energy anomalies
• Cyclostationary divergence: Identifies modulation changes invisible to PSD
• Constellation divergence: Flags symbol-level distortions from hardware faults or spoofing
• Fusion rule: Weighted sum or maximum divergence across modalities

KL Divergence excels at detecting Low Probability of Intercept (LPI) signals designed to blend into noise. While such signals may have power levels below a simple energy detector's threshold, their statistical structure—captured by higher-order moments—diverges measurably from Gaussian noise.

• Noise baseline: A Gaussian distribution fitted to ambient noise floor
• LPI detection: Spread spectrum signals exhibit non-Gaussian kurtosis detectable via KL Divergence
• Adversarial robustness: Attackers cannot easily mimic the full statistical fingerprint of authorized transmitters

When combined with Radio Environment Maps (REMs), KL Divergence scores from distributed sensors are interpolated to create geospatial heatmaps of spectral abnormality. This enables operators to visually identify the physical location of anomalous emitters and track their movement over time.

• Sensor grid: Multiple monitoring nodes each compute local divergence scores
• Spatial interpolation: Kriging or inverse distance weighting generates continuous anomaly surfaces
• Temporal tracking: Sequential heatmaps reveal emitter trajectories and dwell patterns