Modulation Confidence Score

The most common form of confidence scoring, derived from the softmax activation layer in a neural network classifier. The softmax function exponentiates the raw logits and normalizes them into a probability distribution over all known modulation classes. The highest probability is taken as the confidence score for the predicted class. However, these probabilities are often poorly calibrated, tending toward overconfidence even on out-of-distribution or noisy inputs. Temperature scaling is a post-hoc calibration technique that smooths the softmax distribution to better align confidence with empirical accuracy.

A more statistically rigorous confidence metric derived from likelihood-based AMC classifiers. The LLR compares the likelihood of the received signal under the best-fit modulation hypothesis against the likelihood under the next-best alternative. A high LLR indicates strong evidence for the top hypothesis. Unlike raw softmax probabilities, LLRs are grounded in the probability density function of the signal model, making them more robust for downstream fusion in cooperative sensing networks where multiple radios share local confidence metrics.

A perfectly calibrated confidence score of 0.9 should mean the classifier is correct 90% of the time when it outputs that score. In practice, deep learning AMC models exhibit miscalibration—they are systematically overconfident. Expected Calibration Error (ECE) is the standard metric for measuring this gap. Techniques to improve calibration include:

• Temperature scaling: A single parameter tuned on a validation set to soften probabilities.
• Platt scaling: Fitting a logistic regression model on top of the classifier's output scores.
• Isotonic regression: A non-parametric method that learns a monotonic mapping from scores to calibrated probabilities.

Downstream cognitive radio systems use confidence scores to implement reject options and risk-aware spectrum access policies. A high threshold (e.g., >0.95) triggers automatic reconfiguration; a moderate score (0.7–0.95) may request additional sensing samples; a low score (<0.7) can force the radio to defer to a safe fallback mode or request human-in-the-loop analysis. This is critical in electronic warfare contexts where misclassifying a hostile signal as benign has severe consequences. The threshold is tuned based on the asymmetric costs of false positives versus false negatives.

The Shannon entropy of the softmax output distribution provides a scalar measure of predictive uncertainty that captures more information than the maximum probability alone. Low entropy indicates the probability mass is concentrated on a single class; high entropy indicates a spread across multiple plausible modulation schemes. This is particularly useful for open-set recognition, where an unknown modulation type will typically produce a high-entropy, flat distribution rather than a spiky, confident one. Entropy-based rejection rules can flag novel waveforms for further analysis.

Bayesian neural networks model epistemic uncertainty—uncertainty in the model's own parameters—by placing probability distributions over weights rather than learning point estimates. At inference time, multiple stochastic forward passes (Monte Carlo dropout) produce a distribution of predictions. The variance of this distribution serves as a confidence score that distinguishes between aleatoric uncertainty (inherent noise in the signal) and epistemic uncertainty (the model encountering unfamiliar data). This decomposition is invaluable for identifying out-of-distribution signals that lie far from the training manifold.

A post-processing calibration technique applied to the softmax layer of a neural network to align raw confidence scores with true empirical probabilities. By dividing logits by a scalar temperature parameter T > 1, the output distribution is softened, reducing overconfidence on misclassified examples. Modern AMC systems use temperature scaling to produce well-calibrated confidence scores that accurately reflect the likelihood of correct modulation recognition, enabling downstream systems to set reliable decision thresholds for autonomous frequency allocation.

A statistical metric representing the logarithm of the ratio of two competing hypothesis probabilities. In modulation classification, LLRs compare the likelihood that a received signal matches one modulation scheme versus another. Unlike raw softmax outputs, LLRs provide a scale-invariant confidence measure that is particularly valuable in likelihood-based AMC architectures. Key properties:

• Positive LLR favors the numerator hypothesis
• Magnitude indicates confidence strength
• Used in sequential probability ratio tests for rapid, risk-bounded classification decisions

A classification paradigm where the model must not only identify known modulation schemes but also detect and reject unknown types not seen during training. The modulation confidence score is the critical gating mechanism: if the maximum softmax probability falls below a calibrated threshold, the input is flagged as an unknown waveform. This prevents forced misclassification of novel or adversarial signals in electronic warfare scenarios. Modern approaches combine extreme value theory on logit distributions with confidence scores to model the boundary between known and unknown signal classes.

The task of identifying input signals that are fundamentally different from the training data distribution. While related to open-set recognition, OOD detection focuses on distributional shift rather than novel classes. Confidence scores serve as a primary detection signal—inputs far from the training manifold typically produce low maximum confidence and high predictive entropy. Techniques include:

• Energy-based models that assign scalar scores to signal typicality
• Mahalanobis distance in feature space
• Gradient-based confidence analysis for anomaly flagging

Neural network architectures that place probability distributions over weights rather than learning point estimates, enabling the model to express both aleatoric uncertainty (inherent noise) and epistemic uncertainty (model ignorance). For modulation confidence scoring, Bayesian approaches decompose the predictive uncertainty into:

• Aleatoric: Irreducible noise from low SNR conditions
• Epistemic: Model uncertainty from limited training data for rare modulation schemes This decomposition allows electronic warfare systems to distinguish between 'I'm uncertain because the signal is noisy' and 'I'm uncertain because I've never seen this waveform before.'

A physical-layer quality metric measuring the Euclidean distance between ideal constellation points and actual received symbols, expressed as a percentage or dB. EVM directly impacts modulation confidence scores—high EVM from amplifier non-linearity, phase noise, or channel distortion increases symbol dispersion, reducing classifier certainty. In cognitive radio pipelines, EVM estimates often serve as an auxiliary input to confidence calibration modules, allowing the system to anticipate degraded classification performance before committing to a modulation recognition decision.