Confidence Calibration

The defining property of calibration: the predicted probability matches the observed frequency of correctness. When a calibrated model assigns 0.8 confidence to 100 signal classifications, exactly 80 should be correct. This is measured using Expected Calibration Error (ECE), which bins predictions and computes the weighted average gap between accuracy and confidence. A perfectly calibrated model achieves an ECE of zero, meaning its confidence scores are not merely ordinal rankings but true statistical probabilities.

In open set recognition, the primary value of calibration is that low confidence reliably indicates an unknown input. A well-calibrated model does not produce overconfident wrong answers for novel modulation types. Instead, it assigns uniformly low, high-entropy probabilities across all known classes. This property enables a simple thresholding strategy:

• Confidence < 0.6: Flag as unknown, route for human analysis
• Confidence ≥ 0.6: Accept classification as known modulation Without calibration, a model might assign 95% confidence to an incorrect guess on a never-before-seen signal.

Calibration and accuracy are orthogonal properties. A model can be highly accurate yet poorly calibrated, or less accurate yet perfectly calibrated. Temperature scaling exploits this separation: it adjusts confidence sharpness without altering the model's ranking of predictions or its top-1 accuracy. This means calibration can be applied as a post-processing step after training, using a held-out validation set to learn the optimal temperature parameter that minimizes the negative log-likelihood of the predicted probabilities.

A calibrated model distinguishes between two types of uncertainty. Aleatoric uncertainty arises from inherent data noise, such as low SNR conditions, and cannot be reduced with more data. Epistemic uncertainty stems from a lack of knowledge about a signal type and is reducible. In a well-calibrated model, epistemic uncertainty manifests as low confidence on inputs far from the training distribution, directly enabling out-of-distribution detection. This property is explicitly trained for using objectives like the entropic open-set loss.

A robustly calibrated model maintains its probability-to-accuracy alignment across varying channel conditions. The confidence for a QPSK signal at 20 dB SNR should be higher than at 5 dB SNR, but both should be calibrated—the drop in confidence must proportionally reflect the drop in classification accuracy. This requires calibration validation across the full operational SNR range, not just at a single reference point. Models that fail this property exhibit overconfidence under distributional shift, silently failing as noise increases.

A well-calibrated model enables optimal decision-making at any operating point without retraining. Because confidence scores are true probabilities, a spectrum monitoring system can dynamically adjust its rejection threshold based on mission requirements:

• High precision mode: Reject below 0.9 confidence, minimizing false alarms
• High recall mode: Accept above 0.5 confidence, capturing more signals The calibration guarantees that the expected cost of any threshold choice is predictable, enabling risk-aware autonomous spectrum management.

The primary empirical metric for measuring miscalibration. It partitions predictions into M equally-spaced bins based on confidence and computes a weighted average of the absolute difference between accuracy and confidence in each bin.

• Perfect Calibration: An ECE of 0 indicates accuracy exactly matches confidence.
• Visualization: Often paired with a Reliability Diagram plotting accuracy vs. confidence.
• Limitation: ECE is sensitive to the chosen binning scheme and can be gamed by uniform predictions.

The model's uncertainty arising from a lack of knowledge or insufficient training data. This is the reducible component of total predictive uncertainty.

• Source: High epistemic uncertainty is triggered by inputs in regions far from the training distribution, such as novel modulation schemes.
• Utility: It is the primary signal for Out-of-Distribution (OOD) Detection and open-set rejection.
• Estimation: Quantified by the variance of predictions from a Deep Ensemble or by using dropout as a Bayesian approximation at inference time.

A framework that treats classification predictions as subjective opinions by placing a Dirichlet distribution over class probabilities, bypassing the need for a separate calibration step.

• Mechanism: The network outputs evidence for each class, and the concentration parameters of the Dirichlet distribution are derived from this evidence.
• Uncertainty Quantification: The total evidence mass inversely relates to predictive uncertainty. A lack of evidence for any known class signals a novel input.
• Loss Function: Trained by minimizing the Bayes risk with a regularizer that shrinks evidence for incorrect classes.

A robust method for uncertainty quantification that trains M independent neural networks with different random initializations and data shuffles.

• Predictive Distribution: The final prediction is a uniformly-weighted mixture of the individual models' SoftMax outputs.
• Uncertainty Decomposition: The variance of the ensemble's predictions captures epistemic uncertainty, while the mean entropy captures aleatoric uncertainty.
• Calibration: Ensembles are typically better calibrated than single models and serve as a strong baseline for OOD detection without requiring a separate calibration dataset.

A training objective specifically designed to calibrate the confidence of unknown inputs without requiring explicit outlier data during training.

• Mechanism: It forces the network to produce a high-entropy, uniform probability distribution for samples that are far from known class prototypes in the embedding space.
• Contrast: Known samples are trained to have low-entropy, peaked predictions.
• Rejection: A simple entropy threshold on the output probability vector can then reliably separate known and unknown modulation types at inference time.