Conformal Prediction

The core guarantee of conformal prediction holds without any assumptions about the data distribution. Unlike Bayesian methods that require a correctly specified prior, or frequentist methods that assume Gaussian errors, conformal prediction provides exact coverage guarantees for any data distribution and any underlying model. This makes it uniquely suited for RF applications where noise characteristics are often non-Gaussian or unknown.

Conformal prediction provides a marginal coverage guarantee that holds for any finite sample size n. For a user-specified error rate α (e.g., 0.1), the prediction set will contain the true label with probability at least 1-α. This is not an asymptotic result—it is a finite-sample theorem. For mission-critical RF systems, this means you can certify performance without waiting for infinite data.

Conformal prediction operates as a wrapper around any pre-trained model—neural network, random forest, or heuristic algorithm. It requires no modification to the model architecture or training procedure. The only requirement is a held-out calibration set of exchangeable data points. This plug-and-play nature allows RF engineers to add rigorous uncertainty quantification to existing signal classifiers without retraining.

The validity guarantee relies on the assumption of exchangeability—that the joint distribution of calibration and test points is invariant under permutation. This is weaker than the independent and identically distributed (i.i.d.) assumption but stronger than no assumption at all. In RF contexts, exchangeability can be violated by concept drift or temporal dependencies, requiring careful calibration set construction.

The mechanism of conformal prediction centers on a nonconformity measure—a function that quantifies how unusual a candidate label is given the input. Common choices include:

• 1 minus softmax probability for classifiers
• Absolute residual for regressors
• Mahalanobis distance for multivariate outputs The choice of nonconformity score determines the efficiency (average set size) of the resulting prediction sets.

The discipline of characterizing all sources of uncertainty in a model's predictions. While conformal prediction produces prediction sets with finite-sample coverage guarantees, broader UQ decomposes uncertainty into:

• Epistemic uncertainty: Reducible ignorance from limited data or model capacity
• Aleatoric uncertainty: Irreducible noise inherent in the RF channel itself For spectrum sensing, distinguishing these types determines whether collecting more IQ samples will improve performance or if the environment is fundamentally ambiguous.

The uncertainty arising from a lack of knowledge that can theoretically be reduced. In RF machine learning, this manifests when:

• A classifier encounters a modulation scheme absent from its training set
• A signal-to-noise ratio regime not covered during data collection appears Conformal prediction's marginal coverage guarantee holds regardless of epistemic gaps, but the resulting prediction sets will be larger in regions where the model lacks training support, providing an operational signal that more data is needed.

The irreducible statistical noise inherent in the data generation process itself. In wireless systems, this includes:

• Thermal noise at the receiver front-end
• Fading and multipath effects that randomize received IQ samples
• Interference from uncoordinated transmitters Conformal prediction adapts to regions of high aleatoric uncertainty by producing wider prediction intervals or larger prediction sets, giving operators a calibrated measure of when the RF environment is fundamentally unpredictable.

The property of a model to maintain correct predictions under intentionally perturbed inputs. In electronic warfare contexts, an adversary may craft over-the-air adversarial waveforms designed to fool an automatic modulation classifier. Conformal prediction complements adversarial training by:

• Flagging inputs that fall outside the model's confident region via empty or oversized prediction sets
• Providing a statistically rigorous detection mechanism for distributional shift
• Enabling the system to fall back to safe defaults when the guarantee cannot be met with a useful set size

The process of aligning a human operator's subjective confidence in an automated system with its objective competence. Conformal prediction directly enables trust calibration by:

• Replacing opaque softmax probabilities with prediction sets that have a proven error rate
• Allowing mission assurance leads to set the desired error tolerance (α) explicitly
• Providing a decision framework where an empty set signals 'no reliable classification possible' This transforms the operator's relationship with the AI from blind faith to statistically informed reliance.

A causal explanation method that identifies the minimal change to an input required to alter a prediction. When combined with conformal prediction, counterfactuals answer: 'What minimal change to this IQ sequence would move it outside the current prediction set?' This is critical for:

• Understanding the decision boundary of an RF classifier in signal space
• Identifying which signal features the conformal predictor is sensitive to
• Debugging why certain modulation formats produce consistently large prediction sets