Confusion Matrix

The most intuitive performance metric, defined as the ratio of correct predictions to total predictions: (TP + TN) / (TP + TN + FP + FN).

• Utility: Best used when class distributions are balanced and the cost of false positives equals false negatives.
• Pitfall: Highly misleading in imbalanced datasets. A classifier that predicts only the majority class can exhibit high accuracy while completely failing on rare modulation schemes.
• Context: In spectrum monitoring, where most signals are noise, a 99% accuracy rate is meaningless if the classifier never detects the target signal.

Quantifies the exactness of positive predictions: TP / (TP + FP). It answers: 'Of all signals the classifier labeled as a specific modulation, how many were actually that modulation?'

• High Precision: Indicates a low false alarm rate. When the system claims a QAM-64 signal is present, it is highly likely to be correct.
• Trade-off: Precision and recall are often in tension. A conservative classifier that only predicts when highly confident will have high precision but may miss many real signals.
• Application: Critical in electronic warfare support where false identification of a threat emitter triggers costly countermeasures.

Measures the completeness of positive predictions: TP / (TP + FN). It answers: 'Of all actual signals of a given modulation, how many did the classifier correctly find?'

• High Recall: Indicates the classifier misses very few instances of a target class. Essential for safety-critical or surveillance applications.
• FN Impact: A false negative in spectrum enforcement means a rogue transmitter operating on a restricted frequency goes undetected.
• Per-Class Metric: Recall is calculated individually for each modulation scheme, revealing which specific signal types the model struggles to detect.

The harmonic mean of precision and recall: 2 * (Precision * Recall) / (Precision + Recall). It provides a single balanced metric when you need both exactness and completeness.

• Harmonic Mean Rationale: Unlike the arithmetic mean, the harmonic mean heavily penalizes extreme disparities. A model with 100% precision and 0% recall yields an F1 score of 0, not 50%.
• Use Case: The standard metric for comparing modulation classifiers on imbalanced datasets where both false alarms and missed detections carry significant operational cost.
• Variants: Macro F1 averages per-class F1 scores equally, while Micro F1 aggregates contributions globally, giving weight to class frequency.

The recall analog for the negative class: TN / (TN + FP). It answers: 'Of all signals that are NOT a target modulation, how many were correctly rejected?'

• Complementary Metric: Often overlooked in multi-class problems but vital for binary detection tasks within a cognitive radio context.
• Spectrum Holes: High specificity ensures that a cognitive radio correctly identifies empty spectrum bands as truly vacant, maximizing opportunistic throughput without causing interference.
• ROC Analysis: Specificity is plotted against recall (1 - Specificity vs. Sensitivity) to generate the Receiver Operating Characteristic curve, tuning the classifier's decision threshold.

A statistical measure of inter-rater agreement that accounts for the possibility of chance agreement: (p_o - p_e) / (1 - p_e), where p_o is observed accuracy and p_e is hypothetical chance accuracy.

• Chance Correction: Unlike simple accuracy, Kappa penalizes the classifier for correct predictions that could have occurred randomly based on class distribution.
• Interpretation: A Kappa of 0 indicates performance equivalent to random guessing. Values above 0.8 suggest strong agreement beyond chance.
• Rigorous Benchmarking: Essential for academic comparison of modulation classifiers, as it normalizes for dataset imbalance and the number of classes, providing a truer measure of learned discriminative ability.

The confusion matrix is an N×N grid where N equals the number of modulation classes. Each cell C(i,j) records the count of signals with true modulation i that were predicted as modulation j. Diagonal elements represent correct classifications, while off-diagonal elements reveal specific misclassification patterns. For a 4-class problem (BPSK, QPSK, 16-QAM, 64-QAM), the matrix immediately exposes whether the classifier confuses 16-QAM with 64-QAM more often than with BPSK.

From the confusion matrix, critical per-modulation metrics are derived:

• True Positive (TP): Correctly identified instances of a specific modulation
• False Positive (FP): Other modulations incorrectly labeled as this class
• False Negative (FN): Instances of this modulation missed by the classifier
• True Negative (TN): All other modulations correctly not labeled as this class

These values compute precision (TP/(TP+FP)) and recall (TP/(TP+FN)) for each modulation scheme independently.

Off-diagonal heat in the confusion matrix reveals systematic confusion patterns between modulation pairs. Common failure modes include:

• QPSK vs. 16-QAM confusion at low SNR when constellation points overlap
• 16-QAM vs. 64-QAM misclassification due to similar circular symmetry
• BPSK vs. QPSK ambiguity under phase rotation

These patterns guide feature engineering and data augmentation strategies, such as adding targeted phase noise or emphasizing higher-order cumulants that discriminate between confused classes.

Confusion matrices can be presented in three normalization modes:

• Raw counts: Absolute number of predictions per cell, useful for understanding dataset imbalance effects
• Row-normalized (recall): Each row sums to 1.0, showing the probability distribution of predictions for a given true class
• Column-normalized (precision): Each column sums to 1.0, revealing what fraction of predictions for a class were correct

Row-normalized matrices are preferred for identifying which modulations are most frequently missed, while column-normalized matrices highlight which predictions are least trustworthy.

The confusion matrix supports both micro-averaging and macro-averaging for overall performance:

• Micro-averaging aggregates all cell contributions equally, favoring performance on majority classes
• Macro-averaging computes per-class metrics then averages, giving equal weight to rare modulation types

For imbalanced spectrum monitoring where some modulations appear rarely, macro-averaged F1-score from the confusion matrix provides a more honest assessment than raw accuracy.

When combined with classifier softmax confidence scores, the confusion matrix identifies overconfident misclassifications. A cell with high off-diagonal counts and high average confidence indicates the model is systematically wrong with high certainty—a dangerous failure mode in autonomous receivers. This analysis drives temperature scaling and label smoothing during training to produce better-calibrated probability estimates for each modulation prediction.

The diagonal elements of a confusion matrix representing correct classifications. A True Positive occurs when the model correctly predicts a specific modulation class (e.g., predicting QPSK when the true label is QPSK). A True Negative represents correctly rejecting a class for a sample that belongs to another. In multi-class signal classification, the sum of the diagonal divided by the total samples gives accuracy, but this can be misleading when classes are imbalanced.

The off-diagonal elements representing classification errors. A False Positive (Type I Error) is an incorrect prediction of a modulation scheme—for instance, classifying noise as 16-QAM. A False Negative (Type II Error) is a missed detection where the true modulation is not identified. In electronic warfare and spectrum monitoring, the cost asymmetry between these errors is critical:

• A high FP rate causes unnecessary alerts
• A high FN rate means missing hostile or unauthorized transmissions

Key per-class metrics derived directly from the confusion matrix. Precision (TP / (TP + FP)) measures how many of the predicted modulations were correct—essential when false alarms are costly. Recall (TP / (TP + FN)) measures how many actual signals of a class were found—critical for signal intelligence completeness. The F1-Score is the harmonic mean of precision and recall, providing a single balanced metric when you need both low false alarms and high detection rates for similar constellations like 16-QAM vs 64-QAM.

The confusion matrix reveals which modulation schemes are systematically misclassified, guiding feature engineering and architecture design. Typical high-confusion pairs include:

• 16-QAM ↔ 64-QAM: Higher-order QAM schemes with similar constellation geometry
• QPSK ↔ 4-PAM: Different modulation families that can appear similar under phase rotation
• 8-PSK ↔ 8-QAM: Schemes with identical symbol counts but different arrangements Analyzing these off-diagonal clusters helps prioritize contrastive learning strategies or additional cyclostationary features to separate ambiguous classes.

Raw confusion matrices can be normalized to reveal patterns obscured by class imbalance. Row-wise normalization divides each cell by the total true samples for that class, showing the true positive rate (recall) on the diagonal and misclassification rates off-diagonal. Column-wise normalization shows precision by dividing by total predictions per class. For RF datasets where some modulations appear rarely (e.g., 256-QAM vs BPSK in real traffic), normalized views prevent common classes from visually dominating the error analysis.

When aggregating confusion matrix metrics across all modulation classes, the averaging strategy matters. Macro-averaging computes the metric independently for each class and then takes the unweighted mean—treating rare modulations like 32-APSK equally with common ones like QPSK. Micro-averaging aggregates the contributions of all classes to compute the average, which weights each sample equally and is dominated by frequent classes. For SIGINT applications where detecting rare signals is paramount, macro-averaged F1 is the preferred benchmark.