Pseudo-Labeling

The foundational iterative process where a model bootstraps its own performance.

• Initialization: A base model is trained exclusively on the small, high-quality labeled dataset.
• Inference: The trained model predicts labels for the much larger unlabeled dataset.
• Selection: Predictions exceeding a strict confidence threshold (e.g., >0.95 probability) are converted into artificial targets.
• Retraining: The model is retrained on the original labeled data combined with the newly pseudo-labeled examples, and the cycle repeats.

The critical gating mechanism that determines which artificial labels are accepted. Only predictions with a softmax probability above a predefined threshold are retained.

• High Threshold (>0.9): Minimizes confirmation bias by ensuring only highly certain predictions are used, but may yield very few pseudo-labels.
• Curriculum Thresholding: The threshold dynamically increases during training, starting low to admit more examples and rising to enforce stricter quality as the model improves.
• Class-Balanced Selection: Prevents the model from only generating pseudo-labels for dominant classes by applying per-class thresholds or capping the number of accepted samples per class.

The primary failure mode of pseudo-labeling, where the model reinforces its own mistakes. If an incorrect prediction is accepted as a pseudo-label, the model will learn to be confidently wrong on subsequent iterations.

• Error Amplification: A single mislabeled example can skew the decision boundary, causing more errors in the next round.
• Mitigation via Ensembles: Using a committee of models to generate pseudo-labels reduces individual bias.
• Mitigation via Noise: Adding Gaussian noise or strong data augmentation to pseudo-labeled inputs prevents the model from memorizing spurious correlations.

A temporal strategy for balancing the contribution of real and pseudo-labeled data during training. The unsupervised loss component is typically scaled by a coefficient that evolves over time.

• Ramp-Up Function: The weight starts at zero and increases linearly or exponentially during early epochs, allowing the model to first learn robust features from real labels before trusting its own predictions.
• Deterministic Annealing: A sigmoid-based schedule that smoothly transitions the model's focus from supervised to semi-supervised objectives.
• Per-Sample Weighting: Instead of a global coefficient, each pseudo-labeled sample is weighted by its specific confidence score, giving higher influence to more certain predictions.

An implicit objective of pseudo-labeling that pushes the model toward low-entropy, high-confidence predictions on unlabeled data. This acts as a regularizer that forces the decision boundary to pass through low-density regions of the feature space.

• Cluster Assumption: The technique relies on the assumption that data points belonging to the same class cluster together, and the true decision boundary lies far from dense data regions.
• Sharpening: Converting soft probability distributions into hard one-hot pseudo-labels is an extreme form of entropy minimization that can be relaxed using a temperature parameter.

Application of the technique to raw IQ samples for modulation and emitter classification.

• Few-Shot Modulation Recognition: A model trained on 5-10 labeled IQ captures per modulation scheme can pseudo-label thousands of unlabeled captures collected from the same spectrum band.
• Consistency Under Channel Impairments: Strong RF-specific augmentations—such as adding Additive White Gaussian Noise (AWGN), applying random phase rotations, or simulating Rician fading—are applied to unlabeled IQ samples. The model must produce consistent pseudo-labels despite these perturbations.
• Open-Set Considerations: A confidence threshold alone is insufficient when unknown emitter types are present. Pseudo-labeling must be combined with out-of-distribution detection to avoid labeling novel signal sources as known classes.

A critical failure mode in self-supervised learning where the encoder produces a constant or non-informative vector for all inputs, achieving zero loss trivially. Prevented by explicit regularization techniques:

• Variance Regularization: Penalizes low standard deviation in batch embeddings
• Covariance Regularization: Decorrelates feature dimensions to prevent informational redundancy
• Contrastive Loss: Implicitly avoids collapse by repelling negative samples

A data augmentation strategy that creates virtual training samples by linearly interpolating raw IQ sequences and their corresponding labels. For two samples (x_i, y_i) and (x_j, y_j), a new sample is formed as λx_i + (1-λ)x_j with label λy_i + (1-λ)y_j. This promotes linear behavior between training examples, reduces adversarial vulnerability, and improves generalization in few-shot modulation recognition tasks.