Contrastive Learning

The fundamental mechanism that defines what the model should treat as 'similar.' In modulation recognition, positive pairs are created by applying stochastic data augmentations to the same IQ sample or constellation image.

• Augmentation strategies: Additive noise, phase rotation, frequency offset, time cropping
• Key principle: Augmentations must preserve modulation identity while varying nuisance parameters
• Example: A 16-QAM signal with simulated AWGN at 10dB SNR and the same signal with a 5-degree phase shift form a positive pair

The mechanism for defining dissimilarity that prevents representational collapse. Negative examples are typically drawn from other modulation classes or different signal instances within a training batch.

• In-batch negatives: All other samples in a mini-batch serve as negatives, enabling efficient computation
• Hard negative mining: Deliberately selecting confusing modulation pairs (e.g., 16-QAM vs. 64-QAM) to improve discriminative power
• Memory bank approaches: Maintaining a queue of recent representations to increase the effective number of negatives without expanding batch size

The dominant contrastive objective, derived from Noise Contrastive Estimation, that maximizes mutual information between differently augmented views of the same signal.

• Formulation: A categorical cross-entropy loss that identifies the positive pair among a set of negative distractors
• Temperature parameter: Controls the concentration of the distribution; lower temperatures sharpen the penalty on hard negatives
• Signal processing benefit: Naturally handles the continuous nature of channel impairments by learning a smooth embedding manifold where SNR variations form continuous trajectories

A small multi-layer perceptron appended to the encoder during training that maps representations to the space where contrastive loss is applied, then discarded after pre-training.

• Purpose: Prevents the encoder from losing information useful for downstream tasks by isolating the contrastive objective
• Typical design: 2-3 fully connected layers with ReLU activation, outputting 128-256 dimensional embeddings
• Modulation recognition insight: The projection head absorbs invariance to augmentations, leaving the encoder's penultimate layer with rich features transferable to classification heads

A foundational contrastive architecture that demonstrated simple yet effective self-supervised learning without specialized architectures or memory banks.

• Core components: Composition of random augmentations, a base encoder (ResNet), a projection head, and the InfoNCE loss
• Critical finding: The composition of augmentations and the use of a non-linear projection head dramatically improve representation quality
• RF application: Directly applicable to constellation diagram classification by treating each augmented view of a modulation scheme as a positive pair, learning features robust to channel distortion

A contrastive framework that decouples the dictionary size from batch size using a momentum-updated encoder and a dynamic queue of representations.

• Key innovation: A slowly evolving key encoder (updated via exponential moving average) maintains consistency in the negative sample dictionary
• Queue mechanism: Stores the most recent K representations as negatives, enabling large and consistent dictionaries without massive batches
• Modulation recognition advantage: Large negative dictionaries are critical for distinguishing between the many fine-grained modulation classes (BPSK, QPSK, 8-PSK, 16-QAM, 64-QAM, etc.) encountered in spectrum monitoring

The overarching training paradigm where a model generates its own supervisory signal from unlabeled data, bypassing the need for manual annotation. Contrastive learning is a dominant approach within this framework.

• Pretext Task: A fabricated challenge, such as instance discrimination, solved to learn data structure.
• Unlabeled Utility: Exploits vast volumes of raw IQ recordings without requiring expert modulation labels.
• Downstream Transfer: Learned representations are fine-tuned for specific tasks like modulation classification.

A foundational metric learning loss function that operates on triplets of data points: an anchor, a positive sample (same class), and a negative sample (different class). The objective is to minimize the distance between the anchor and positive while maximizing the distance to the negative by a defined margin.

• Hard Negative Mining: The practice of selecting negative samples that are deceptively close to the anchor, which is critical for learning discriminative signal features.
• Margin Parameter: A hyperparameter enforcing a minimum separation between dissimilar signal embeddings.

Info Noise-Contrastive Estimation, a popular loss function that treats the contrastive task as a categorical classification problem. It uses a softmax over similarity scores to identify the positive pair among a set of negative distractors.

• Temperature Parameter: A critical hyperparameter controlling the concentration of the distribution, dynamically adjusting the penalty on hard negative signal examples.
• Mutual Information: The loss provides a lower bound on the mutual information between different views of the same underlying signal.

A neural architecture employing two or more identical sub-networks that share the same weights and parameters. These parallel branches process different augmented views of a signal to produce embeddings that are then compared by a contrastive loss function.

• Weight Sharing: Guarantees that identical IQ samples processed through different branches produce identical output vectors.
• Symmetry: The architectural symmetry ensures that the distance metric between two signal representations is consistent and valid.

The systematic application of label-preserving transformations to raw IQ data to create the multiple views required by contrastive frameworks. Effective augmentations define the invariances the model learns.

• Channel Impairments: Applying simulated Additive White Gaussian Noise (AWGN), phase rotation, and frequency offset creates realistic positive pairs.
• Time-Frequency Masking: Randomly masking time steps or frequency bins forces the model to learn robust global signal structure rather than local noise.

A high-dimensional vector space where semantically similar signals are mapped to proximal points and dissimilar signals to distant points. The geometric structure of this space is the direct output of contrastive training.

• Cosine Similarity: A common metric for measuring angular distance between normalized signal embedding vectors, ignoring magnitude differences.
• Uniformity and Alignment: Two key properties of a well-formed space: embeddings are uniformly distributed on the hypersphere, and positive pairs are tightly aligned.