InfoNCE Loss

InfoNCE loss directly maximizes a lower bound on mutual information between the learned representations of related signal views. By contrasting positive pairs (e.g., different augmentations of the same IQ sample) against negative pairs (different signals), the encoder learns to extract the underlying semantic content while discarding noise. This property is critical for RF applications where the goal is to learn modulation-invariant or emitter-specific features from raw, unlabeled spectrum captures.

The temperature hyperparameter (τ) controls the concentration of the similarity distribution. Low temperatures create a sharper distribution, forcing the model to focus on hard negative mining—negative samples most similar to the anchor. High temperatures smooth the distribution, treating all negatives more uniformly. In RFML, tuning τ is essential for handling the fine-grained differences between modulation schemes or hardware impairments where hard negatives dominate the learning signal.

InfoNCE relies on a large and diverse set of negative samples to prevent representation collapse. Key strategies include:

• In-batch negatives: Using other samples in the mini-batch as negatives, enabling efficient computation.
• Memory bank (MoCo): Maintaining a dynamic queue of past representations to decouple batch size from negative set size.
• Hard negative mining: Actively selecting negatives that are difficult to distinguish, which is vital for separating spectrally similar emitters in RF fingerprinting tasks.

The InfoNCE objective can be decomposed into two emergent properties:

• Alignment: Positive pairs from the same underlying signal are mapped close together in the embedding space, achieving augmentation invariance.
• Uniformity: The encoder distributes representations uniformly across the hypersphere, preserving maximal information and preventing dimensional collapse. This dual behavior is why InfoNCE-pre-trained encoders transfer effectively to downstream tasks like few-shot modulation recognition without requiring labeled data during pre-training.

In frameworks like SimCLR and MoCo, the InfoNCE loss is applied asymmetrically: gradients flow only through the anchor (query) branch, while the positive and negative keys are treated as constant targets via a stop-gradient operation. This architectural choice prevents the model from finding a trivial collapsed solution where all representations map to the same vector. In RF self-supervised learning, this asymmetry is essential for learning meaningful signal features from unlabeled IQ streams.

InfoNCE performance scales with the number of negative samples. In SimCLR, this requires very large batch sizes (e.g., 4096+) to provide sufficient negatives within each batch. For RF applications with high-dimensional IQ inputs, this creates significant GPU memory pressure. Alternatives like MoCo's momentum queue or BYOL's negative-free approach were developed specifically to decouple contrastive learning from batch size constraints, enabling effective pre-training on resource-limited signal processing hardware.

A self-supervised framework that directly applies InfoNCE loss to sequential data. CPC learns representations by training an autoregressive model to predict future latent representations from past context, using the InfoNCE objective to distinguish the true future from randomly sampled negatives. In RF applications, CPC captures the temporal structure of IQ streams, enabling models to learn modulation patterns and channel dynamics without labels. The mutual information maximization property of InfoNCE is central to CPC's theoretical foundation.

The performance of InfoNCE loss critically depends on the quality and quantity of negative samples. Key strategies include:

• In-batch negatives: Using other samples in the mini-batch as negatives, efficient but prone to false negatives
• Memory bank: Storing representations from previous batches to increase negative pool size
• Momentum queue: A dynamic dictionary that decouples batch size from negative sample count In RF domains, hard negative mining—selecting negatives with similar modulation or channel characteristics—significantly improves the discriminative power of learned representations.

The temperature hyperparameter (τ) in InfoNCE loss controls the concentration of the similarity distribution. Lower temperatures sharpen the distribution, penalizing hard negatives more heavily and producing more discriminative embeddings. Higher temperatures soften the distribution, treating all negatives more uniformly. In RF self-supervised learning, temperature tuning is essential because signal similarity exists on a continuum—two QPSK samples may be legitimate positives or hard negatives depending on the downstream task. Typical values range from 0.07 to 0.5.

InfoNCE loss provides a variational lower bound on mutual information between the anchor and positive sample. Minimizing the loss maximizes I(x; c), the mutual information between input x and context c. This theoretical property justifies why representations learned via InfoNCE capture the underlying structure of RF signals—the encoder must preserve information that distinguishes true signal pairs from negatives. The bound tightens as the number of negative samples approaches infinity, motivating large negative pools in practice.

A small MLP projection head is typically attached to the encoder backbone when using InfoNCE loss. The loss is computed in the projection space, but the projection head is discarded after pre-training, and downstream tasks use the frozen encoder representations. This design prevents the loss from forcing the encoder to discard information useful for downstream tasks. In RF applications, a 2–3 layer projection head with 128–256 dimensions maps IQ embeddings to the contrastive space, preserving signal features in the backbone for modulation recognition and emitter identification.

InfoNCE loss requires defining positive pairs—two views of the same underlying signal. These are created through stochastic augmentations applied to the same IQ sample:

• Additive noise: Simulating varying SNR conditions
• Frequency shift: Emulating oscillator drift
• Time cropping: Extracting different temporal windows
• Phase rotation: Modeling channel rotation The encoder learns invariance to these augmentations while discriminating between different signals, producing representations robust to real-world RF channel impairments.