SimCLR

SimCLR relies on a composition of random augmentations applied to each input sample to generate two correlated views (positive pair). For RF applications, this translates to domain-specific augmentations:

• Random phase rotation and frequency offset
• Additive white Gaussian noise injection
• Time cropping and amplitude scaling
• Random resampling with small timing jitter The choice of augmentation pipeline is critical—it defines the invariances the encoder learns. For IQ data, augmentations should preserve modulation identity while varying nuisance parameters like channel effects.

A small multi-layer perceptron (MLP) attached to the encoder backbone during pre-training. The projection head maps representations to a lower-dimensional space where the contrastive loss is applied. Key properties:

• Typically 2-3 layers with ReLU activation and a linear output
• The output dimension is often 128
• Discarded after pre-training—only the backbone encoder is kept for downstream tasks This design prevents the contrastive loss from discarding information useful for downstream tasks by isolating the loss from the representation space.

The Normalized Temperature-scaled Cross Entropy loss operates on a batch of N samples, each with two augmented views (2N total). For each positive pair, the remaining 2(N-1) views serve as negatives. The loss:

• Uses cosine similarity between L2-normalized embeddings
• Applies a temperature parameter τ (typically 0.1) to control concentration
• Computed symmetrically for both views in each pair This formulation treats every other sample in the batch as a negative, creating an implicit contrastive task that scales with batch size.

SimCLR performance improves significantly with larger batch sizes because more negative examples are available per positive pair. The original implementation used:

• Batch sizes from 256 to 8192
• Training on 128 TPU v3 cores for large-scale experiments
• LARS optimizer to stabilize training at extreme batch sizes For RF datasets with limited samples, this requirement can be challenging. Alternatives include using a memory bank (as in MoCo) or applying gradient accumulation across smaller batches to approximate larger effective batch sizes.

SimCLR is backbone-agnostic—it works with any encoder architecture that produces a fixed-dimensional representation vector. For RF applications, common backbones include:

• ResNet-50 adapted for 1D complex-valued IQ inputs
• CLDNN (Convolutional, LSTM, Deep Neural Network) hybrids
• Vision Transformer (ViT) variants with IQ spectrogram patches
• Complex-valued CNNs that preserve phase information The framework's simplicity allows rapid experimentation with different encoder architectures without modifying the contrastive learning logic.

After self-supervised pre-training on unlabeled RF data, the encoder is fine-tuned on a small labeled dataset for specific tasks:

• Few-shot modulation classification with as few as 5 examples per class
• Specific emitter identification using hardware fingerprint features
• Spectrum anomaly detection via representation quality assessment SimCLR representations consistently outperform supervised baselines when labeled data is scarce, making it ideal for RF domains where annotation requires expert signal analysts and expensive equipment.

The contrastive objective function at the heart of SimCLR. It maximizes the mutual information between an anchor sample and its positive pair while pushing away a set of negative samples. For a minibatch of N examples, each augmented view creates a positive pair, while the other 2(N-1) augmented views serve as negatives. The loss is computed as a categorical cross-entropy over the similarity scores, effectively treating the positive pair identification as a classification problem. In RF applications, this loss learns representations invariant to channel impairments and noise.

A small multi-layer perceptron (MLP) attached to the backbone encoder during pre-training. SimCLR demonstrated that applying the contrastive loss on the output of this non-linear projection head, rather than directly on the encoder's representation, significantly improves the quality of learned features. The projection head maps representations to a space where contrastive learning is more effective, but it is discarded entirely after pre-training. Downstream tasks use the frozen encoder's output, which retains more general-purpose signal features. Typical architecture: a 2- or 3-layer MLP with ReLU activations.

SimCLR's performance depends critically on the composition of stochastic augmentations that define the invariant features. For IQ signal data, effective augmentations include:

• Additive white Gaussian noise (AWGN) injection
• Random phase rotation to simulate carrier offset
• Time shifting and cropping of IQ sequences
• Amplitude scaling to mimic fading
• Frequency offset simulation via complex exponential multiplication These augmentations teach the encoder that the emitter identity or modulation scheme remains constant despite channel variations, forcing it to learn the underlying signal structure.

A critical failure mode in self-supervised learning where the encoder produces a constant or non-informative vector for all inputs, achieving zero loss trivially. SimCLR avoids collapse through its contrastive formulation: the use of negative samples in the denominator of the InfoNCE loss creates a repulsive force that prevents all representations from converging to a single point. The large batch sizes (typically 4096 or larger) provide sufficient negative diversity. Alternative non-contrastive methods like BYOL and Barlow Twins use variance and covariance regularization instead of negative pairs to prevent collapse.

A scalar hyperparameter (τ) in the softmax formulation of the contrastive loss that controls the concentration of the similarity distribution. Lower temperatures (e.g., τ=0.1) create a sharper distribution, making the model focus intensely on hard negative samples—those most similar to the positive. Higher temperatures (e.g., τ=0.5) produce a softer distribution, treating all negatives more uniformly. In RF domains with subtle inter-class differences, careful temperature tuning is essential to balance discriminative feature learning against overfitting to noise artifacts in the negative set.

SimCLR's architecture fundamentally relies on aggressive batch sizes (4096 to 8192 samples) to provide a sufficiently large and diverse set of negative examples within each training step. This contrasts with MoCo, which decouples batch size from negative sample count via a momentum-updated queue. Large batch training requires distributed infrastructure with gradient accumulation across multiple GPUs or TPUs. For RF datasets with limited samples per class, this can be challenging; domain-specific solutions include gradient checkpointing and mixed-precision training to fit large batches in memory.