Contrastive Predictive Coding (CPC)

CPC employs a causal autoregressive model, typically a Gated Recurrent Unit (GRU) or causal Transformer, to compress a history of latent signal embeddings into a single dense context vector. This vector summarizes all relevant past information without looking at future samples, preserving the temporal causality of the RF stream.

The model is trained using InfoNCE (Noise Contrastive Estimation) loss, which maximizes the lower bound on mutual information between the context vector and future latent representations. In practice, this forces the model to identify the true future signal from a batch of negative distractors:

• Positive sample: The actual future latent embedding.
• Negative samples: Randomly sampled embeddings from other time steps. This contrastive task teaches the model to capture slow, high-level features that are predictive over long horizons.

Unlike patch-based vision CPC, RF applications often use a dense predictive coding variant where the model predicts latent representations for every future time step, not just a single offset. This is crucial for capturing the continuous temporal dynamics of IQ sample streams and complex modulations. The architecture learns features that are invariant to nuisance parameters like phase rotation and frequency offset.

After pre-training on massive unlabeled raw IQ data, the learned encoder is frozen or fine-tuned for specific tasks. The representations serve as a universal feature extractor for:

• Automatic Modulation Classification (AMC): Linear probing on frozen CPC features often matches fully supervised baselines with a fraction of the labeled data.
• Specific Emitter Identification (SEI): The model captures subtle hardware impairments that are predictive over long sequences.
• Anomaly Detection: The context vector provides a compact baseline for identifying deviations from normal spectrum activity.

The negative sampling strategy in CPC acts as an implicit regularizer against noise. By forcing the model to distinguish the true future signal from a large set of negative examples—many of which are noise-dominated—the learned representations become inherently robust to low Signal-to-Noise Ratio (SNR) conditions. This is a key advantage over reconstruction-based self-supervised methods like autoencoders, which often waste capacity modeling stochastic noise.

Standard CPC operates on real-valued latent vectors, but RF-native implementations extend the framework to complex-valued neural networks (CVNNs). By preserving the phase information in the latent space and using complex-valued InfoNCE loss, the model learns richer representations of quadrature-modulated signals. This avoids the information loss associated with converting IQ data to separate real and imaginary channels or magnitude/phase representations.

The core contrastive objective function used in CPC. It trains the model to maximize mutual information between context vectors and future latent representations.

• Uses a noise-contrastive estimation approach
• Scores positive samples (true future) against negative samples (distractor signals)
• The model learns to distinguish temporally coherent signal features from random noise
• Critical for learning representations that capture the slow, meaningful variations in RF waveforms while ignoring high-frequency noise

The sequential model component in CPC that summarizes past latent representations into a single compact context vector for prediction.

• Typically implemented as a Gated Recurrent Unit (GRU) or causal transformer
• Processes the sequence of encoded IQ samples to capture temporal dynamics
• The context vector must retain information about modulation patterns, symbol timing, and channel effects
• For RF applications, this model learns the underlying temporal structure of transmissions without explicit labels

The theoretical principle behind CPC: maximizing the mutual information between context representations and future observations.

• CPC provides a lower bound on mutual information via the InfoNCE loss
• Higher mutual information indicates the representation captures signal-specific structure rather than noise
• In RF, this translates to learning features that distinguish modulation schemes, symbol rates, and transmitter fingerprints
• Enables downstream tasks like AMC and SEI with minimal labeled data

The method for selecting distractor samples that the model must discriminate against during CPC training.

• Same-sequence sampling: negative samples drawn from other time steps in the same recording
• Cross-sequence sampling: negatives drawn from entirely different signal captures
• Hard negative mining: selecting challenging distractors with similar characteristics to the positive
• In RF, cross-sequence sampling prevents the model from learning trivial shortcuts based on channel conditions or receiver artifacts

The process of adapting a CPC-pretrained encoder for specific RF tasks using limited labeled data.

• The pretrained encoder serves as a frozen feature extractor or is fine-tuned end-to-end
• Common downstream tasks: Automatic Modulation Classification (AMC), Specific Emitter Identification (SEI), and anomaly detection
• Dramatically reduces the need for expensive labeled RF datasets
• Representations learned via CPC often generalize better across different SNR conditions and channel impairments than supervised baselines

A broader family of self-supervised methods that CPC belongs to, all leveraging time as a supervisory signal.

• Includes variants like Temporal Neighborhood Coding (TNC) and Time-Contrastive Networks (TCN)
• Assumes that nearby time steps share semantic content while distant steps are statistically independent
• For RF streams, this exploits the fact that modulation parameters and transmitter characteristics are slowly varying
• Contrasts with spatial augmentations used in vision; RF requires domain-specific temporal augmentations like phase rotation and frequency shift