Masked Autoencoder (MAE)

MAE employs an asymmetric architecture where the encoder only processes visible (unmasked) patches—typically 25% of the input—while a lightweight decoder reconstructs the full signal from encoded visible tokens and shared mask tokens. This design reduces compute by 3-4x compared to symmetric architectures, as the heavy encoder never sees the majority of the input. For RF applications, this means the encoder operates on a sparse subset of IQ samples, learning high-level spectral features without being burdened by dense, redundant time-domain data.

MAE uses an aggressively high masking ratio—typically 75% to 90% of input patches are removed before encoding. This is fundamentally different from denoising autoencoders or BERT-style masking (15%). The high ratio forces the model to learn holistic, semantic representations rather than memorizing local statistics:

• Eliminates spatial redundancy that trivial interpolation could solve
• Prevents the model from simply copying neighboring visible patches
• For RF spectrograms, this compels the encoder to understand global spectral occupancy patterns rather than local noise textures

The adaptation of MAE to radio frequency data, termed Masked IQ Modeling, treats raw in-phase and quadrature (IQ) samples or spectrogram frames as patches to be masked and reconstructed. Key adaptations include:

• Complex-valued patching: IQ samples are grouped into 2D patches preserving both temporal and quadrature structure
• Spectrogram tokenization: Time-frequency representations are divided into non-overlapping patches, with the model learning to reconstruct masked time-frequency bins
• This approach learns channel-agnostic features that generalize across different center frequencies, bandwidths, and receiver hardware without requiring labeled modulation or protocol data

The self-supervised pretext task is pixel-level reconstruction of the masked patches in the input space. The decoder outputs predicted pixel values only for masked positions, and the loss function is Mean Squared Error (MSE) computed exclusively on the masked patches—visible patches are ignored in the loss. This focused objective ensures:

• The encoder learns a compressed latent representation that captures essential signal structure
• The decoder learns to map from this latent code back to detailed signal morphology
• For RF, this means reconstructing raw IQ values or spectral magnitudes, forcing the model to internalize physics-level signal properties like carrier structure, pulse shapes, and modulation constellations

After pre-training, the decoder is discarded entirely—only the encoder is retained as a universal feature extractor for downstream tasks. The encoder produces rich, transferable representations that can be fine-tuned on small labeled datasets for:

• Automatic modulation classification with as few as 5-10 examples per class
• Specific emitter identification using hardware impairment fingerprints
• Spectrum anomaly detection by measuring reconstruction error on novel signals This decoupling of pre-training from downstream task heads is what makes MAE a powerful foundation model paradigm for RF machine learning.

Unlike contrastive methods such as SimCLR or MoCo, MAE does not require negative pairs, large batches, or momentum encoders to prevent representation collapse. The reconstruction objective itself acts as a natural regularizer:

• The decoder must produce diverse outputs for different masked inputs, preventing the encoder from collapsing to a constant representation
• No stop-gradient operations or exponential moving average mechanisms are needed
• This makes MAE significantly simpler to implement and more stable to train than BYOL or Barlow Twins, while achieving competitive or superior transfer performance on RF benchmarks

The foundational paradigm where a neural network learns general-purpose representations from unlabeled data by solving a pretext task. For MAE, the pretext task is reconstructing masked input patches. This pre-training phase teaches the encoder to understand the statistical structure of raw IQ samples without requiring expensive manual labeling. The learned weights are then transferred to a downstream task like modulation recognition via fine-tuning.

A key architectural innovation of MAE that enables efficient pre-training. The encoder operates only on visible, unmasked patches of the input spectrogram or IQ sequence, dramatically reducing compute. A lightweight decoder then reconstructs the full signal from the encoded visible patches and learned mask tokens. This asymmetry means the heavy encoder sees only a fraction of the data, making large-scale pre-training feasible on long RF recordings.

Unlike images where random patch masking works well, RF signals require adapted strategies. High masking ratios (e.g., 75-90%) are typical, forcing the model to learn long-range spectral dependencies. Masking can be applied to:

• Random time-frequency patches in spectrograms
• Contiguous blocks of IQ samples to simulate interference
• Structured masking that respects signal burst boundaries The goal is to remove enough information to make reconstruction non-trivial, preventing the model from simply interpolating adjacent samples.

A critical failure mode in self-supervised learning where the encoder produces a constant or non-informative output for all inputs, cheating the pretext task. MAE inherently resists collapse because the reconstruction objective on masked patches demands diverse, informative latent representations. Unlike contrastive methods that require negative pairs or momentum encoders, MAE's pixel-level reconstruction loss provides a natural, strong supervisory signal that prevents the encoder from degenerating.

The ability of a model trained on one RF environment to perform accurately on unseen target domains with different channel conditions, hardware impairments, or noise profiles. MAE pre-training on large, unlabeled spectrum captures forces the encoder to learn invariant spectral features rather than spurious correlations. This results in robust representations that transfer effectively across diverse receivers, frequency bands, and propagation environments without requiring domain-specific labeled data.

A downstream task where a model classifies signal modulation types using only a handful of labeled examples per class (e.g., 1-5 shots). MAE pre-training excels here because the encoder has already learned a rich, structured latent space of signal features from massive unlabeled data. When combined with prototypical networks or simple linear probing, the frozen MAE encoder can achieve high accuracy on novel modulations with minimal labeled support, critical for electronic warfare and spectrum monitoring.