Transfer Learning

The process begins by training a deep neural network on a large, labeled source dataset containing abundant signal types. The model learns a rich, hierarchical feature representation of the radio frequency environment. For modulation recognition, this often involves training on high-SNR synthetic signals. The quality and diversity of this source data directly dictate the transferability of learned features to the target domain.

Two dominant strategies exist for adapting a pre-trained model:

• Feature Extraction: The pre-trained convolutional base is frozen and treated as a fixed feature extractor. Only a new classifier head is trained on the target data. This is computationally cheap and prevents overfitting on extremely small datasets.
• Fine-Tuning: A subset of the pre-trained network's layers are unfrozen and jointly trained with the new classifier on the target data. This allows the model to adapt its low-level feature detectors to the specific channel conditions of the new domain, often yielding higher accuracy if sufficient target data exists.

The effectiveness of transfer learning is bounded by the similarity between the source and target domains. If the source signals (e.g., clean synthetic waveforms) are fundamentally different from the target signals (e.g., over-the-air captures with multipath fading), the transferred features may be irrelevant or harmful. This phenomenon, known as negative transfer, degrades performance below that of training a small model from scratch. Domain adaptation techniques are required to bridge this gap.

Not all layers in a deep network are equally transferable. Lower layers typically learn general-purpose features like energy distribution and spectral edges, which transfer well across different modulation tasks. Higher layers learn task-specific features that are highly sensitive to the original source domain. Effective transfer learning often involves selectively fine-tuning higher layers while freezing lower ones, a strategy that balances knowledge retention with domain adaptation.

When fine-tuning a pre-trained model on a new target task, the network can abruptly overwrite its previously learned, generalizable features. This is catastrophic forgetting. Mitigation strategies include:

• Using a small learning rate to constrain weight updates.
• Elastic Weight Consolidation (EWC), which penalizes changes to parameters deemed important for the source task.
• Interleaving target data with a small replay buffer of source data during training.

A pre-trained model carries an inductive bias—a set of assumptions learned from the source domain about what patterns are important. For signal classification, a model pre-trained on a vast corpus of modulation types has already learned that cyclostationary patterns and constellation geometries are salient. This bias allows the model to converge faster and generalize better from a few examples than a randomly initialized network, which must learn these fundamental signal properties from scratch.