Transformer Network

The core innovation of the transformer is the self-attention mechanism, which computes a weighted representation of every position in an input sequence relative to all others simultaneously. For modulation recognition, this means the model can directly relate a symbol at time t to a symbol at time t+n without the sequential bottleneck of an RNN. Each IQ sample or time-step embedding generates query, key, and value vectors. The dot-product attention score determines how much focus each position places on every other position, enabling the network to capture long-range dependencies such as cyclic prefixes in OFDM or spreading codes in DSSS signals. This global receptive field is critical for distinguishing modulation schemes with similar local structures but different temporal patterns.

Rather than computing a single attention function, the transformer employs multi-head attention, which projects the queries, keys, and values into multiple lower-dimensional subspaces. Each head learns to attend to different aspects of the signal:

• Head 1: May focus on phase transitions between consecutive symbols
• Head 2: May attend to amplitude envelope variations
• Head 3: Could capture periodic preamble structures
• Head 4: Might identify inter-symbol interference patterns The outputs are concatenated and linearly projected, allowing the model to jointly attend to information from different representation subspaces. This parallelized analysis is particularly effective for high-order QAM and APSK schemes where subtle amplitude-phase relationships define the modulation class.

Since the self-attention operation is permutation-invariant (it has no inherent notion of sequence order), transformers require positional encoding to inject information about the position of each element in the input sequence. For modulation recognition, this is typically implemented using sinusoidal positional encodings or learned position embeddings added to the input IQ sample representations. The encoding uses sine and cosine functions of different frequencies:

• PE(pos, 2i) = sin(pos / 10000^(2i/d_model))
• PE(pos, 2i+1) = cos(pos / 10000^(2i/d_model)) This allows the model to distinguish between identical constellation points occurring at different times, which is essential for recognizing differential modulation schemes like DQPSK where the information is encoded in the phase difference between consecutive symbols.

Each transformer layer contains a position-wise feed-forward network applied identically to each position, consisting of two linear transformations with a non-linear activation (typically GELU or ReLU) in between. This component introduces non-linearity and increases the representational capacity of the model. Crucially, both the multi-head attention and feed-forward sub-layers are wrapped with residual connections followed by layer normalization:

• Output = LayerNorm(x + Sublayer(x)) These skip connections enable gradient flow through very deep architectures, allowing the construction of transformers with 12, 24, or more layers for complex modulation recognition tasks. The residual structure prevents the vanishing gradient problem that plagued early deep networks, ensuring stable training even on noisy, low-SNR signal data.

When processing time-frequency representations rather than raw IQ samples, transformers adapted from vision architectures use patch embedding to convert spectrograms into token sequences. The spectrogram image is divided into fixed-size patches (e.g., 16×16 frequency-time bins), which are flattened and linearly projected into the model's embedding dimension. A learnable [CLS] token is prepended to the sequence, and its final hidden state serves as the aggregate representation for modulation classification. This approach allows the transformer to learn joint time-frequency features without the inductive bias of convolutional kernels, often outperforming CNN-based spectrogram classifiers on modulation datasets like RadioML and HisarMod.

In advanced modulation recognition architectures, cross-attention extends the self-attention mechanism to fuse information from multiple signal representations. Instead of queries, keys, and values coming from the same sequence, cross-attention allows one modality to attend to another:

• Queries: Derived from IQ sample embeddings
• Keys and Values: Derived from constellation diagram features or cyclostationary signatures This enables the model to dynamically integrate complementary signal views, such as correlating a transient time-domain anomaly with its spectral signature. Cross-attention is particularly valuable in MIMO modulation recognition where spatial stream separation benefits from fusing per-antenna IQ data with channel state information estimates.