Recurrent Neural Network (RNN)

The core architectural innovation of an RNN is its hidden state vector, which acts as a persistent memory that is updated at each time step. Unlike feedforward networks that process inputs independently, the RNN feeds its previous hidden state back into the network along with the current input. This creates a recurrent loop that allows information to persist across time steps, enabling the network to condition its current output on the entire history of the input sequence. For IQ sample streams, this means the network can remember phase transitions and amplitude patterns from earlier symbols to inform the classification of the current modulation scheme.

During training, the recurrent structure is conceptually unrolled through time into a deep feedforward network where each layer corresponds to a time step. This unrolling reveals that an RNN is essentially a very deep network with shared weights across all time steps. The shared weight matrices (W for input-to-hidden, U for hidden-to-hidden, V for hidden-to-output) are identical at every temporal layer, drastically reducing the parameter count compared to a naive deep network. This weight sharing enforces the inductive bias that the same temporal dynamics apply regardless of absolute position in the sequence, making the architecture naturally suited for variable-length IQ sample streams.

The primary architectural limitation of vanilla RNNs stems from the backpropagation through time (BPTT) algorithm. When gradients are propagated backward through the unrolled network, they are repeatedly multiplied by the recurrent weight matrix. If the largest eigenvalue of this matrix is less than 1, gradients vanish exponentially, preventing the network from learning long-range dependencies. If it is greater than 1, gradients explode, causing training instability. This fundamental challenge makes standard RNNs ineffective at capturing dependencies spanning more than approximately 10 time steps, directly limiting their ability to model long symbol sequences in low-SNR modulation classification scenarios.

RNNs support multiple input-output configuration patterns, making them highly adaptable to different signal processing tasks:

• Many-to-One: A sequence of IQ samples is mapped to a single modulation class label, the standard configuration for AMC.
• Many-to-Many (Synced): Each input time step produces a corresponding output, useful for per-symbol demodulation or channel equalization.
• Many-to-Many (Unsynced): An encoder processes the full input sequence into a context vector, then a decoder generates an output sequence, applicable to signal translation tasks.
• One-to-Many: A single input generates a sequence, used in synthetic signal generation conditioned on modulation type.

A bidirectional RNN (BiRNN) processes the input sequence in both forward and reverse directions using two separate hidden state sequences. The forward pass captures causal dependencies from past to present, while the backward pass captures anti-causal dependencies from future to present. The two hidden states are concatenated at each time step, providing the output layer with complete contextual awareness of the entire sequence. For modulation classification, this is particularly valuable because the optimal decision for a received symbol often depends on both preceding and subsequent samples, especially in schemes with memory like GMSK or when inter-symbol interference is present.

Multiple RNN layers can be stacked hierarchically to form a deep recurrent network, where the hidden state sequence of one layer serves as the input sequence to the next. Lower layers typically learn short-term temporal features such as symbol transitions and phase shifts, while higher layers aggregate these into longer-term abstractions representing modulation-specific patterns. Each layer operates at a different temporal granularity, creating a hierarchical representation of the signal. This depth is critical for distinguishing between modulation schemes that share local characteristics but differ in their higher-order temporal structure, such as QPSK versus OQPSK.

A streamlined RNN variant that combines the forget and input gates into a single update gate, and merges the cell state with the hidden state. GRUs have fewer parameters than LSTMs, making them computationally efficient for real-time modulation classification on edge hardware.

• Update gate: Controls how much past information is carried forward
• Reset gate: Determines how much past information to forget

Often preferred when training data is limited or inference latency is critical.

The training algorithm that unrolls an RNN across its temporal steps and applies standard backpropagation to compute gradients. BPTT treats the network as a deep feedforward network with shared weights across all time steps.

• Truncated BPTT: Limits the unroll length to manage memory and mitigate vanishing gradients
• Full BPTT: Processes the entire sequence, computationally expensive for long IQ streams

Understanding BPTT is essential for debugging training instability in temporal classifiers.

Fundamental training pathologies in RNNs where gradients either shrink exponentially (vanishing) or grow uncontrollably (exploding) during BPTT. Vanishing gradients prevent the network from learning long-range dependencies, while exploding gradients cause weight updates to destabilize.

• Gradient clipping: Caps gradient magnitudes to prevent explosions
• Skip connections: Allow gradients to bypass temporal steps
• Gating mechanisms: LSTMs and GRUs were explicitly designed to address this problem

These issues are why vanilla RNNs are rarely used for sequences exceeding 10-20 time steps.

An architecture that processes input sequences in both forward and backward directions using two separate hidden layers, then concatenates their outputs. This gives the network access to future context at each time step, not just past information.

• Forward layer: Processes the sequence from t=1 to t=T
• Backward layer: Processes from t=T to t=1
• Concatenated output: Combines both directional representations

Particularly useful for offline modulation classification where the entire signal burst is available before inference.

An encoder-decoder architecture where an RNN encoder compresses a variable-length input sequence into a fixed-length context vector, and an RNN decoder generates an output sequence from that representation. In modulation recognition, this enables tasks beyond classification, such as signal denoising or symbol decoding.

• Encoder: Maps input IQ samples to a latent representation
• Decoder: Generates target sequence from the latent vector
• Attention mechanisms: Allow the decoder to focus on specific encoder time steps

Extends RNNs from simple classification to structured output generation.