Long Short-Term Memory (LSTM)

The foundational mechanism of LSTM that preserves gradient flow across many time steps. The CEC is a self-connected linear unit with a fixed weight of 1.0, allowing error signals to propagate backward through time without vanishing or exploding. This enables the network to bridge time lags exceeding 1000 discrete steps, making it uniquely suited for capturing long-term modulation pattern dependencies in raw IQ streams.

A sigmoid-activated neural layer that outputs values between 0 and 1 for each component of the cell state, determining what information to discard. A value of 0 means 'completely forget,' while 1 means 'completely retain.' In modulation recognition, this gate learns to flush irrelevant channel noise and outdated signal characteristics when the transmission environment changes, preventing stale information from corrupting classification decisions.

A two-part mechanism combining a sigmoid layer that decides which values to update and a tanh layer that creates a vector of new candidate values. This gate acts as a filter, admitting only salient signal features—such as phase transitions or amplitude shifts—into the cell state while blocking redundant or noisy input. The tanh activation squashes candidate values to the range [-1, 1], stabilizing the learning dynamics.

The final gating mechanism that determines what parts of the cell state are exposed to the hidden state and subsequent layers. It applies a sigmoid filter to the current input and previous hidden state, then multiplies the result with a tanh-squashed version of the updated cell state. This decoupling allows the LSTM to maintain a rich internal memory while revealing only task-relevant information for modulation classification at each time step.

An architectural variant that processes the input sequence in both forward and backward directions using two independent LSTM layers. The hidden states from both directions are concatenated at each time step, providing the classifier with full temporal context—past and future signal samples—simultaneously. This is particularly effective for modulation schemes where the meaning of a symbol depends on surrounding symbols, such as differential encoding formats.

An architectural enhancement where the cell state is directly exposed to the forget, input, and output gates. Unlike the standard LSTM where gates only see the hidden state and current input, peephole connections allow the cell state itself to influence gating decisions. This provides finer temporal precision, enabling the network to learn precise timing patterns in signals, such as symbol boundary transitions in burst-mode transmissions.

The foundational sequential architecture from which LSTMs evolved. Standard RNNs process time-series data by maintaining a hidden state that is updated at each time step, but they suffer from the vanishing gradient problem when modeling long-range dependencies. LSTMs explicitly address this limitation through their gating mechanisms, making them the preferred choice for tasks like raw IQ sample processing where temporal correlations span hundreds of time steps.

A streamlined variant of the LSTM that merges the cell state and hidden state into a single vector and combines the forget and input gates into a single update gate. GRUs have fewer parameters than LSTMs, making them computationally lighter and often equally effective for modulation classification tasks where training data is limited. Key trade-off: reduced representational capacity for capturing very long-term dependencies in exchange for faster convergence.

The core training pathology that LSTMs were designed to solve. In deep or recurrent networks, gradients propagated backward through time can decay exponentially, preventing the model from learning long-range temporal dependencies. LSTMs mitigate this through a constant error carousel—the cell state pathway—which allows gradients to flow across hundreds of time steps without multiplicative decay, enabling effective learning from distant signal events.

A critical LSTM component that controls what information is discarded from the cell state. It takes the previous hidden state and current input, passes them through a sigmoid activation, and outputs a value between 0 and 1 for each cell state element. A value near 0 means 'completely forget this,' while 1 means 'completely retain this.' In modulation recognition, this allows the network to reset its memory when a signal's modulation characteristics fundamentally change.

An architecture that runs two independent LSTM layers—one processing the input sequence forward and the other backward—then concatenates their hidden states. This gives the network access to both past and future context at every time step. For modulation classification, BiLSTMs are particularly effective because the optimal decision for a signal segment often depends on both preceding and subsequent IQ samples, capturing non-causal signal structure.

A complementary technique often paired with LSTMs to improve sequence modeling. While the LSTM maintains a compressed summary of the past in its hidden state, attention allows the decoder to dynamically weight and access all previous hidden states when making a prediction. In modulation recognition, attention helps the model focus on the most discriminative signal segments—such as symbol transitions—rather than relying solely on the final hidden state.