Recurrent Neural Network (RNN)

A specialized RNN architecture engineered to solve the vanishing gradient problem that plagues standard RNNs during long sequence training. LSTMs introduce a memory cell regulated by three gates:

• Forget gate: Decides what information to discard from the cell state
• Input gate: Determines which new information to store
• Output gate: Controls what information to expose to the next layer This gating mechanism allows LSTMs to learn dependencies spanning hundreds of time steps, making them ideal for analyzing extended signal bursts where identifying features may be separated by significant temporal gaps.

A streamlined variant of the LSTM that merges the cell state and hidden state into a single vector, reducing computational complexity while maintaining comparable performance. The GRU uses only two gates:

• Reset gate: Controls how much past information to forget
• Update gate: Balances between retaining old information and adding new input With fewer parameters than LSTMs, GRUs train faster and generalize well on smaller datasets, making them a practical choice for edge deployment scenarios where model size and inference speed are critical constraints for real-time signal classification.

An architectural pattern that processes input sequences in both forward and backward directions simultaneously, combining two independent hidden states. This dual-pass approach provides the network with full temporal context at every time step—knowledge of both past and future signal characteristics relative to the current point. In RF fingerprinting, bidirectional processing is particularly valuable because transient features at the beginning of a burst may correlate with steady-state behavior later in the transmission. The architecture effectively doubles the network's receptive field without increasing sequence length.

A computational module originally developed to augment RNN-based sequence-to-sequence models by allowing the decoder to dynamically focus on relevant parts of the input sequence. Rather than compressing the entire input into a fixed-length context vector, attention computes a weighted sum of all encoder hidden states:

• Alignment scores measure relevance between decoder state and each encoder position
• Softmax normalization produces interpretable attention weights
• Context vector aggregates information proportionally to importance This mechanism directly inspired the Transformer architecture and remains critical for identifying which specific time segments of a signal contain the most discriminative fingerprinting features.

An encoder-decoder framework built on RNNs where one network encodes a variable-length input sequence into a fixed-dimensional context vector, and a second network decodes this representation into a target sequence. Key characteristics include:

• Teacher forcing: During training, the decoder receives ground-truth previous outputs rather than its own predictions, accelerating convergence
• Inference decoding: Beam search or greedy decoding generates output sequences autoregressively In signal processing, Seq2Seq models can perform tasks like denoising corrupted IQ samples or translating raw waveforms into structured modulation descriptions.

A fundamental training obstacle in deep RNNs where gradients propagated backward through time diminish exponentially, preventing the network from learning long-range dependencies. The mathematical root lies in the repeated multiplication of weight matrices during Backpropagation Through Time (BPTT):

• If the largest eigenvalue of the recurrent weight matrix is less than 1, gradients shrink toward zero
• Early layers receive negligible update signals, effectively preventing learning Solutions include LSTM/GRU architectures with additive gradient paths, gradient clipping to prevent the complementary exploding gradient problem, and skip connections that provide direct gradient highways across time steps.