Gated Temporal Convolution

The core mechanism splits the input into two parallel paths: a filter path that applies a standard temporal convolution to extract features, and a gate path that applies a separate convolution followed by a sigmoid activation to learn a dynamic mask. The outputs are multiplied element-wise, allowing the network to selectively retain or discard temporal information at each time step. This is mathematically expressed as:

• output = tanh(W_f * x) ⊙ σ(W_g * x)
• The tanh activation on the filter path provides non-linearity, while the sigmoid on the gate path outputs values between 0 and 1, acting as a soft on/off switch.

GTC blocks employ dilated convolutions to exponentially increase the receptive field without adding parameters or layers. Dilation inserts gaps between kernel elements, allowing the network to capture very long-range temporal dependencies efficiently. Causal padding ensures that the prediction at time step t depends only on inputs from time t and earlier, making the architecture suitable for streaming and real-time signal processing where future samples are unavailable. This combination is critical for processing long IQ sequences.

Each gated temporal convolution block is wrapped in a residual connection that adds the block's input directly to its output. This architectural choice mitigates the vanishing gradient problem in deep networks and allows the model to learn identity mappings when beneficial. For signal processing, this means the network can easily pass raw waveform information forward while the gated convolution learns a residual correction or enhancement, stabilizing training and enabling the stacking of many layers for hierarchical feature extraction.

GTC blocks are often used as a local feature extractor preceding a transformer's self-attention layers. The gated convolutions capture fine-grained temporal patterns and local signal structure, while the transformer models global, long-range dependencies across the entire sequence. This hybrid design overcomes the quadratic complexity of pure self-attention on long sequences by providing a compressed, high-quality feature representation. It is a foundational pattern in architectures like the Conformer, adapted for RF and audio processing.

Unlike static convolutional filters, the gating mechanism makes the block input-dependent and dynamic. The gate path learns to modulate the filter's output based on the specific content of the signal at each moment. This allows the network to:

• Suppress noise in low-SNR regions of a waveform.
• Amplify transient events like signal bursts or preamble sequences.
• Ignore silent periods or irrelevant interference. This adaptivity is a key advantage over standard temporal convolutions for non-stationary RF environments.

Gated temporal convolutions offer a highly efficient alternative to recurrent neural networks (RNNs) and pure self-attention for sequence modeling. The operations are fully parallelizable across the time dimension during training, unlike the sequential computation of RNNs. Compared to self-attention, the computational cost scales linearly with sequence length rather than quadratically. This makes GTC blocks ideal for deployment on edge hardware and for processing the high sample rates common in RF machine learning applications.

The core logic of a gated temporal convolution involves a learned sigmoid gate that controls the flow of information. The output is typically the element-wise product of a linear convolution and a gating convolution.

• Formula: Output = Conv1(X) ⊙ σ(Conv2(X))
• Purpose: Allows the network to selectively retain or discard temporal features.
• Benefit: Mitigates the vanishing gradient problem in very deep temporal models by providing a linear path for gradients.

A broader architecture that often utilizes gated temporal convolutions as its primary block. TCNs use dilated causal convolutions to exponentially increase the receptive field without losing resolution.

• Causality: Ensures no information leakage from future time steps.
• Dilation: Allows the network to capture very long-range dependencies efficiently.
• Comparison: Often outperforms canonical RNNs and LSTMs in sequence modeling benchmarks due to superior parallelism and memory retention.

A design pattern where gated temporal convolutions act as a local feature extractor feeding into a global self-attention mechanism. The convolution captures fine-grained temporal dynamics, while the transformer models long-range dependencies.

• Role of GTC: Pre-processes raw waveforms into structured tokens.
• Role of Transformer: Applies self-attention to these tokens for global context.
• Use Case: Highly effective in DeepRx and IQ Transformer models for physical-layer wireless processing.

The specific activation function often used within a gated temporal convolution block. A GLU splits a feature map in half, applying a sigmoid gate to one half and multiplying it with the other.

• Variants: GELU-GLU, Swish-GLU, and ReGLU are common replacements for the sigmoid.
• Property: Provides a linear path for gradients, enabling training of much deeper networks.
• Origin: Popularized in natural language processing before being adopted for time-series and signal data.

An attention masking pattern that restricts the model to only attend to past and present time steps. When combined with gated convolutions, it ensures the entire architecture is suitable for real-time streaming inference.

• Constraint: Prevents the model from peeking at future IQ samples.
• Integration: Gated convolutions naturally enforce causality; causal attention extends this to the global context layer.
• Application: Critical for online channel equalization and adaptive filtering.

A convolution where the kernel is applied over an area larger than its length by skipping input values with a specific step (dilation factor). In a gated temporal block, this allows the receptive field to grow exponentially with depth.

• Receptive Field: For a kernel size k and dilation d, the effective window is k + (k-1)(d-1).
• Advantage: Captures very long sequences without requiring a massive number of layers or pooling.
• Synergy: The gating mechanism ensures only the most salient features from this wide window are propagated.