Backpropagation Through Time (BPTT)

BPTT transforms a recurrent neural network into a feedforward computational graph by creating a copy of the network for each time step in the input sequence. For a predistorter processing a sequence of I/Q samples, the network is unrolled across N time steps, creating N identical copies with shared weights. The error signal is then computed at the final output and propagated backward through every unrolled layer simultaneously. This allows the gradient to flow through the temporal dependencies that model the power amplifier's memory effects. The unrolling depth directly determines how far back in time the algorithm can assign credit or blame for the current predistortion error.

Full BPTT unrolls the network over the entire input sequence, which becomes computationally prohibitive for long signal recordings. Truncated BPTT limits the backward pass to a fixed window of K time steps, significantly reducing memory consumption and training time. For power amplifier linearization, this truncation is physically justified because the PA's memory effects are finite—the current output depends only on a limited history of past inputs. The truncation length K is typically set to exceed the PA's memory depth, ensuring that all relevant temporal dependencies are captured without wasting computation on negligible long-range effects.

During the backward pass, BPTT computes the gradient of the loss with respect to each shared weight by summing the gradients from every unrolled time step. For a weight matrix W that appears at every time step, the total gradient is:

• ∂L/∂W = Σ ∂Lₜ/∂W for t = 1 to T

This accumulation captures how each parameter influences the predistortion error across the entire temporal sequence. The shared-weight constraint ensures that the recurrent network learns a single, time-invariant transformation that generalizes across all time steps, which is essential for modeling the PA's consistent physical behavior.

BPTT is susceptible to vanishing gradients when the recurrent weight matrix has eigenvalues less than 1, causing the error signal to decay exponentially as it propagates backward through time. This prevents the network from learning long-range temporal dependencies. Conversely, exploding gradients occur when eigenvalues exceed 1, causing weight updates to become unstable. For predistorter training:

• Gradient clipping caps the gradient norm to prevent explosions
• Gated architectures like LSTMs mitigate vanishing gradients
• Proper weight initialization using Xavier or orthogonal initialization stabilizes early training
• Skip connections in deep recurrent structures provide gradient highways

Real-Time Recurrent Learning (RTRL) is an alternative to BPTT that computes gradients forward in time rather than backward. While BPTT requires storing the entire unrolled network state for the backward pass, RTRL maintains a Jacobian matrix that tracks how each hidden state changes with respect to each weight. For predistorter applications:

• RTRL enables true online learning without sequence storage
• The computational cost is O(n³) for n hidden units, versus BPTT's O(n²)
• RTRL is impractical for large networks but useful for small, continuous-adaptation predistorters
• BPTT remains the dominant algorithm for offline training due to its computational efficiency

In the Indirect Learning Architecture (ILA) for digital predistortion, BPTT trains a postdistorter neural network on the PA's output signal. The training process:

1. Unroll the recurrent postdistorter over the PA output sequence
2. Forward pass: Generate postdistorted estimates at each time step
3. Compute loss: Compare postdistorter output to the desired predistorter input
4. Backward pass: Propagate error gradients through time using BPTT
5. Update weights: Apply accumulated gradients via an optimizer like Adam

Once trained, the postdistorter weights are copied directly to the predistorter, assuming the nonlinearity commutability property holds for the PA model.

A practical variant of BPTT that limits the unrolling depth to a fixed number of time steps rather than the full sequence length. This reduces computational cost and prevents vanishing gradients in very long sequences.

• Processes sequences in manageable chunks of k time steps
• Resets the hidden state between batches to maintain manageable memory usage
• Essential for real-time DPD where infinite streaming data must be processed
• Trade-off: shorter truncation lengths may miss long-term PA memory effects

An alternative gradient computation algorithm that maintains the exact gradient at each time step without requiring unrolling. Unlike BPTT, RTRL computes forward-mode differentiation, updating weights online as each sample arrives.

• Eliminates the need to store past activations for backward passes
• Computational complexity scales as O(n⁴) for n hidden units, making it expensive for large networks
• Attractive for streaming DPD applications where low latency is critical
• Often approximated via sparse or low-rank variants for practical implementation

The fundamental numerical instability that BPTT must overcome when training recurrent networks. During backpropagation, gradients are repeatedly multiplied by the recurrent weight matrix, causing them to decay exponentially (vanish) or grow exponentially (explode) over long sequences.

• Vanishing gradients prevent learning long-range PA memory dependencies
• Exploding gradients cause weight updates to oscillate or diverge
• Mitigation strategies include gradient clipping, LSTM/GRU cells, and proper weight initialization
• Critical consideration when modeling thermal memory effects spanning thousands of samples

A gated recurrent architecture specifically designed to mitigate vanishing gradients during BPTT training. LSTMs introduce input, forget, and output gates that regulate information flow, enabling the network to learn dependencies over hundreds of time steps.

• The constant error carousel (CEC) preserves gradient magnitude across long sequences
• Forget gates allow the network to dynamically reset memory when PA operating conditions change
• Particularly effective for modeling long-term thermal memory effects in GaN power amplifiers
• Requires more parameters than simple RNNs, increasing inference latency on FPGA implementations

A streamlined recurrent architecture that simplifies the LSTM by merging the cell state and hidden state, using only reset and update gates. GRUs achieve comparable long-term memory modeling with fewer parameters.

• Reduced gate count lowers computational overhead during BPTT training
• Update gate controls the blend between previous hidden state and new candidate activation
• Reset gate determines how much past information to discard
• Preferred for resource-constrained DPD implementations where parameter count directly impacts FPGA LUT utilization

A training strategy where the ground-truth previous output is fed as input during BPTT unrolling, rather than the network's own prediction. This stabilizes early training by preventing error accumulation through the sequence.

• Accelerates convergence when the network initially produces poor predistortion estimates
• Creates a train-test mismatch known as exposure bias
• Scheduled sampling gradually transitions from teacher forcing to self-generated inputs
• Relevant for DLA architectures where the predistorter must eventually operate in open-loop mode