Long Short-Term Memory PA Model

The core architectural innovation enabling LSTM-based PA models to capture long-term memory effects without suffering from vanishing gradients.

• Forget Gate: Selectively discards irrelevant past state information, preventing accumulation of stale thermal or trapping data
• Input Gate: Controls which new signal features are stored in the cell state, gating instantaneous nonlinear contributions
• Output Gate: Regulates what hidden state information propagates to the next layer, enabling multi-timescale representation

This gating mechanism allows the model to maintain error gradients across hundreds of time steps during backpropagation-through-time training, unlike standard RNNs that fail beyond 10-20 steps.

LSTM PA models simultaneously represent short-term electrical memory (nanosecond-scale trapping effects) and long-term thermal memory (millisecond-scale self-heating dynamics) within a single unified architecture.

• Short-term effects: Bias circuit impedance interactions, surface state trapping in GaN HEMTs
• Long-term effects: Channel temperature variation, substrate thermal time constants
• The cell state acts as an analog memory that can maintain information indefinitely when gates are appropriately configured

This eliminates the need for separate static nonlinearity and linear filter blocks found in Wiener/Hammerstein cascade models, reducing total parameter count while improving fidelity for signals with high peak-to-average power ratios.

Unlike memory polynomial models that operate on a fixed-length tapped delay line, LSTM PA models process variable-length input sequences and learn optimal temporal receptive fields automatically.

• Input: Complex baseband IQ samples presented sequentially
• Hidden state: Encodes the amplifier's dynamic operating point including charge trapping state and junction temperature
• Output: Predicted complex envelope incorporating both AM-AM and AM-PM distortion

The recurrent structure naturally handles envelope-dependent memory effects where the time constants vary with signal amplitude, a phenomenon poorly captured by fixed-coefficient Volterra kernels.

For offline behavioral modeling and model extraction applications, bidirectional LSTM architectures process the input sequence in both forward and reverse directions.

• Forward pass: Captures causal memory effects (past inputs influencing present output)
• Reverse pass: Learns anti-causal dependencies useful for identifying pre-pulse artifacts
• Concatenated hidden states provide richer feature representation for coefficient extraction

This is particularly valuable when characterizing amplifiers with complex biasing networks where energy storage elements create both pre- and post-cursor distortion components that unidirectional models cannot fully represent.

Specialized LSTM variants incorporate domain knowledge from power amplifier physics to improve modeling efficiency:

• Input augmentation: Feeding polynomial basis functions |x(n)|, |x(n)|², |x(n)|³ alongside raw IQ samples to provide explicit nonlinear features
• Output layer design: Separate dense layers for I and Q components to handle IQ imbalance jointly with nonlinear distortion
• State regularization: Adding penalties on hidden state magnitude to prevent overfitting to measurement noise during training

These augmentations reduce the number of LSTM cells required by 40-60% compared to generic architectures while achieving equivalent NMSE performance on wideband LTE and 5G NR test signals.

Deployment of LSTM PA models in digital predistortion feedback paths requires careful optimization of the inference pipeline:

• Weight pruning: Removing near-zero gate weights reduces multiply-accumulate operations by 30-50% with minimal NMSE degradation
• Quantization: INT8 representation of weights and activations enables efficient FPGA implementation without floating-point units
• Stateful batching: Processing consecutive sample windows while preserving hidden state across batch boundaries maintains temporal continuity
• Teacher-student distillation: Training a smaller student LSTM to mimic a larger teacher model preserves memory modeling capability at reduced complexity

Optimized LSTM DPD implementations achieve sub-microsecond latency on Xilinx RFSoC platforms, meeting the tight timing constraints of 5G NR TDD frame structures.

The dependence of a power amplifier's current output on past input values due to thermal, electrical, or charge-trapping phenomena. This causes frequency-dependent distortion that static models cannot capture.

• Thermal memory: Die temperature changes with signal envelope
• Electrical memory: Bias network impedance variations
• Trapping effects: Slow charge states in GaN HEMT devices

LSTM models are specifically designed to learn these long-range temporal dependencies.

The two fundamental nonlinear distortion mechanisms in power amplifiers:

• AM-AM Distortion: Input amplitude variations cause nonlinear output amplitude changes (gain compression or expansion)
• AM-PM Distortion: Input amplitude variations induce unwanted output phase shifts

LSTM behavioral models must simultaneously capture both conversion mechanisms across the entire signal bandwidth to enable effective digital predistortion.

The theoretical foundation for all nonlinear dynamic system modeling. Uses multi-dimensional convolution kernels to represent systems with memory.

• Provides a complete mathematical description of nonlinear memory
• Suffers from exponential complexity growth with memory depth and nonlinear order
• LSTM models offer a compact alternative that learns equivalent representations without explicit kernel estimation

A simplified Volterra series that retains only diagonal terms, dramatically reducing computational complexity while capturing essential memory effects.

• Efficient for moderate memory depths
• Generalized Memory Polynomial (GMP) adds cross-terms for stronger memory effects
• LSTM models can outperform GMP when memory effects span very long time horizons that polynomial structures cannot efficiently represent

The appearance of unwanted frequency components in adjacent channels caused by intermodulation distortion when a band-limited signal passes through a nonlinear PA.

• Quantified by Adjacent Channel Power Ratio (ACPR)
• Primary target for digital predistortion correction
• LSTM-based DPD must accurately predict and cancel regrowth across wide bandwidths for 5G signals

The process of determining model parameters from measured input-output data and verifying generalization performance.

• Normalized Mean Square Error (NMSE): Quantifies in-band modeling fidelity
• Adjacent Channel Error Power Ratio (ACEPR): Assesses out-of-band prediction accuracy
• Cross-validation: Prevents overfitting by testing on unseen data

LSTM training requires careful sequence preparation and regularization to avoid memorizing noise.