LUT Convergence

The error vector magnitude (EVM) floor reached when iterative updates no longer produce meaningful coefficient changes. Convergence is declared when the mean squared error between the desired linear output and the actual PA output stabilizes below a predefined threshold, typically -50 dBc for modern base stations. The residual error floor is fundamentally limited by LUT quantization noise, feedback path SNR, and unmodeled memory effects.

The closed-loop behavior governed by the LMS step size (μ) and loop delay. A large step size accelerates convergence but increases steady-state jitter, while a small step size yields smooth coefficients but risks failing to track rapid thermal transients. The convergence time constant τ is inversely proportional to μ and the input signal power. Practical systems often employ gear-shifting—starting with a large μ for rapid acquisition, then reducing it for fine tracking.

Even after convergence, individual LUT entries exhibit stochastic fluctuation around their optimal values due to gradient noise amplification. This jitter manifests as elevated noise floor in the transmitted spectrum. The variance is proportional to μ² and inversely proportional to the averaging window. Techniques to mitigate jitter include:

• Momentum-based updates (exponential smoothing of gradient)
• LUT smoothing filters applied across adjacent entries
• Sign-sign LMS for reduced hardware complexity at the cost of higher residual variance

Several conditions prevent stable convergence:

• Insufficient excitation: Input signal must exercise all LUT address regions; narrowband signals leave high-power bins untrained
• Feedback path nonlinearity: Distortion in the observation receiver biases coefficient estimates
• Loop delay misalignment: Fractional-sample timing errors between the reference and feedback paths cause coefficient divergence in memory-polynomial LUTs
• PA hysteresis: Thermal memory effects with time constants longer than the adaptation loop cause slow coefficient drift

Engineers monitor multiple indicators to confirm convergence:

• Adjacent Channel Leakage Ratio (ACLR): Stabilization within ±0.2 dB over successive adaptation cycles
• Normalized Mean Squared Error (NMSE): Typically below -35 dB for converged LUTs
• Coefficient delta magnitude: The Euclidean norm of the difference between successive LUT coefficient vectors falling below 10⁻⁴
• Spectral mask compliance: Real-time spectrum analyzer confirming no regrowth spikes

PA nonlinear characteristics drift with junction temperature, forcing the LUT to reconverge. GaN-based Doherty PAs exhibit significant AM-PM drift of up to 5 degrees over a 60°C range. The adaptation loop must reconverge faster than the thermal time constant (typically milliseconds for short-term memory). Advanced systems use temperature-indexed LUT banks that store pre-converged coefficient sets for different thermal states, enabling instantaneous switching without re-adaptation.

The adaptation rate governs how aggressively LUT coefficients are updated each iteration. It represents a fundamental convergence-speed vs. noise trade-off:

• Fast adaptation: Tracks rapid PA characteristic changes (thermal drift, supply voltage sag) but introduces steady-state jitter that causes spectral regrowth
• Slow adaptation: Produces clean, low-noise coefficients but may fail to track dynamic envelope-dependent memory effects
• Typical implementations use variable step sizes that start large for rapid acquisition and decay for fine convergence
• In 5G wideband systems, adaptation rates must exceed the PA's thermal time constants (microseconds to milliseconds)

Interpolation error is the residual distortion that remains after LUT convergence due to approximating a continuous predistortion function with discrete table entries. Even a perfectly converged LUT exhibits this error:

• Linear interpolation between adjacent entries reduces error compared to zero-order hold (nearest-neighbor)
• Error magnitude scales with LUT granularity — finer spacing reduces interpolation error but increases memory footprint
• Cubic or polynomial interpolation can further suppress error at the cost of additional multiply-accumulate operations
• In practice, interpolation error sets the noise floor for achievable adjacent channel leakage ratio (ACLR) improvement

Quantization error arises from representing predistortion coefficients with finite bit-widths in hardware. This error limits the ultimate convergence floor of any LUT-based DPD system:

• Coefficient quantization: Storing complex-gain values with limited precision (e.g., 12-bit I/Q) introduces granular noise
• Address quantization: Mapping continuous input envelope to discrete LUT addresses creates binning error
• The combined effect produces a distortion floor that no amount of adaptation can overcome
• Typical implementations use 14-16 bit coefficients to push quantization noise below -65 dBc relative to the carrier
• Joint optimization of address and coefficient bit-widths is critical for ASIC implementations targeting minimal power consumption

A dual-buffer memory scheme that decouples LUT adaptation from real-time predistortion to ensure seamless, glitch-free convergence:

• Active table: Drives the predistorter in the forward signal path without interruption
• Shadow table: Updated in the background by the adaptation algorithm using the latest error measurements
• Once the shadow table converges, the buffers are atomically swapped
• Eliminates the risk of transmitting partially-updated, incoherent coefficients that cause spectral splatter
• Essential for mission-critical wireless infrastructure where link continuity cannot be compromised during retraining

A post-convergence filtering technique applied across adjacent LUT entries to suppress adaptation noise and prevent discontinuous coefficient transitions:

• Moving average filters across neighboring bins remove random jitter from LMS updates
• Polynomial fitting enforces smooth AM-AM and AM-PM curves consistent with PA physics
• Smoothing prevents sharp gain discontinuities that generate spectral regrowth in adjacent channels
• Must be applied judiciously — over-smoothing can mask genuine nonlinear structure and degrade linearization
• Often implemented as a low-pass filter on the coefficient vector after each adaptation epoch