Convergence Rate

The learning rate (μ in LMS, η in SGD) is the primary control knob for convergence speed. A larger step size enables faster initial convergence but increases steady-state misadjustment—the residual error floor after convergence. The optimal value represents a trade-off:

• Too large: Coefficient trajectories oscillate or diverge, especially with noisy gradient estimates
• Too small: Convergence is unacceptably slow, failing to track thermal drift or channel changes
• Adaptive step size: Algorithms like NLMS normalize the step by instantaneous signal power, maintaining stability across varying input levels without manual tuning

The condition number—the ratio of the largest to smallest eigenvalue of the input correlation matrix—directly determines convergence speed limits. A wide eigenvalue spread indicates highly correlated basis functions, causing gradient-based algorithms to converge slowly along directions corresponding to small eigenvalues.

• Well-conditioned: Eigenvalues clustered near unity → rapid, uniform convergence
• Ill-conditioned: Eigenvalues spanning orders of magnitude → slow convergence dominated by the smallest mode
• Mitigation: QR decomposition and orthogonal basis functions (e.g., orthogonal memory polynomials) pre-whiten the input, dramatically improving conditioning

The fundamental algorithm family dictates the convergence trajectory:

• LMS/SGD: First-order stochastic methods with O(N) complexity per iteration. Convergence is linear and sensitive to eigenvalue spread. Simple to implement but slow for ill-conditioned problems
• RLS: Second-order recursive method with O(N²) complexity. Achieves an order of magnitude faster convergence by inverting the correlation matrix recursively, making it nearly insensitive to eigenvalue spread
• NLMS: Normalizes the LMS update by input power, providing a middle ground with improved convergence stability for non-stationary signals at minimal added cost

The forgetting factor (λ in RLS, typically 0.95–0.999) controls the algorithm's memory horizon. It exponentially weights recent data more heavily, enabling tracking of time-varying PA characteristics:

• λ close to 1: Long memory, low steady-state error, but slow to respond to changes in amplifier behavior due to thermal drift or channel switching
• λ smaller: Short memory, rapid adaptation to non-stationary conditions, but higher noise sensitivity and misadjustment
• Effective memory: Approximately 1/(1-λ) samples, defining the window over which the algorithm averages past observations

Convergence requires persistent excitation—the input signal must sufficiently excite all modes of the predistorter model. Without rich spectral content spanning the basis function space, certain coefficients remain unobservable:

• Low PAPR signals: Narrow dynamic range provides poor excitation of high-order nonlinear terms, slowing convergence of those coefficients
• Silence intervals: Periods of no transmission cause coefficient drift; coefficient freeze mechanisms halt adaptation to prevent divergence
• Crest factor reduction: Deliberate PAPR reduction improves PA efficiency but can reduce excitation richness, requiring careful learning rate scheduling during low-variance intervals

Misalignment between the reference and feedback signals introduces a phase error that corrupts the error signal driving adaptation. Even sub-sample misalignment creates a frequency-dependent phase rotation that:

• Slows convergence: The gradient direction is rotated, forcing the algorithm to take indirect paths toward the optimum
• Increases misadjustment: Residual alignment error appears as uncorrelated noise, raising the steady-state error floor
• Requires fractional delay filters: Sub-sample alignment using Farrow structure interpolators is essential for wideband signals where the loop delay is not an integer multiple of the sample period

A hyperparameter that controls the step size of coefficient updates in gradient-based optimization. It directly governs the convergence rate and steady-state misadjustment.

• High learning rate: Faster initial convergence but risks overshooting the minimum and oscillating around the Wiener solution.
• Low learning rate: Slower, more stable convergence with lower steady-state error.
• Adaptive strategies: Time-varying learning rates (e.g., decaying schedules) can accelerate initial acquisition while maintaining low final misadjustment.
• In NLMS, the learning rate is effectively normalized by input signal power, decoupling convergence speed from signal amplitude.

An adaptive algorithm that recursively minimizes a weighted linear least squares cost function. RLS achieves an order-of-magnitude faster convergence rate than LMS by using all available data at each iteration.

• Mechanism: Updates the inverse correlation matrix directly via the matrix inversion lemma, avoiding explicit matrix inversion.
• Convergence: Typically converges in ~2N iterations where N is the number of filter taps, compared to ~10N+ for LMS.
• Cost: O(N²) computational complexity per iteration versus O(N) for LMS, making hardware implementation challenging.
• Forgetting factor (λ): Introduces exponential weighting to track time-varying systems, trading steady-state error for tracking agility.

A weighting parameter (λ, typically 0.95–0.999) in recursive algorithms like RLS that exponentially discounts older data. It directly influences the convergence rate and tracking capability in non-stationary environments.

• λ close to 1: Longer memory, better steady-state performance, slower adaptation to PA characteristic changes.
• λ smaller: Shorter memory, faster tracking of thermal drift and bias shifts, but higher steady-state misadjustment.
• Effective memory: The asymptotic data window length is approximately 1/(1-λ) samples.
• Critical for tracking thermal memory effects in GaN power amplifiers where trap states evolve over milliseconds.

The excess mean squared error above the theoretical Wiener optimum that persists after convergence. It represents the fundamental trade-off with convergence rate.

• Misadjustment ∝ μ: In LMS, steady-state error scales linearly with step size, creating a direct trade-off with convergence speed.
• Misadjustment ∝ (1-λ): In RLS, smaller forgetting factors increase misadjustment while improving tracking agility.
• Practical targets: For ACLR-limited systems, misadjustment must remain below -50 dB relative to the optimal solution to avoid degrading linearization performance.
• The error floor is ultimately bounded by feedback receiver SNR and time-alignment residual jitter.

The robustness of the coefficient estimation algorithm to finite-precision arithmetic and ill-conditioned correlation matrices. Poor numerical stability can stall or reverse convergence rate in fixed-point hardware.

• Ill-conditioning: When basis functions are highly correlated (e.g., closely spaced memory taps), the condition number of the correlation matrix explodes, amplifying quantization noise in gradient calculations.
• Regularization: Adding a small diagonal term (δI) to the correlation matrix improves conditioning at the cost of introducing a slight bias that slows convergence.
• QR decomposition: Preferred over direct matrix inversion for block-based estimation because it maintains orthogonality and avoids squaring the condition number.
• Fixed-point considerations: We use 18-bit or 25-bit word lengths with careful scaling to prevent overflow during the recursive update of the inverse correlation matrix.

A control mechanism that halts the adaptation loop to lock predistorter coefficients, preventing divergence that would destroy the convergence rate gains achieved during valid operation.

• Trigger conditions: Low input signal power, feedback receiver saturation, or detected loop instability all trigger a freeze to hold the last valid coefficient set.
• Hysteresis: Freeze/unfreeze thresholds include hysteresis to prevent rapid toggling that would introduce transient spectral regrowth.
• Recovery: Upon unfreeze, the adaptation resumes from the frozen state rather than resetting, preserving prior convergence progress.
• Essential for TDD systems where the PA is inactive during receive slots and the feedback path sees only noise.