Least Mean Squares

LMS operates as a stochastic approximation of the steepest descent method. Instead of computing the true gradient over the entire statistical ensemble, it uses the instantaneous gradient of the squared error at each iteration. This eliminates the need for autocorrelation matrix estimation, drastically reducing complexity from O(N²) to O(N) per iteration, where N is the number of filter taps.

The core update equation is: w(n+1) = w(n) + μ * e(n) * x(n)

Where:

• w(n): Current coefficient vector
• μ (mu): Step-size parameter controlling convergence speed
• e(n): Instantaneous error signal (desired minus actual output)
• x(n): Input signal vector

This simple multiply-and-accumulate structure maps directly to FPGA DSP slices for high-throughput implementation.

The step-size parameter μ governs a fundamental trade-off:

• Large μ: Fast convergence but large steady-state misadjustment and potential instability
• Small μ: Low misadjustment noise but slow tracking of time-varying systems

The stability bound is: 0 < μ < 2 / λ_max

where λ_max is the largest eigenvalue of the input autocorrelation matrix. In practice, a normalized LMS variant uses a time-varying step-size to improve robustness against input power fluctuations.

Even after convergence, LMS produces a steady-state mean-square error that exceeds the theoretical Wiener minimum. This excess is quantified by the misadjustment parameter:

M = (μ * tr[R]) / 2

where tr[R] is the trace of the input correlation matrix. For DPD applications, misadjustment directly translates to residual adjacent channel power ratio degradation, making step-size selection critical for meeting spectral mask requirements.

Convergence speed depends on the eigenvalue spread of the input signal. A high condition number (ratio of largest to smallest eigenvalue) causes slow convergence along modes corresponding to small eigenvalues. For communication signals with high peak-to-average power ratio, this can lead to non-uniform adaptation rates across different power regions, motivating variants like sign-data LMS or leaky LMS for improved robustness.

In the Indirect Learning Architecture for digital predistortion, LMS is employed to identify the postdistorter coefficients. The algorithm minimizes the error between the attenuated PA output and the predistorter output. Key advantages:

• Enables sample-by-sample coefficient updates without matrix inversion
• Supports online tracking of PA behavior changes due to temperature and aging
• Implementable in hardware-in-the-loop for real-time adaptive linearization

The batch-mode parent of LMS that solves for optimal coefficients by minimizing the sum of squared residuals across an entire data block. Unlike LMS, it processes all samples at once using matrix inversion or decomposition techniques.

• Requires solving the normal equations: w = (X^H X)^(-1) X^H y
• Provides the Wiener-Hopf optimal solution in stationary environments
• Computationally intensive O(N³) but yields exact solution for given data
• Forms the theoretical baseline against which LMS convergence is measured

An adaptive algorithm that recursively computes the least squares solution with exponentially weighted past data, achieving faster convergence than LMS at the cost of higher computational complexity.

• Updates the inverse autocorrelation matrix with each new sample
• Convergence rate independent of input signal eigenvalue spread
• Computational complexity O(N²) vs LMS O(N)
• Ideal for rapidly changing amplifier characteristics where LMS convergence is too slow

A variant of LMS that normalizes the step size by the instantaneous signal power, making convergence behavior independent of input signal amplitude fluctuations.

• Update rule: w(n+1) = w(n) + (μ / (||x(n)||² + ε)) e*(n) x(n)
• The regularization parameter ε prevents division by zero during silent periods
• Exhibits faster convergence than standard LMS for correlated inputs
• Essential for DPD systems handling signals with high peak-to-average power ratios

The broader optimization framework that LMS instantiates, where the true gradient of the cost function is replaced by a noisy but unbiased estimate computed from a single sample or small mini-batch.

• LMS uses the instantaneous gradient of the squared error as the descent direction
• The noisy gradient estimate causes gradient noise that prevents convergence to exact optimum
• Trade-off: noisy updates enable tracking of time-varying systems like thermally drifting amplifiers
• Forms the basis for stochastic training of neural network DPD models

The mathematical study of how LMS coefficient estimates evolve toward the Wiener solution, characterized by learning curves and misadjustment metrics.

• Mean convergence: E[w(n)] → w_opt requires 0 < μ < 2/λ_max
• Mean-square convergence: E[||w(n) - w_opt||²] bounded by misadjustment M = μ·tr(R)/(2 - μ·tr(R))
• Time constant τ_mse ≈ 1/(2μλ_k) for each mode k, with slowest mode dominating
• Eigenvalue spread of input autocorrelation matrix R determines convergence speed disparity

A regularized LMS variant that introduces a leakage factor to prevent coefficient drift in poorly excited modes, critical for fixed-point DPD implementations with finite precision arithmetic.

• Update rule: w(n+1) = (1 - μγ) w(n) + μ e*(n) x(n) where γ is the leakage parameter
• Prevents coefficient overflow in fixed-point hardware implementations
• Adds controlled bias to combat numerical instability in low-precision DSP
• Particularly useful when input signal has insufficient spectral excitation for all model dimensions