Mean Squared Error (MSE)

MSE produces a convex quadratic error surface with respect to the predistorter coefficients. This is the single most important property for optimization.

• The cost function has a single global minimum—no local minima traps
• The gradient is linear, enabling simple and efficient gradient-based updates
• The Wiener-Hopf solution provides the theoretical optimum in closed form
• This convexity guarantees that LMS and RLS algorithms will converge to the optimal coefficients given proper step-size selection

The instantaneous gradient of MSE with respect to the coefficient vector is proportional to the product of the error signal and the input regressor vector.

• For a memory polynomial predistorter: ∇J = -2 · e(n) · u(n), where u(n) contains delayed and nonlinear basis terms
• This simplicity enables real-time stochastic gradient descent updates on FPGA hardware
• The gradient is an unbiased estimate of the true gradient when using instantaneous samples
• Computational cost scales linearly with the number of coefficients, making it suitable for wideband applications

MSE penalizes errors quadratically, making it highly sensitive to large deviations between the desired and actual output.

• Large errors from impulsive noise or PA saturation events dominate the cost function
• This sensitivity can pull coefficient estimates away from the optimal linearization point
• In practice, crest factor reduction is often applied before DPD to limit peak excursions
• Alternative cost functions like Least Absolute Deviation (LAD) or Huber loss may be preferred in high-PAPR scenarios
• The quadratic penalty aligns well with ACLR minimization, as spectral regrowth is proportional to distortion power

Minimizing MSE is equivalent to maximizing the likelihood under the assumption of additive white Gaussian noise in the observation path.

• The optimal predictor is the conditional expectation: ŷ = E[y|x]
• MSE decomposes into bias² + variance + irreducible noise
• In DPD, the irreducible noise floor is set by observation receiver SNR and ADC quantization
• This decomposition guides regularization strategies: when the input correlation matrix is ill-conditioned, variance dominates and Tikhonov regularization becomes essential
• The condition number of the data matrix directly impacts MSE estimation variance

In adaptive DPD systems using LMS, the steady-state MSE exceeds the theoretical minimum by an amount called misadjustment.

• Misadjustment M = (μ/2) · tr(R), where μ is the step size and R is the input correlation matrix
• Smaller step sizes reduce misadjustment but slow convergence rate
• This fundamental tradeoff drives the selection of Normalized LMS (NLMS) or RLS for time-varying PA characteristics
• RLS achieves zero misadjustment asymptotically with a forgetting factor of 1.0
• In practice, a forgetting factor of 0.97-0.995 balances tracking capability against steady-state error

Solving the MSE normal equations directly requires inverting the autocorrelation matrix, which can become numerically unstable.

• QR decomposition and SVD are preferred over direct matrix inversion for batch solutions
• The condition number of the data matrix grows with polynomial order and memory depth
• Ill-conditioning manifests as coefficient drift without corresponding MSE improvement
• QR-RLS using Givens rotations maintains numerical stability in fixed-point FPGA implementations
• Adding a regularization parameter δ to the diagonal of R effectively bounds the condition number

A batch estimation method that computes the optimal coefficient vector by minimizing the sum of squared residuals between observed and desired signals. The solution is obtained by solving the Normal Equation in closed form.

• Processes an entire block of captured data at once
• Provides the optimal solution for stationary systems
• Computationally intensive for large datasets due to matrix inversion
• Forms the theoretical baseline against which adaptive algorithms are compared

A foundational stochastic gradient descent algorithm that updates filter coefficients iteratively using the instantaneous estimate of the MSE gradient. Its simplicity makes it ideal for real-time hardware implementation.

• Requires only O(N) operations per iteration
• No matrix inversion or correlation matrix storage needed
• Convergence rate depends on eigenvalue spread of input
• The step-size parameter controls the bias-variance tradeoff

An adaptive algorithm that recursively updates the inverse of the input correlation matrix to achieve an order-of-magnitude faster convergence than LMS. It minimizes a weighted least squares cost function with exponential forgetting.

• Converges in approximately 2N iterations regardless of input conditioning
• Computational complexity of O(N²) per iteration
• The forgetting factor λ controls memory length for tracking time-varying systems
• Numerically stable implementations use QR decomposition or Cholesky factorization

A variant of LMS that normalizes the step size by the instantaneous power of the input vector. This makes convergence behavior independent of signal level fluctuations, a critical property for communication signals with varying envelope power.

• Step size is divided by ||x(n)||² + ε where ε prevents division by zero
• Converges to the Wiener solution under stationary conditions
• Provides stable adaptation without manual gain scaling
• Often the practical default choice for real-time DPD coefficient tracking

The fundamental linear equation R·w_opt = p that defines the optimal weight vector for a Wiener filter. It expresses the solution as the product of the inverse autocorrelation matrix and the cross-correlation vector.

• Represents the theoretical minimum of the MSE cost surface
• Requires knowledge of second-order statistics of the signals
• The misadjustment of any adaptive algorithm measures deviation from this optimum
• Solving this equation directly is the goal of LS batch estimation

An optimization algorithm that updates model parameters using the gradient of the loss function computed from a randomly selected subset of data. LMS is a special case of SGD applied to the MSE cost function.

• Enables online learning without storing the full dataset
• The stochastic gradient is an unbiased estimate of the true gradient
• Mini-batch variants trade variance reduction for computational cost
• Momentum and adaptive learning rate extensions improve convergence in deep networks