Least Squares Estimation

For linear-in-parameters models like the Memory Polynomial, the least squares estimate has a direct analytical solution: β̂ = (XᵀX)⁻¹Xᵀy. This requires constructing the regressor matrix X from delayed and exponentiated input samples. The solution minimizes the sum of squared residuals ||y - Xβ||² by setting the gradient to zero, yielding the normal equations. This deterministic, non-iterative nature makes it ideal for offline model extraction in DPD systems where computational determinism is critical.

Least squares yields the Best Linear Unbiased Estimator (BLUE) only when key assumptions hold:

• Linearity in parameters: The model output is a linear combination of basis functions (true for Volterra series).
• Zero mean errors: Measurement noise has an expected value of zero.
• Homoscedasticity: The error variance is constant across all observations.
• No autocorrelation: Errors are uncorrelated with each other.
• Full column rank: The regressor matrix X has linearly independent columns. Violations, such as correlated noise from a spectrum analyzer, degrade estimate efficiency.

When the regressor matrix X is ill-conditioned—common with high-order Volterra models—ordinary least squares produces unstable, high-variance coefficient estimates. Ridge regression adds an L2 penalty: β̂ = (XᵀX + λI)⁻¹Xᵀy. The hyperparameter λ controls the bias-variance tradeoff:

• Large λ: Reduces coefficient variance but introduces bias.
• Small λ: Approaches the OLS solution. This is essential for generalized memory polynomial extraction where cross-terms create near-collinearity.

For online DPD adaptation, the batch solution is impractical. Recursive Least Squares updates coefficient estimates sample-by-sample without re-inverting large matrices. The algorithm maintains a gain vector and an inverse correlation matrix P(n), updating β̂(n) = β̂(n-1) + k(n)e(n), where e(n) is the prediction error. A forgetting factor λ (typically 0.95–0.999) exponentially weights recent data, enabling tracking of time-varying PA behavior due to thermal drift or aging.

Directly computing (XᵀX)⁻¹ is numerically unstable for ill-conditioned problems. The condition number κ(XᵀX) = κ(X)² squares the original matrix condition, amplifying rounding errors. Production implementations avoid explicit inversion by solving the normal equations via:

• QR decomposition: X = QR, then solve Rβ̂ = Qᵀy. More stable, O(N³) complexity.
• Singular Value Decomposition (SVD): X = UΣVᵀ, then β̂ = VΣ⁺Uᵀy. Handles rank deficiency gracefully by zeroing small singular values.

When measurement noise is heteroscedastic—common when capturing PA data across varying input power levels—ordinary least squares is inefficient. Weighted Least Squares minimizes Σ wᵢ(yᵢ - xᵢβ)², where weights wᵢ are inversely proportional to the noise variance at each sample. In DPD extraction, higher weights are often assigned to samples in the PA's linear region where SNR is highest, improving model fidelity for low-EVM applications like 1024-QAM.