Ridge Regression

Ridge regression augments the standard least squares cost function with a penalty term proportional to the squared L2-norm of the coefficient vector. This penalty is scaled by the regularization parameter (λ). The modified cost function becomes: ||y - Xβ||² + λ||β||². This shrinks coefficient magnitudes toward zero but never exactly to zero, unlike L1 (LASSO) regularization. In DPD, this prevents the estimator from assigning excessively large, oscillating values to correlated basis functions that would otherwise cancel each other on training data but fail on unseen signals.

Memory polynomial and GMP basis function matrices often exhibit high multicollinearity—near-linear dependencies between columns caused by correlated signal samples and envelope cross-terms. This makes the ordinary least squares (OLS) matrix XᴴX ill-conditioned or nearly singular. Ridge regression adds a diagonal loading term: β̂ = (XᴴX + λI)⁻¹Xᴴy. The λI term stabilizes the matrix inversion, dramatically reducing the condition number and preventing the explosive variance in coefficient estimates that plagues unregularized solutions in low-EVM scenarios.

Ridge regression introduces a controlled amount of estimation bias to achieve a larger reduction in prediction variance. Key characteristics:

• λ = 0: Reduces to OLS—unbiased but high variance with ill-conditioned data
• λ → ∞: Coefficients shrink to zero—high bias, zero variance
• Optimal λ: Minimizes mean squared error on validation data by balancing the two

In DPD, a small bias in coefficient magnitude is acceptable because it yields a predistorter that generalizes robustly across varying signal statistics, rather than overfitting to a specific training waveform.

Unlike iterative gradient-based methods, ridge regression admits a direct closed-form solution: β̂_ridge = (XᴴX + λI)⁻¹Xᴴy. This is computationally advantageous for offline DPD model extraction where the entire training dataset is available. The solution requires a single matrix inversion of size (M × M), where M is the number of basis functions. For typical DPD models with hundreds of coefficients, this is tractable on a host processor. The analytical nature guarantees a global minimum of the regularized cost function, avoiding convergence uncertainty.

The primary practical benefit of ridge regression in DPD is improved linearization robustness when the predistorter processes signals with statistics different from the training data. An unregularized LS solution may achieve perfect cancellation on training data but produce spectral regrowth on a different modulation or power level. Ridge regression constrains coefficient magnitudes, producing a smoother predistorter transfer function that interpolates more reliably. This is critical for commercial transmitters that must maintain ACLR compliance across dynamic traffic patterns and varying resource block allocations.

Ridge regression has a direct Bayesian interpretation: it is equivalent to maximum a posteriori (MAP) estimation under a Gaussian prior on the coefficients with zero mean and variance proportional to 1/λ. This perspective provides a principled framework for selecting the regularization parameter based on expected coefficient distributions. In DPD, this aligns with the physical expectation that higher-order nonlinear terms should have progressively smaller magnitudes, reflecting the natural decay of Volterra kernel significance with increasing nonlinear order.

The foundational batch algorithm that ridge regression modifies. LS estimation finds the coefficient vector that minimizes the sum of squared errors between the power amplifier's measured output and the model's prediction.

• Solves the normal equations directly: w = (XᴴX)⁻¹ Xᴴ y
• Highly susceptible to ill-conditioning when basis functions are correlated
• Serves as the unregularized baseline against which ridge regression is compared

A scalar hyperparameter that controls the trade-off between model fit and coefficient magnitude. A larger λ heavily penalizes large coefficients, shrinking them toward zero to prevent overfitting.

• λ = 0 reduces to ordinary least squares
• λ → ∞ forces all coefficients to zero
• Optimal λ is typically selected via cross-validation or the L-curve criterion
• In DPD, λ prevents the predistorter from fitting measurement noise

A numerical conditioning process that transforms correlated polynomial basis functions into an orthogonal set before coefficient estimation. This preprocessing step dramatically improves the stability of the normal equations.

• Common methods include Gram-Schmidt and QR decomposition
• Orthogonalized bases produce a diagonal XᴴX matrix
• Often used in conjunction with ridge regression for severely ill-conditioned memory polynomial models
• Reduces the condition number of the data matrix by orders of magnitude

A matrix factorization method that decomposes the basis function matrix X into an orthogonal matrix Q and an upper triangular matrix R. QRD-RLS variants solve the least squares problem with superior numerical stability.

• Avoids explicitly computing the ill-conditioned XᴴX matrix
• Enables direct solution via back-substitution: R w = Qᴴ y
• The preferred method for implementing ridge regression in fixed-point FPGA hardware
• Provides a natural framework for adding the regularization penalty to the diagonal of R

A dimensionality reduction technique applied to the basis function matrix to identify and retain only the most significant principal components. PCA addresses multicollinearity by projecting data onto orthogonal axes of maximum variance.

• Eigenvalues of XᴴX indicate the variance captured by each component
• Discarding low-variance components acts as a form of regularization
• Often combined with ridge regression in a technique called principal component regression
• Reduces model complexity while preserving linearization accuracy

A greedy sparse approximation algorithm that iteratively selects the most correlated basis function from a dictionary to build a compact predistorter model. OMP provides an alternative to ridge regression for managing model complexity.

• At each iteration, selects the basis function with the highest correlation to the current residual
• Re-estimates all coefficients via least squares on the selected subset
• Terminates when a sparsity target or error threshold is reached
• Produces inherently sparse models without requiring a regularization parameter