Regularization Parameter

The regularization parameter directly balances underfitting and overfitting in memory polynomial coefficient extraction. A λ value of zero reduces to ordinary least squares, which may fit training data perfectly but produce high variance when the signal statistics change. Increasing λ introduces bias by shrinking coefficient magnitudes, which reduces sensitivity to noise in the observation matrix. The optimal λ minimizes the mean squared error on validation data, not training data.

In ridge regression, the regularization parameter adds a diagonal loading term (λI) to the autocorrelation matrix before inversion. This operation improves the condition number of the matrix, making the solution more stable when basis functions are highly correlated. For memory polynomial models with deep memory taps, the regressor matrix often becomes ill-conditioned. A well-chosen λ prevents the coefficient vector from oscillating wildly due to small singular values.

The optimal regularization parameter is typically determined through k-fold cross-validation. The training dataset is partitioned into k subsets, and the model is trained k times with each subset serving as a validation fold. The λ that produces the lowest average normalized mean squared error (NMSE) across folds is selected. Common search strategies include grid search over a logarithmic scale (e.g., 10⁻⁶ to 10²) and more efficient L-curve curvature analysis.

While ridge regression (L2 penalty) shrinks coefficients toward zero without eliminating them, the regularization parameter in LASSO regression (L1 penalty) forces some coefficients to exactly zero. This is valuable for Volterra kernel pruning, where the regularization path identifies which cross-terms contribute meaningfully to linearization. The λ value directly controls the sparsity level: higher values produce fewer non-zero coefficients and a more computationally efficient predistorter.

In recursive least squares (RLS) with exponential forgetting, the regularization parameter initializes the inverse correlation matrix as λ⁻¹I. This prevents the algorithm from producing unbounded coefficient updates during the first few samples when the matrix is nearly singular. Some adaptive DPD architectures employ a time-varying λ that decreases as more data is observed, transitioning from a heavily regularized startup phase to a more agile tracking phase.

The required regularization strength is inversely related to the signal-to-noise ratio (SNR) of the measurement system. When extracting PA models from noisy vector network analyzer data, a larger λ is necessary to prevent the model from fitting the noise floor. Conversely, high-SNR benchtop measurements with calibrated equipment permit smaller λ values, allowing the model to capture subtle nonlinear behaviors without over-penalizing coefficient magnitudes.