Regularization Parameter

Adds a penalty proportional to the squared L2-norm of the coefficient vector to the least squares cost function. This shrinks coefficients toward zero, reducing variance at the cost of introducing some bias. The modified normal equation becomes (XᵀX + λI)β = Xᵀy, where λI is the regularization term added to the diagonal of the correlation matrix.

• Directly addresses ill-conditioned correlation matrices by ensuring positive definiteness
• Equivalent to imposing a Gaussian prior on coefficients in a Bayesian framework
• Larger λ values produce smoother predistorter responses with reduced sensitivity to measurement noise

When the input correlation matrix XᵀX is near-singular (high condition number), small perturbations in measurement data cause large swings in estimated coefficients. Adding λ to the diagonal elements increases all eigenvalues by λ, dramatically reducing the condition number and stabilizing the matrix inversion.

• Transforms an ill-posed problem into a well-posed one
• Critical for wideband signals where adjacent samples are highly correlated
• Prevents catastrophic amplification of quantization noise during fixed-point implementation

Without regularization, high-order polynomial predistorters can fit measurement noise rather than the true amplifier nonlinearity. This manifests as erratic coefficient values and degraded linearization performance on signals not seen during training. The regularization parameter penalizes large coefficient magnitudes, enforcing a smoothness constraint on the predistorter transfer function.

• Prevents the model from learning spurious patterns unique to the training capture
• Improves Adjacent Channel Leakage Ratio (ACLR) on validation signals
• Essential when the number of model parameters approaches the number of training samples

The optimal λ is typically determined through k-fold cross-validation or generalized cross-validation (GCV). The training data is partitioned; the model is trained on subsets with varying λ values and evaluated on held-out partitions. The λ minimizing validation error is selected.

• L-curve criterion: Plots solution norm vs. residual norm to identify the corner of maximum curvature
• Discrepancy principle: Chooses λ such that residual norm matches known noise variance
• In adaptive DPD systems, λ may be scheduled—starting large for initial acquisition, then reduced for fine tracking

In nonlinear least squares problems (e.g., direct learning architecture for DPD), the regularization parameter appears as a damping factor in the Levenberg-Marquardt algorithm. It interpolates between Gauss-Newton (λ → 0) and gradient descent (λ → ∞) behavior.

• Large λ when far from optimum: ensures descent direction and prevents divergence
• Small λ near convergence: exploits quadratic convergence of Gauss-Newton
• Adaptive damping strategies multiply or divide λ based on whether an iteration reduces the cost function

In Recursive Least Squares (RLS) algorithms, the forgetting factor (0 < μ ≤ 1) plays a role analogous to 1/λ in batch regularization. A smaller forgetting factor increases the effective regularization by weighting recent data more heavily, preventing the inverse correlation matrix from growing unbounded in non-stationary environments.

• μ = 1: infinite memory, no regularization, suitable for time-invariant systems
• μ < 1: exponential window, implicit regularization, enables tracking of thermal memory effects
• The effective regularization is approximately (1-μ)R, where R is the input correlation matrix

The condition number is the ratio of the largest to smallest singular value of the correlation matrix. It quantifies how sensitive the solution of a linear system is to small perturbations in the input data. A high condition number indicates an ill-conditioned matrix where the regularization parameter becomes critical for numerical stability. In DPD coefficient estimation, matrices with condition numbers exceeding 10^6 are common, making unregularized solutions wildly oscillatory and useless for practical predistortion.

The bias-variance tradeoff is the fundamental tension governing regularization. Increasing the regularization parameter introduces bias by shrinking coefficient magnitudes away from the least-squares optimum, but simultaneously reduces variance by preventing the model from fitting noise. The optimal regularization parameter minimizes the mean squared error, which equals bias² + variance + irreducible error. In power amplifier modeling, excessive variance manifests as predistorter coefficients that change erratically with minor temperature fluctuations.

Overfitting occurs when the extracted model parameters fit the training data noise rather than the underlying amplifier dynamics. This results in excellent performance on the captured dataset but poor generalization to new signals. Key indicators include:

• Excessively large coefficient magnitudes
• Oscillatory predistorter transfer functions
• Degraded ACLR on validation signals Regularization directly combats overfitting by penalizing the L2 norm of the coefficient vector, forcing the solution toward smoother, more physically plausible predistorter characteristics.

Tikhonov regularization, also known as ridge regression, is the mathematical framework that formalizes adding a penalty term to the least-squares cost function. The augmented cost becomes ||Ax - b||² + λ||x||², where λ is the regularization parameter. This is equivalent to solving (A^T A + λI)x = A^T b, which adds the scalar λ to the diagonal of the correlation matrix. The technique was developed by Andrey Tikhonov in the 1940s to solve ill-posed inverse problems and remains the standard approach in DPD coefficient estimation.

The normal equation is the closed-form solution to the linear least-squares problem: x = (A^T A)^(-1) A^T b. When the regularization parameter is incorporated, it becomes x = (A^T A + λI)^(-1) A^T b. The addition of λI ensures the matrix is invertible even when A^T A is rank-deficient. In practical DPD implementations, solving the regularized normal equation via Cholesky decomposition provides a computationally efficient batch estimation method, though iterative approaches like RLS are preferred for online adaptation.

Cross-validation is the standard technique for selecting the optimal regularization parameter. The process involves:

• Partitioning captured PA data into training and validation sets
• Solving for coefficients using a range of λ values on the training set
• Evaluating NMSE or ACLR on the held-out validation set
• Selecting the λ that minimizes validation error The L-curve criterion offers an alternative by plotting solution norm versus residual norm and identifying the point of maximum curvature, which balances regularization and data fidelity.