Regularization

Regularization directly combats overfitting by penalizing large coefficient values. In PA modeling, a model with unconstrained coefficients may perfectly fit measurement noise rather than the underlying amplifier physics. This results in excellent Normalized Mean Square Error (NMSE) on training data but poor generalization to new signals. Regularization techniques like L2 (Ridge) penalize the squared magnitude of coefficients, shrinking them toward zero and forcing the model to learn only the most significant nonlinear dynamics.

The Least Squares (LS) estimation problem in PA model extraction often involves ill-conditioned data matrices, especially when modeling strong memory effects or using high-order polynomial terms. The condition number of the regression matrix can become extremely large, making the solution hypersensitive to small perturbations in measurement data. Regularization, particularly Tikhonov regularization, adds a diagonal loading term that improves the condition number, ensuring stable coefficient extraction even with finite-precision arithmetic on embedded processors.

L1 (Lasso) regularization adds a penalty proportional to the absolute value of coefficients. This drives many coefficients exactly to zero, producing a sparse model. For Generalized Memory Polynomial (GMP) or Volterra series models with thousands of terms, L1 regularization automatically performs pruning—identifying and eliminating redundant basis functions. The resulting sparse model requires fewer multiply-accumulate operations in FPGA-based Digital Predistortion (DPD) implementations, reducing hardware resource utilization without sacrificing linearization performance.

Regularization explicitly manages the bias-variance tradeoff in behavioral modeling:

• High regularization: Increases bias (simpler model, may underfit true nonlinearity)
• Low regularization: Increases variance (complex model, fits noise) The optimal regularization parameter λ is typically selected via cross-validation, where the dataset is partitioned into training and validation subsets. The λ that minimizes validation error provides the best balance between model fidelity and generalization for unseen modulation schemes.

In online training scenarios for adaptive DPD, regularization prevents coefficient drift during continuous operation. When using gradient-based algorithms like Least Mean Squares (LMS) or Recursive Least Squares (RLS), unregularized updates can cause coefficients to grow unboundedly in response to transient signal conditions or measurement noise. A regularization term acts as a forgetting mechanism that pulls coefficients back toward a stable operating point, ensuring reliable long-term operation in deployed base station equipment.

Elastic Net combines both L1 and L2 penalties to leverage the advantages of each:

• L1 component: Promotes sparsity by zeroing out irrelevant terms
• L2 component: Stabilizes the solution when predictors are highly correlated This is particularly valuable in memory polynomial models where delayed terms at similar tap positions exhibit high correlation. Elastic Net prevents the arbitrary selection of one correlated term over another that can occur with pure Lasso, while still producing a compact model suitable for hardware implementation.