Early Stopping

Early stopping works by monitoring a validation metric—typically the normalized mean squared error (NMSE) on a held-out dataset—during the iterative training process. As optimization proceeds, the training error decreases monotonically. However, the validation error follows a U-shaped curve: it initially decreases, reaches a minimum, and then begins to rise as the model starts memorizing noise rather than learning the underlying system dynamics. The algorithm terminates training at the point of minimum validation error.

• Patience parameter: Number of epochs to continue after validation error stops improving
• Snapshot restoration: Best model weights are saved and restored at termination
• No additional hyperparameters: Unlike L2 regularization, no penalty coefficient tuning is required

When training neural network-based predistorters or iterative coefficient solvers, the optimization path itself provides a form of capacity control. Early iterations learn the dominant, low-frequency components of the power amplifier's nonlinear response—the signal in the data. Later iterations begin fitting high-frequency noise components and measurement artifacts that do not generalize.

• Spectral bias: Gradient descent naturally fits low-frequency components first
• Effective model complexity increases with each iteration
• Equivalent to constraining the parameter norm without explicitly computing it
• Particularly effective for ill-conditioned estimation problems common in wideband DPD

The effectiveness of early stopping depends critically on the validation set construction. For power amplifier behavioral modeling, the validation data must represent the operational signal conditions the predistorter will encounter in deployment. A poorly chosen validation set leads to premature or delayed stopping.

• Hold-out ratio: Typically 15-20% of captured I/Q data reserved for validation
• Signal diversity: Validation set should span expected modulation schemes, bandwidths, and power levels
• Temporal separation: Avoid temporal correlation between training and validation samples
• Cross-validation: K-fold splitting provides more robust stopping point estimates at increased computational cost

Early stopping offers distinct advantages over Tikhonov regularization (L2 penalty) and other explicit methods in adaptive DPD contexts. It requires no modification to the objective function and automatically determines the effective regularization strength from the data.

• No ridge parameter tuning: Avoids grid search over the regularization parameter λ
• Computational efficiency: Single training run vs. multiple runs for hyperparameter selection
• Online adaptation: Naturally integrates with iterative learning control (ILC) and recursive algorithms
• Limitation: Requires sufficient validation data; less effective with very small datasets where explicit regularization with a carefully chosen parameter may outperform

In digital predistortion systems, early stopping is applied during offline training of neural network predistorters and during iterative coefficient refinement in indirect learning architectures. The technique prevents the predistorter from overfitting to specific power amplifier operating points.

• Neural network DPD: Stop training when adjacent channel leakage ratio (ACLR) improvement on validation data plateaus
• Iterative learning control: Halt ILC iterations before error signal begins fitting measurement noise
• Memory polynomial extraction: Apply when using gradient-based solvers on ill-conditioned regression matrices
• Practical threshold: Stop when validation NMSE improvement is less than 0.1 dB over the patience window

Early stopping directly navigates the bias-variance tradeoff by controlling the effective number of optimization iterations. Too few iterations yield high bias—the model underfits and fails to capture the PA's nonlinear memory effects. Too many iterations yield high variance—the model overfits and produces erratic predistortion that degrades ACLR on new signals.

• Optimal stopping point balances systematic error (bias) against sensitivity to training data noise (variance)
• Double descent phenomenon: In overparameterized neural DPD models, validation error may decrease again after the initial peak—early stopping captures the first minimum
• Connection to condition number: Ill-conditioned estimation problems (high condition number) benefit most from early stopping due to amplified noise sensitivity

A modeling failure where extracted predistorter coefficients fit the training data noise rather than the underlying power amplifier nonlinearity. When an iterative algorithm runs too long, the model begins to memorize measurement artifacts and thermal noise transients, resulting in poor Adjacent Channel Leakage Ratio (ACLR) performance on new signal waveforms. Early stopping directly mitigates this by halting optimization before the error surface minima trap the solution in a high-variance state.

The fundamental tension between a model's ability to fit training data accurately (low bias) and its ability to generalize to unseen data (low variance). In Digital Predistortion (DPD), a model with too many iterations achieves low bias on the captured I/Q samples but high variance when the carrier frequency or temperature shifts. Early stopping acts as an explicit hyperparameter to find the Pareto-optimal point on the bias-variance curve, minimizing the Normalized Mean Squared Error (NMSE) on validation data.

A scalar value added to the diagonal of the autocorrelation matrix to improve numerical stability and prevent overfitting. While Tikhonov regularization explicitly penalizes large coefficient magnitudes, early stopping achieves a similar effect implicitly by restricting the effective number of iterations in algorithms like Least Mean Squares (LMS) or Recursive Least Squares (RLS). This is particularly critical when the condition number of the input signal matrix is high, indicating an ill-posed estimation problem.

The standard implementation of early stopping requires monitoring a validation metric on a held-out data set distinct from the training data. In DPD coefficient estimation, the algorithm tracks the Error Vector Magnitude (EVM) or Mean Squared Error (MSE) on a separate capture of the power amplifier output. Training halts when the validation error begins to increase consistently over a patience window, even as the training error continues to decrease. This signals the onset of noise fitting.

A measure of how quickly an adaptive algorithm approaches the Wiener-Hopf optimal solution. Algorithms with fast convergence, such as QR-RLS, may require aggressive early stopping because they rapidly descend into noise-fitting regimes. Conversely, slow-converging algorithms like standard LMS may naturally stall before overfitting. The interaction between the step size and the early stopping patience must be carefully tuned to balance tracking speed against generalization in time-varying Doherty amplifier linearization.

The normalized difference between the steady-state Mean Squared Error (MSE) of an adaptive filter and the minimum MSE achievable by the optimal Wiener filter. In online DPD training, misadjustment represents the excess error caused by gradient noise in algorithms like Normalized LMS (NLMS). Early stopping can be used to halt adaptation before the misadjustment noise floor dominates, effectively trading a small amount of bias for a significant reduction in steady-state variance.