Stochastic Gradient Descent (SGD)

Unlike batch gradient descent which computes the gradient over the entire dataset, SGD estimates the gradient using a single sample or mini-batch. This introduces gradient noise that helps escape shallow local minima in the non-convex loss landscapes typical of DPD coefficient estimation.

• Single-sample update: Computes gradient from one I/Q sample pair
• Mini-batch mode: Averages gradient over a small block of samples
• Gradient noise: Acts as implicit regularization, preventing overfitting to measurement artifacts

SGD exhibits a characteristic noisy convergence trajectory toward the optimal predistorter coefficients. The convergence rate is governed by the learning rate and the condition number of the data covariance matrix.

• Transient phase: Rapid initial reduction in NMSE as coefficients move toward the Wiener solution
• Steady-state misadjustment: Excess error beyond the theoretical minimum caused by gradient noise
• Convergence rate: Scales inversely with the eigenvalue spread of the input correlation matrix

The step size or learning rate critically controls the stability and speed of SGD. In DPD applications, adaptive scheduling strategies are essential to balance fast initial convergence with low steady-state misadjustment.

• Constant rate: Simple but requires careful tuning to avoid divergence
• Step decay: Reduces learning rate at predetermined intervals
• Exponential decay: Smoothly decreases rate over time
• Momentum: Accumulates a velocity vector to dampen oscillations in narrow valleys of the cost function

The Least Mean Squares (LMS) algorithm is a specific instantiation of SGD applied to adaptive filtering. LMS uses the instantaneous squared error as the cost function, making it computationally lightweight and suitable for sample-by-sample DPD coefficient updates.

• LMS update rule: w(n+1) = w(n) + μ·e(n)·x(n)
• Computational cost: O(N) per iteration where N is the number of coefficients
• Stability condition: Learning rate μ must satisfy 0 < μ < 2/λ_max where λ_max is the largest eigenvalue of the input autocorrelation matrix

Mini-batch SGD balances the variance of gradient estimates against computational throughput. Larger batches provide more accurate gradient estimates but reduce update frequency, creating a fundamental tradeoff in real-time DPD systems.

• Small batches: High gradient noise helps escape saddle points but slows convergence
• Large batches: Lower variance enables larger learning rates but risks sharp minima that generalize poorly
• Batch size selection: Typically tuned based on available hardware parallelism and latency constraints

Several SGD variants address specific challenges in DPD coefficient estimation, including ill-conditioning and non-stationary PA behavior:

• Normalized LMS (NLMS): Normalizes the step size by the input signal power to improve stability with varying signal levels
• Affine Projection Algorithm (APA): Uses multiple past input vectors to accelerate convergence for correlated signals
• Adam-style optimizers: Adaptive moment estimation adjusts per-parameter learning rates, useful for neural network-based DPD

The simplest stochastic gradient descent variant used in adaptive DPD. LMS updates coefficients using the instantaneous estimate of the mean squared error gradient, requiring only O(N) complexity per iteration.

• Extremely low computational cost for FPGA implementation
• No matrix inversion required
• Convergence rate depends on eigenvalue spread of input covariance
• Step size μ trades off convergence speed vs. misadjustment

The speed at which an SGD-based DPD algorithm approaches the Wiener optimum. In PA linearization, convergence must occur within the guard interval or training preamble to avoid spectral mask violations.

• Determined by learning rate and input signal conditioning
• Ill-conditioned data slows convergence dramatically
• Normalized LMS (NLMS) improves rate by scaling step size with input power
• RLS offers order-of-magnitude faster convergence at higher compute cost

The excess mean squared error beyond the theoretical minimum (MMSE) caused by gradient noise in stochastic updates. In DPD systems, misadjustment directly degrades EVM and ACPR performance.

• Proportional to step size μ and number of coefficients
• Fundamental trade-off: faster convergence → higher misadjustment
• Momentum and adaptive learning rate schedules reduce steady-state noise
• Critical specification for 1024-QAM and higher-order modulation schemes

The random perturbation in coefficient updates caused by using single-sample or mini-batch gradient estimates instead of the true gradient. This noise prevents convergence to the exact optimum but enables tracking of time-varying PA nonlinearities.

• Acts as implicit regularization against overfitting
• Enables tracking of thermal memory effects and aging
• Variance reduction techniques (SVRG, SAGA) improve steady-state accuracy
• Mini-batch SGD balances noise reduction with update latency

Extensions to vanilla SGD that accelerate convergence in ill-conditioned optimization landscapes common in wideband DPD coefficient estimation.

• Momentum: Accumulates a velocity vector to damp oscillations in narrow valleys
• Adam: Adaptive per-parameter learning rates using first and second moment estimates
• RMSprop: Normalizes gradients by a running average of recent magnitudes
• Particularly beneficial for deep neural network-based DPD architectures