Stochastic Gradient Descent (SGD)

Unlike batch gradient descent, which computes the gradient over the entire dataset, SGD estimates the true gradient using a single sample or a mini-batch. This introduces noise into the update direction, which can help escape shallow local minima but causes the parameter trajectory to oscillate around the optimum. The noisy gradient acts as an implicit regularizer, often improving generalization in non-convex optimization landscapes like deep neural networks.

SGD processes data samples sequentially, making it inherently suited for online training where data arrives in a continuous stream. This is critical for adaptive digital predistortion systems where power amplifier characteristics drift due to temperature changes, aging, or varying operating conditions. The algorithm can track time-varying parameters without requiring a complete retraining cycle or storing the entire history of transmitted signals.

The step size or learning rate (η) critically governs convergence behavior:

• Too large: Divergence or wild oscillations around the minimum
• Too small: Impractically slow convergence
• Scheduling: Decaying the learning rate over time (e.g., η_t = η_0 / (1 + t)) satisfies the Robbins-Monro conditions for guaranteed convergence in convex settings. In practice, adaptive methods like AdaGrad or Adam dynamically adjust per-parameter learning rates.

SGD achieves O(d) time and memory complexity per update, where d is the number of model parameters. This constant scaling is independent of dataset size N, unlike batch methods requiring O(Nd) per iteration. For large-scale coefficient estimation in memory polynomial predistorters with hundreds of parameters, SGD enables real-time adaptation on resource-constrained FPGA or embedded DSP platforms where matrix inversion is prohibitively expensive.

The stochastic nature creates a fundamental tradeoff:

• Convergence rate: Larger step sizes accelerate initial convergence toward the Wiener optimum
• Misadjustment: The steady-state mean squared error (MSE) exceeds the minimum MSE by an amount proportional to the step size and gradient noise variance This excess error, known as misadjustment, is the price paid for computational efficiency and tracking capability. In DPD applications, this manifests as residual nonlinearity in the linearized output spectrum.

Modern implementations use mini-batches—small random subsets of data—to balance gradient accuracy and computational cost:

• Variance reduction: Averaging over m samples reduces gradient noise variance by a factor of 1/m
• Hardware utilization: Mini-batches exploit vectorized operations and GPU parallelism
• Typical sizes: 32, 64, or 128 samples per batch In DPD coefficient estimation, mini-batch SGD processes blocks of I/Q samples, providing a smoother convergence trajectory than single-sample updates while maintaining online adaptability.

A foundational stochastic gradient descent algorithm that updates filter coefficients iteratively based on the instantaneous estimate of the mean squared error gradient. Unlike batch methods, LMS processes one sample at a time, making it suitable for real-time adaptation in power amplifier linearization.

• Update Rule: w(n+1) = w(n) + μ·e(n)·x(n)
• Complexity: O(N) per iteration, where N is the number of coefficients
• Convergence: Governed by the step size μ and the eigenvalue spread of the input correlation matrix
• Misadjustment: Excess MSE beyond the Wiener optimum, proportional to step size

A variant of the LMS algorithm that normalizes the step size by the instantaneous power of the input signal vector. This normalization improves convergence stability when signal levels fluctuate, a common scenario in wideband communication systems with varying envelope signals.

• Update Rule: w(n+1) = w(n) + (μ / ||x(n)||² + ε)·e(n)·x(n)
• Benefit: Convergence rate becomes independent of input signal scaling
• Regularization: Small constant ε prevents division by zero during silent periods
• Application: Standard choice for adaptive DPD when computational resources are constrained

An adaptive algorithm that recursively updates the inverse of the input correlation matrix to achieve faster convergence than gradient-based methods like LMS. RLS converges in approximately 2N iterations regardless of eigenvalue spread, but at the cost of O(N²) computational complexity.

• Key Mechanism: Maintains and updates the inverse autocorrelation matrix P(n)
• Forgetting Factor: λ weights recent data exponentially to track time-varying systems
• Tradeoff: Superior convergence speed versus significantly higher computational load
• Stability: Numerically sensitive; QR-RLS variants use Givens rotations for robustness

A postdistorter identification method that trains the predistorter by placing a copy of it after the power amplifier in the feedback path. The algorithm minimizes the error between the postdistorter output and the predistorter input, avoiding the need for a PA inverse model.

• Training Path: PA output → Postdistorter → Compare with desired input
• Assumption: Postdistorter and predistorter are interchangeable (commutative)
• Advantage: Formulates coefficient estimation as a standard system identification problem
• Limitation: Assumes the PA characteristic is invertible and the commutativity holds

A closed-loop estimation method that identifies predistorter parameters directly by minimizing the error between the desired linear output and the actual power amplifier output. DLA does not assume commutativity, making it more robust for strongly nonlinear systems.

• Error Signal: e(n) = y_desired(n) - y_PA(n)
• Gradient Computation: Requires backpropagation through the PA model or measured response
• Advantage: Directly optimizes the linearization objective without commutativity assumptions
• Challenge: Requires accurate PA behavioral model or real-time output measurement

A measure of how quickly an adaptive algorithm approaches the optimal solution, typically defined by the number of iterations required for the error to reach a steady-state value. In DPD applications, convergence rate determines how fast the system can track changing amplifier characteristics due to temperature drift or aging.

• LMS: Convergence depends on eigenvalue spread of input correlation matrix
• RLS: Converges in approximately 2N iterations, independent of eigenvalue spread
• SGD: Mini-batch size trades off convergence speed against gradient noise
• Practical Impact: Faster convergence enables tracking of thermal memory effects