Cost Function

The fundamental cost function in DPD training that computes the average squared difference between the desired input signal and the actual PA output.

Key characteristics:

• Penalizes large errors quadratically, making it sensitive to outliers and peak distortion
• Provides a convex optimization surface for linear-in-parameters models like memory polynomials
• Directly relates to Normalized MSE (NMSE) , the standard DPD performance metric

Example: For a 100 MHz 5G NR signal, minimizing MSE typically achieves NMSE values below -40 dB after convergence.

Cost functions that apply frequency-dependent weighting to prioritize linearization accuracy in specific spectral regions.

Common weighting strategies:

• In-band emphasis: Higher weights on in-band frequencies to minimize EVM degradation
• Out-of-band emphasis: Higher weights on adjacent channels to meet stringent ACPR specifications
• Frequency-domain weighting: Apply FFT to error signal and weight bins according to spectral mask requirements

Application: Critical for 5G NR systems where 3GPP specifies different ACLR limits for adjacent and alternate channels, requiring targeted optimization.

Augmented objective functions that add penalty terms to prevent overfitting and improve numerical stability during coefficient estimation.

Regularization types:

• L2 (Ridge): Adds λ||w||² to penalize large coefficient magnitudes, preventing coefficient drift
• L1 (Lasso): Adds λ||w||₁ to promote sparsity, useful for reducing LUT entries
• Elastic Net: Combines L1 and L2 penalties for both sparsity and stability

Practical impact: Without regularization, DPD coefficients can drift by 3-5 dB in NMSE performance over temperature cycles of -40°C to +85°C.

An iterative optimization approach that updates DPD coefficients using instantaneous gradient estimates rather than batch processing.

Algorithm characteristics:

• Update rule: w(n+1) = w(n) - μ∇J(w(n)) where μ is the learning rate
• Converges to Wiener solution for MSE cost functions when μ is properly chosen
• Trade-off between convergence rate (larger μ) and misadjustment (steady-state error)

Implementation: LMS algorithm is the most common SGD variant in DPD, requiring only O(N) operations per sample versus O(N³) for LS solutions.

Cost functions designed to specifically penalize distortion at high signal amplitudes where PA compression is most severe.

Advanced formulations:

• Huber loss: Quadratic for small errors, linear for large errors—robust to outliers
• Log-cosh loss: Smooth approximation of absolute error, less sensitive to measurement noise
• Custom weighting: Apply amplitude-dependent weights emphasizing the PAPR region (top 1% of samples)

Benefit: Reduces peak EVM by 2-3 dB compared to uniform MSE minimization, critical for high-order QAM constellations (256-QAM, 1024-QAM).

The most common cost function in DPD training, defined as the average of the squared differences between the desired signal (x(n)) and the PA output (y(n)).

• Formula: (J = \frac{1}{N} \sum_{n=1}^{N} |x(n) - y(n)|^2)
• Properties: Convex, differentiable, and heavily penalizes large errors due to squaring
• Use case: Default choice for offline coefficient estimation using least squares or gradient descent
• Limitation: Sensitive to outliers in the training data, which can bias the predistorter model

A normalized variant of MSE that expresses the linearization error relative to the input signal power, enabling fair comparison across different signal levels and test conditions.

• Formula: (\text{NMSE} = 10 \log_{10} \left( \frac{\sum |x(n) - y(n)|^2}{\sum |x(n)|^2} \right)) dB
• Typical targets: -35 dB to -45 dB for high-quality linearization
• Advantage: Provides a scale-invariant metric that is independent of absolute signal amplitude
• Industry standard: Widely reported in academic literature and DPD benchmark comparisons

An extension of the least squares cost function that assigns different importance weights to different signal regions, prioritizing linearization accuracy where it matters most.

• High-power weighting: Emphasizes accuracy near PA compression where nonlinearity is strongest
• Spectral weighting: Applies frequency-domain weights to suppress spectral regrowth in specific adjacent channels
• Implementation: (J = \sum w_n |x(n) - y(n)|^2) where (w_n) is the sample weight
• Benefit: Allows engineers to trade off in-band EVM against out-of-band ACPR performance

An additional term added to the cost function to prevent overfitting and improve numerical stability during coefficient estimation.

• Tikhonov (L2) regularization: (J = \text{MSE} + \lambda |\mathbf{w}|^2_2), penalizes large coefficient magnitudes
• Lasso (L1) regularization: (J = \text{MSE} + \lambda |\mathbf{w}|_1), promotes sparse predistorter models
• Role: Controls the condition number of the estimation matrix, preventing ill-conditioned solutions
• Parameter (\lambda): The regularization hyperparameter balancing model fit against model complexity

A frequency-domain cost function that directly minimizes spectral regrowth by penalizing error power in adjacent channels rather than across the entire bandwidth.

• Motivation: Standard MSE optimizes in-band error but does not explicitly target ACPR compliance
• Formulation: (J = \sum_{k \in \text{adj}} |E(f_k)|^2) where (E(f)) is the error spectrum
• Application: Used when strict regulatory spectral masks must be met
• Trade-off: May increase in-band EVM while improving out-of-band suppression

The iterative optimization algorithm that minimizes the cost function by updating predistorter coefficients in the direction of the negative gradient, computed on small batches of samples.

• Update rule: (\mathbf{w}_{k+1} = \mathbf{w}_k - \mu \nabla J(\mathbf{w}_k))
• Learning rate (\mu): Controls step size; too large causes instability, too small slows convergence rate
• Online variant: Enables sample-by-sample update for real-time adaptive DPD
• Relationship to LMS: The Least Mean Squares algorithm is a special case of SGD applied to the MSE cost function