Ill-Conditioning

The condition number κ of the correlation matrix R quantifies ill-conditioning. It is defined as the ratio of the largest singular value to the smallest: κ = σ_max / σ_min. A matrix is considered ill-conditioned when κ exceeds 10^6 to 10^12 in double-precision arithmetic.

• κ < 10^3: Well-conditioned, stable inversion
• 10^3 < κ < 10^6: Moderately conditioned, monitor for precision loss
• κ > 10^6: Ill-conditioned, solutions unreliable without regularization

The condition number directly bounds the relative error magnification: small perturbations in the observation vector b are amplified by a factor of κ in the solution x.

The correlation matrix R = Φ^H Φ becomes nearly singular when its columns—the basis function outputs—are highly correlated. This occurs when:

• Redundant basis functions: Multiple polynomial terms capture the same nonlinear behavior, creating linear dependencies
• Narrowband or constant-envelope signals: The input signal lacks sufficient spectral richness to excite all basis dimensions independently
• Insufficient sample diversity: The observation window is too short relative to the model's degrees of freedom

A near-singular R has eigenvalues approaching zero, making its inverse R⁻¹ explode with large, oscillating entries that amplify noise.

In an ill-conditioned system, coefficient estimates become hypersensitive to minuscule changes in the measured data. A perturbation as small as 1 LSB in the ADC feedback sample or a 0.01 dB change in signal power can cause the estimated DPD coefficients to swing wildly.

• Symptom: Coefficients vary erratically between adaptation cycles even when the PA's physical state is unchanged
• Root cause: The solution vector x lies in a direction nearly orthogonal to the dominant singular vectors, so small projection errors map to large coefficient errors
• Practical impact: The predistorter's transfer function becomes non-repeatable, defeating the purpose of adaptive linearization

Ill-conditioning acts as a noise amplifier in the coefficient estimation pipeline. Measurement noise in the feedback receiver—thermal noise, quantization error, phase noise—is magnified by the inversion of small eigenvalues.

• Each eigenvalue λ_i of R contributes a noise amplification factor of 1/λ_i
• The smallest eigenvalues dominate the noise floor of the coefficient estimate
• Result: The estimated predistorter coefficients contain a large random component that does not represent the true PA inverse, degrading ACLR rather than improving it

This is why direct matrix inversion without regularization often produces coefficients that perform worse than no predistortion at all.

Adaptive algorithms exhibit pathological convergence behavior when operating on ill-conditioned cost surfaces. The error contour becomes a highly elongated valley rather than a symmetric bowl.

• LMS/NLMS: The convergence rate is bounded by λ_min/λ_max. As this ratio approaches zero, convergence stalls or the algorithm diverges entirely
• RLS: While theoretically immune to eigenvalue spread, finite-precision RLS suffers from covariance matrix blow-up when R is ill-conditioned
• SGD: The gradient direction oscillates along the steep eigen-directions while making negligible progress along flat ones, requiring impractically small learning rates

Monitoring the error signal power over adaptation cycles reveals whether the system is converging or trapped in an ill-conditioned stall.

A hallmark of ill-conditioned least-squares solutions is an excessively large coefficient vector norm ||x||. The estimator compensates for near-collinearity in the basis functions by assigning large, opposing-magnitude coefficients that cancel each other out.

• Indicator: Coefficient magnitudes exceed physically meaningful bounds for the PA's nonlinear order
• Mechanism: The solution exploits numerical cancellation between nearly dependent basis functions to fit noise rather than signal
• Hardware consequence: Large coefficients demand extended bit-widths in FPGA implementations, increasing resource utilization and latency

This is the mathematical manifestation of overfitting in the linear least-squares context, and it is the primary motivation for applying Tikhonov regularization (ridge regression) to penalize large ||x||.

A scalar metric defined as the ratio of the largest to smallest singular value of the correlation matrix. A high condition number (>> 1) indicates ill-conditioning, meaning the matrix is nearly singular and small input perturbations or rounding errors cause large swings in the estimated DPD coefficients. It directly quantifies the worst-case error amplification in solving the normal equations.

A scalar value (often denoted λ) added to the diagonal of the correlation matrix before inversion. This technique, known as Tikhonov regularization or ridge regression, artificially improves the condition number by preventing eigenvalues from approaching zero. It introduces a small bias to drastically reduce variance in the coefficient estimates, stabilizing the adaptive filter against ill-conditioning caused by highly correlated basis functions.

A numerically stable matrix factorization that decomposes the data matrix into an orthogonal matrix Q and an upper triangular matrix R. Solving the least-squares problem using QR decomposition avoids explicitly forming and inverting the ill-conditioned correlation matrix (AᵀA), making it the preferred method for block-based coefficient estimation in fixed-point hardware where numerical precision is limited.

The primary cause of ill-conditioning in digital predistortion. When the nonlinear basis functions (e.g., polynomial terms x|x|ⁿ) are highly correlated with each other or with their delayed copies, the columns of the data matrix become nearly linearly dependent. This multicollinearity inflates the condition number, making it impossible for the estimator to distinguish the contribution of individual terms. Orthogonal basis functions (e.g., Chebyshev polynomials) are often used to mitigate this.

The robustness of an algorithm to finite-precision arithmetic and rounding errors. An algorithm is numerically unstable if small floating-point errors accumulate and overwhelm the true solution. In the context of ill-conditioning, direct matrix inversion is notoriously unstable. Stable alternatives include:

• QR decomposition
• Singular Value Decomposition (SVD)
• Iterative methods with early stopping

A weighting parameter (λ, 0 << λ ≤ 1) in Recursive Least Squares (RLS) that exponentially discounts older data. While essential for tracking time-varying PA nonlinearity, a small forgetting factor effectively reduces the number of effective samples, which can exacerbate ill-conditioning if the input signal is not persistently exciting. Balancing tracking speed against numerical stability is a critical design trade-off in online training algorithms.