Normal Equation

The Normal Equation solves for optimal coefficients θ̂ in a single step using the formula θ̂ = (XᵀX)⁻¹ Xᵀ y. Unlike gradient descent methods, there is no iteration, no learning rate to tune, and no convergence monitoring required. The solution is deterministic: given the same design matrix X and target vector y, the result is always identical. This makes it ideal for offline coefficient extraction where computational determinism is critical.

The computational cost is dominated by the matrix inversion of XᵀX, which is an (n × n) matrix where n is the number of model coefficients. Standard inversion requires O(n³) operations. For a memory polynomial predistorter with 50 coefficients, this is trivial. However, for models with thousands of parameters, the cubic scaling becomes prohibitive, making iterative methods like Stochastic Gradient Descent or Recursive Least Squares more practical.

The Normal Equation explicitly computes (XᵀX)⁻¹, which squares the condition number of the original design matrix X. If X is nearly rank-deficient—common with highly correlated basis functions in Volterra series—the inversion becomes numerically unstable. In such cases, more robust factorization methods are preferred:

• QR Decomposition: Solves Rx = Qᵀy without forming XᵀX
• Singular Value Decomposition (SVD) : Handles rank deficiency by discarding near-zero singular values

When the noise in the observed data is zero-mean, uncorrelated (white), and has constant variance (homoscedastic), the Normal Equation yields the Best Linear Unbiased Estimator (BLUE) . This is the Gauss-Markov theorem. In power amplifier modeling, measurement noise from vector signal analyzers typically satisfies these conditions, making the Normal Equation the statistically optimal batch estimator for extracting Memory Polynomial coefficients.

Unlike iterative algorithms, the Normal Equation has no step size (μ), no forgetting factor (λ), and no momentum parameter. This eliminates the need for hyperparameter sweeps and convergence analysis. The trade-off is the inability to track time-varying systems. For static PA behavioral models extracted in a controlled lab environment, this zero-tuning property significantly reduces engineering time compared to tuning an LMS or NLMS algorithm.

When XᵀX is ill-conditioned, a regularization parameter (δ) is added to the diagonal: θ̂ = (XᵀX + δI)⁻¹ Xᵀ y. This is known as ridge regression or Tikhonov regularization. The parameter δ introduces a small bias but dramatically reduces variance, preventing the coefficient vector from exploding due to near-singularity. This is essential when extracting high-order polynomial models where basis functions become highly correlated.