Inferensys

Glossary

Recursive Least Squares (RLS)

Recursive Least Squares (RLS) is an adaptive filtering algorithm that recursively finds the coefficients minimizing a weighted linear least squares cost function, offering faster convergence than LMS.
Developer reviewing LLM cost optimization spreadsheet on laptop, calculator and coffee on desk, casual finance-technical moment.
ADAPTIVE FILTERING

What is Recursive Least Squares (RLS)?

An adaptive algorithm that recursively finds the filter coefficients minimizing a weighted linear least squares cost function, offering significantly faster convergence than gradient-based methods like LMS.

Recursive Least Squares (RLS) is an adaptive filtering algorithm that recursively computes the filter coefficients minimizing a weighted linear least squares cost function. Unlike the Least Mean Squares (LMS) algorithm, which relies on stochastic gradient descent, RLS utilizes all available input data to achieve an exact least squares solution at each iteration, resulting in an order-of-magnitude faster convergence rate.

The algorithm maintains and updates an inverse correlation matrix of the input signal, making it computationally intensive—complexity scales with O(n²)—but highly effective for tracking time-varying systems. In digital predistortion, RLS is often used within an Indirect Learning Architecture to rapidly estimate postdistorter coefficients, though numerical stability techniques like QR-RLS or Tikhonov Regularization are required to prevent coefficient drift due to ill-conditioning.

ADAPTIVE FILTERING

Key Characteristics of RLS

Recursive Least Squares (RLS) is a deterministic, second-order adaptive algorithm that recursively minimizes a weighted linear least squares cost function. Unlike stochastic gradient methods, RLS leverages the full input correlation matrix to achieve an order-of-magnitude faster convergence rate.

01

Exponential Forgetting Factor

The forgetting factor (λ, typically 0.95 < λ < 1) assigns exponentially decreasing weights to older data, enabling the algorithm to track time-varying systems. A smaller λ provides faster tracking of non-stationary PA characteristics due to thermal drift, while a λ closer to 1 yields better steady-state accuracy in stationary conditions. This mechanism directly addresses coefficient drift in long-running DPD systems.

02

Matrix Inversion Lemma

RLS avoids the O(N³) computational cost of direct matrix inversion by applying the Woodbury matrix identity to recursively update the inverse of the input autocorrelation matrix. This reduces per-iteration complexity to O(N²), where N is the number of DPD coefficients. The recursion propagates both the gain vector and the inverse correlation matrix, maintaining numerical precision in sample-by-sample update modes.

03

Convergence Rate Superiority

RLS converges in approximately 2N iterations, where N is the filter order, compared to LMS which requires 10N–30N iterations. This is because RLS is a second-order method that whitens the input signal using the inverse correlation matrix, decorrelating the tap inputs. For wideband DPD with 50+ memory polynomial coefficients, this translates to convergence within microseconds rather than milliseconds.

04

QR-RLS Numerical Stability

Standard RLS suffers from covariance matrix ill-conditioning when the condition number grows large, leading to divergence. QR-RLS mitigates this by operating directly on the square-root of the covariance matrix using Givens rotations or Householder reflections. This doubles the dynamic range and guarantees positive-definiteness, making it the preferred variant for fixed-point FPGA implementations in closed-loop DPD systems.

05

Computational Cost Trade-off

The O(N²) complexity of RLS is significantly higher than the O(N) complexity of Least Mean Squares (LMS). For a predistorter with 100 coefficients, RLS requires approximately 10,000 multiply-accumulate operations per sample versus 100 for LMS. This computational burden often restricts RLS to burst training modes or offline coefficient estimation, rather than continuous sample-by-sample adaptation in resource-constrained FPGAs.

06

Regularization for Ill-Conditioned Inputs

When the input signal has low persistent excitation—common during silent intervals or narrowband transmissions—the autocorrelation matrix becomes singular. Tikhonov regularization adds a diagonal loading term (δI) to the initial covariance matrix, stabilizing the inversion. This prevents the gain vector from exploding and ensures robust coefficient updates during signal dropouts in open-loop DPD architectures.

ADAPTIVE ALGORITHM COMPARISON

RLS vs. LMS: Convergence and Complexity

Comparative analysis of Recursive Least Squares and Least Mean Squares algorithms for digital predistortion coefficient estimation, highlighting convergence speed, computational cost, and numerical stability.

FeatureRLSLMSQR-RLS

Convergence Rate

Fast (order of 2N iterations)

Slow (order of 10N+ iterations)

Fast (order of 2N iterations)

Computational Complexity per Iteration

O(N²)

O(N)

O(N²)

Sensitivity to Eigenvalue Spread

Insensitive

Highly sensitive

Insensitive

Numerical Stability

Moderate

Stable

Excellent

Misadjustment at Steady-State

Low

Higher

Low

Memory Requirement

O(N²)

O(N)

O(N²)

Suitable for Time-Varying Systems

Requires Matrix Inversion

RLS ALGORITHM INSIGHTS

Frequently Asked Questions

Clear, technically precise answers to the most common questions about Recursive Least Squares for adaptive digital predistortion coefficient estimation.

The Recursive Least Squares (RLS) algorithm is an adaptive filtering method that recursively finds the coefficients minimizing a weighted linear least squares cost function, offering significantly faster convergence than gradient-based alternatives like LMS. Unlike stochastic gradient methods that rely on instantaneous error estimates, RLS computes the exact solution to the least squares problem at each time step by maintaining and updating an estimate of the inverse autocorrelation matrix of the input signal. The algorithm operates by iteratively applying the matrix inversion lemma to avoid explicit matrix inversion, updating the coefficient vector with a gain vector that weights the prediction error. For digital predistortion, this means RLS can rapidly identify the inverse nonlinear characteristic of a power amplifier, making it particularly suitable for tracking time-varying distortion caused by thermal memory effects, bias drift, and signal envelope fluctuations. The core recursion consists of computing the Kalman gain vector, updating the coefficient estimate, and recursively updating the inverse correlation matrix, all with O(n²) computational complexity per iteration.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.