Recursive Least Squares (RLS) is an adaptive filtering algorithm that recursively updates the inverse autocorrelation matrix of the input signal to compute the exact least-squares solution at each time step. Unlike stochastic gradient methods, RLS minimizes a deterministic weighted sum of squared errors, achieving an order of magnitude faster convergence than the Least Mean Squares (LMS) algorithm.
Glossary
Recursive Least Squares (RLS)

What is Recursive Least Squares (RLS)?
An adaptive filtering algorithm that updates model coefficients iteratively as new data arrives, offering faster convergence than LMS at the cost of higher computational complexity.
The algorithm's superior tracking performance comes at the cost of O(N²) computational complexity per iteration, where N is the number of filter coefficients. A forgetting factor λ exponentially weights recent data to track time-varying systems, making RLS essential for online training of digital predistorters where rapid adaptation to changing amplifier characteristics is critical.
Key Characteristics of RLS
Recursive Least Squares (RLS) is a deterministic, second-order adaptive filtering algorithm that recursively updates the inverse of the input correlation matrix to achieve an order of magnitude faster convergence than gradient-based methods like LMS.
Exponential Weighting via Forgetting Factor
RLS incorporates a forgetting factor (λ, typically 0.95 < λ < 1) that applies exponential decay to historical data. This mechanism ensures the algorithm prioritizes recent observations over stale ones, enabling it to track time-varying amplifier behavior caused by thermal drift, bias shifts, and aging. A smaller λ provides faster tracking but increases steady-state misadjustment, creating a fundamental trade-off between tracking agility and estimation variance.
Direct Inverse Correlation Matrix Update
Unlike LMS which uses a scalar step size, RLS recursively computes the inverse of the input autocorrelation matrix (P(n)) using the Woodbury matrix identity. This second-order approach whitens the input signal in the parameter space, ensuring uniform convergence rates across all eigenmodes. The result is convergence within 10-20 samples for stationary signals, compared to hundreds or thousands for LMS, making RLS ideal for rapid DPD coefficient extraction during training bursts.
Computational Complexity: O(N²)
The primary drawback of RLS is its quadratic computational complexity relative to the number of filter taps (N). Each iteration requires:
- Gain vector computation: O(N²) operations
- Inverse matrix update: O(N²) via the Riccati equation
- Coefficient update: O(N) operations For a 50-tap memory polynomial DPD model, this translates to approximately 2,500 multiply-accumulate operations per sample, making real-time FPGA implementation challenging without algorithmic optimizations or resource sharing.
Initialization and Regularization
RLS requires careful initialization of the inverse correlation matrix P(0). The standard approach sets:
- P(0) = δ⁻¹I, where δ is a small positive constant (typically 0.01 to 0.001)
- w(0) = 0 for the coefficient vector This initialization acts as implicit regularization, preventing numerical instability during the first few iterations when the autocorrelation estimate is rank-deficient. For ill-conditioned DPD basis functions, explicit L2 regularization can be incorporated by adding a diagonal loading term to the correlation matrix.
Convergence Rate vs. Misadjustment
RLS exhibits a deterministic convergence trajectory that is independent of the input signal's eigenvalue spread—a critical advantage over LMS when modeling power amplifiers with correlated nonlinear basis functions. The steady-state misadjustment (excess MSE) is approximately:
- M = (1-λ)/(1+λ) × N This reveals that misadjustment grows linearly with the number of taps (N) and inversely with λ. For a 30-coefficient GMP model with λ=0.99, the theoretical misadjustment is approximately 0.15%, demonstrating RLS's superior steady-state accuracy compared to stochastic gradient methods.
Numerical Stability and Square-Root Variants
Standard RLS can suffer from covariance matrix blow-up due to the accumulation of round-off errors, causing P(n) to lose positive definiteness. Mitigation strategies include:
- Square-root RLS: Propagates the Cholesky factor of P(n) to guarantee numerical stability
- QR-decomposition RLS: Uses Givens rotations to maintain orthogonality
- Periodic reinitialization: Resets P(n) when condition number exceeds a threshold These variants are essential for fixed-point FPGA implementations where limited precision arithmetic exacerbates numerical issues.
RLS vs. LMS vs. NLMS
Comparative analysis of recursive least squares, least mean squares, and normalized least mean squares algorithms for online coefficient estimation in digital predistortion systems.
| Feature | RLS | LMS | NLMS |
|---|---|---|---|
Convergence Speed | Fast (typically < 50 samples) | Slow (100-1000+ samples) | Moderate (50-200 samples) |
Computational Complexity per Iteration | O(N²) to O(N³) | O(N) | O(N) |
Steady-State Misadjustment | Low | Moderate to high | Moderate |
Sensitivity to Input Signal Power Fluctuations | Low (inherently robust) | High (requires careful step size tuning) | Low (normalized by input power) |
Memory Requirement | High (stores covariance matrix) | Minimal (coefficient vector only) | Minimal (coefficient vector only) |
Numerical Stability | Condition-dependent (may require regularization) | Stable with appropriate step size | Stable with appropriate step size |
Tracking Capability for Time-Varying Systems | Excellent (exponential forgetting factor) | Limited (fixed step size constraint) | Moderate (adaptive effective step size) |
Typical Use Case in DPD | Offline model extraction and initial coefficient bootstrapping | Ultra-low-complexity real-time tracking | Real-time adaptation with moderate complexity |
Frequently Asked Questions
Clear, technical answers to common questions about the Recursive Least Squares algorithm, its convergence properties, and its role in adaptive digital predistortion.
The Recursive Least Squares (RLS) algorithm is an adaptive filtering technique that recursively updates model coefficients to minimize a weighted linear least-squares cost function as each new data sample arrives. Unlike batch Least Squares (LS), which requires a full data matrix and a computationally expensive pseudoinverse, RLS maintains and updates the inverse of the input autocorrelation matrix directly using the matrix inversion lemma (Woodbury matrix identity). At each time step n, the algorithm computes a gain vector, updates the coefficient vector based on the a priori estimation error, and recursively updates the inverse correlation matrix. This structure gives RLS an order of magnitude faster convergence than gradient-based methods like Least Mean Squares (LMS) in colored input signal environments, at the cost of O(N^2) computational complexity per iteration, where N is the number of filter taps.
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.

Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.

Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
Related Terms
Recursive Least Squares (RLS) is a cornerstone of adaptive filtering, offering a deterministic optimization approach that contrasts sharply with stochastic methods. The following concepts define its operational ecosystem.
Least Mean Squares (LMS)
A stochastic gradient descent algorithm that updates coefficients sample-by-sample using only the instantaneous gradient estimate. Unlike RLS, LMS does not require matrix inversion, making it computationally trivial but significantly slower to converge, especially for highly correlated input signals. It trades mathematical rigor for simplicity.
Forgetting Factor (λ)
A critical scalar parameter (0 < λ ≤ 1) that applies exponential weighting to past data. A smaller λ allows RLS to track rapidly time-varying systems by discounting old measurements, while λ=1 gives equal weight to all history, suitable for stationary environments. It directly controls the algorithm's memory and adaptability.
Least Squares (LS)
The batch estimation counterpart to RLS. LS computes the optimal coefficients in a single step by solving the normal equations using the entire dataset. It is the theoretical gold standard for offline model extraction but is computationally prohibitive for real-time adaptation where RLS provides a recursive, sample-by-sample equivalent.
Covariance Matrix (P)
The inverse of the weighted input autocorrelation matrix, updated recursively in RLS. It quantifies the uncertainty in coefficient estimates. A large initial P matrix represents high uncertainty, allowing rapid initial convergence. As data accumulates, P shrinks, reducing the gain applied to new measurements.
Kalman Filter
A state-space estimation framework that RLS is a special case of. When the state transition matrix is identity and the process noise is zero, the Kalman filter equations collapse to RLS. This connection provides a Bayesian interpretation where RLS computes the maximum a posteriori estimate under a Gaussian random walk model.
Ill-Conditioning
A numerical instability where the input correlation matrix becomes nearly singular, causing the covariance matrix P to explode. In RLS, this leads to wild coefficient fluctuations. Regularization techniques or square-root formulations (e.g., QR-RLS) are essential to maintain numerical robustness in fixed-point hardware implementations.

About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us