Inferensys

Glossary

Recursive Least Squares (RLS)

An adaptive filtering algorithm that updates model coefficients iteratively as new data arrives, offering faster convergence than LMS at the cost of higher computational complexity.
Data scientist building training data pipeline on laptop, data preprocessing visible, technical workspace.
ADAPTIVE FILTERING ALGORITHM

What is Recursive Least Squares (RLS)?

An adaptive filtering algorithm that updates model coefficients iteratively as new data arrives, offering faster convergence than LMS at the cost of higher computational complexity.

Recursive Least Squares (RLS) is an adaptive filtering algorithm that recursively updates the inverse autocorrelation matrix of the input signal to compute the exact least-squares solution at each time step. Unlike stochastic gradient methods, RLS minimizes a deterministic weighted sum of squared errors, achieving an order of magnitude faster convergence than the Least Mean Squares (LMS) algorithm.

The algorithm's superior tracking performance comes at the cost of O(N²) computational complexity per iteration, where N is the number of filter coefficients. A forgetting factor λ exponentially weights recent data to track time-varying systems, making RLS essential for online training of digital predistorters where rapid adaptation to changing amplifier characteristics is critical.

ADAPTIVE ESTIMATION

Key Characteristics of RLS

Recursive Least Squares (RLS) is a deterministic, second-order adaptive filtering algorithm that recursively updates the inverse of the input correlation matrix to achieve an order of magnitude faster convergence than gradient-based methods like LMS.

01

Exponential Weighting via Forgetting Factor

RLS incorporates a forgetting factor (λ, typically 0.95 < λ < 1) that applies exponential decay to historical data. This mechanism ensures the algorithm prioritizes recent observations over stale ones, enabling it to track time-varying amplifier behavior caused by thermal drift, bias shifts, and aging. A smaller λ provides faster tracking but increases steady-state misadjustment, creating a fundamental trade-off between tracking agility and estimation variance.

02

Direct Inverse Correlation Matrix Update

Unlike LMS which uses a scalar step size, RLS recursively computes the inverse of the input autocorrelation matrix (P(n)) using the Woodbury matrix identity. This second-order approach whitens the input signal in the parameter space, ensuring uniform convergence rates across all eigenmodes. The result is convergence within 10-20 samples for stationary signals, compared to hundreds or thousands for LMS, making RLS ideal for rapid DPD coefficient extraction during training bursts.

03

Computational Complexity: O(N²)

The primary drawback of RLS is its quadratic computational complexity relative to the number of filter taps (N). Each iteration requires:

  • Gain vector computation: O(N²) operations
  • Inverse matrix update: O(N²) via the Riccati equation
  • Coefficient update: O(N) operations For a 50-tap memory polynomial DPD model, this translates to approximately 2,500 multiply-accumulate operations per sample, making real-time FPGA implementation challenging without algorithmic optimizations or resource sharing.
04

Initialization and Regularization

RLS requires careful initialization of the inverse correlation matrix P(0). The standard approach sets:

  • P(0) = δ⁻¹I, where δ is a small positive constant (typically 0.01 to 0.001)
  • w(0) = 0 for the coefficient vector This initialization acts as implicit regularization, preventing numerical instability during the first few iterations when the autocorrelation estimate is rank-deficient. For ill-conditioned DPD basis functions, explicit L2 regularization can be incorporated by adding a diagonal loading term to the correlation matrix.
05

Convergence Rate vs. Misadjustment

RLS exhibits a deterministic convergence trajectory that is independent of the input signal's eigenvalue spread—a critical advantage over LMS when modeling power amplifiers with correlated nonlinear basis functions. The steady-state misadjustment (excess MSE) is approximately:

  • M = (1-λ)/(1+λ) × N This reveals that misadjustment grows linearly with the number of taps (N) and inversely with λ. For a 30-coefficient GMP model with λ=0.99, the theoretical misadjustment is approximately 0.15%, demonstrating RLS's superior steady-state accuracy compared to stochastic gradient methods.
06

Numerical Stability and Square-Root Variants

Standard RLS can suffer from covariance matrix blow-up due to the accumulation of round-off errors, causing P(n) to lose positive definiteness. Mitigation strategies include:

  • Square-root RLS: Propagates the Cholesky factor of P(n) to guarantee numerical stability
  • QR-decomposition RLS: Uses Givens rotations to maintain orthogonality
  • Periodic reinitialization: Resets P(n) when condition number exceeds a threshold These variants are essential for fixed-point FPGA implementations where limited precision arithmetic exacerbates numerical issues.
ADAPTIVE ALGORITHM COMPARISON

RLS vs. LMS vs. NLMS

Comparative analysis of recursive least squares, least mean squares, and normalized least mean squares algorithms for online coefficient estimation in digital predistortion systems.

FeatureRLSLMSNLMS

Convergence Speed

Fast (typically < 50 samples)

Slow (100-1000+ samples)

Moderate (50-200 samples)

Computational Complexity per Iteration

O(N²) to O(N³)

O(N)

O(N)

Steady-State Misadjustment

Low

Moderate to high

Moderate

Sensitivity to Input Signal Power Fluctuations

Low (inherently robust)

High (requires careful step size tuning)

Low (normalized by input power)

Memory Requirement

High (stores covariance matrix)

Minimal (coefficient vector only)

Minimal (coefficient vector only)

Numerical Stability

Condition-dependent (may require regularization)

Stable with appropriate step size

Stable with appropriate step size

Tracking Capability for Time-Varying Systems

Excellent (exponential forgetting factor)

Limited (fixed step size constraint)

Moderate (adaptive effective step size)

Typical Use Case in DPD

Offline model extraction and initial coefficient bootstrapping

Ultra-low-complexity real-time tracking

Real-time adaptation with moderate complexity

RECURSIVE LEAST SQUARES

Frequently Asked Questions

Clear, technical answers to common questions about the Recursive Least Squares algorithm, its convergence properties, and its role in adaptive digital predistortion.

The Recursive Least Squares (RLS) algorithm is an adaptive filtering technique that recursively updates model coefficients to minimize a weighted linear least-squares cost function as each new data sample arrives. Unlike batch Least Squares (LS), which requires a full data matrix and a computationally expensive pseudoinverse, RLS maintains and updates the inverse of the input autocorrelation matrix directly using the matrix inversion lemma (Woodbury matrix identity). At each time step n, the algorithm computes a gain vector, updates the coefficient vector based on the a priori estimation error, and recursively updates the inverse correlation matrix. This structure gives RLS an order of magnitude faster convergence than gradient-based methods like Least Mean Squares (LMS) in colored input signal environments, at the cost of O(N^2) computational complexity per iteration, where N is the number of filter taps.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.