The Least Mean Squares (LMS) algorithm is a stochastic gradient descent adaptive filter that iteratively updates its coefficients to minimize the instantaneous squared error. It operates by computing the gradient of the squared error with respect to the filter weights and taking a small step in the opposite direction, governed by a learning rate parameter. Its primary advantage is extremely low computational complexity, requiring only (O(N)) multiplications per iteration, making it ideal for real-time implementation on resource-constrained hardware like FPGAs.
Glossary
Least Mean Squares (LMS)

What is Least Mean Squares (LMS)?
The Least Mean Squares (LMS) algorithm is a foundational stochastic gradient descent method used to adaptively update filter coefficients by minimizing the instantaneous squared error between a desired and actual signal.
In closed-loop DPD systems, LMS provides a simple mechanism for online coefficient tracking. The algorithm's convergence behavior is directly influenced by the eigenvalue spread of the input signal's correlation matrix; ill-conditioned signals cause slow convergence. To mitigate this, variants like Normalized LMS (NLMS) adapt the step size based on instantaneous signal power, improving stability. While LMS converges slower than Recursive Least Squares (RLS), its robustness and minimal computational overhead make it a persistent choice for background calibration loops tracking slowly varying PA nonlinearities.
Key Characteristics of LMS
The Least Mean Squares (LMS) algorithm is the workhorse of online training for Digital Pre-Distortion, prized for its extreme computational simplicity and robust real-time performance.
Stochastic Gradient Descent Engine
LMS is a stochastic approximation of the steepest descent method. Instead of computing the true gradient over the entire statistical ensemble, it uses the instantaneous gradient of the squared error. This noisy estimate is computationally cheap and allows the algorithm to track time-varying systems like a warming power amplifier.
Computational Complexity: O(N)
The primary advantage of LMS in FPGA or ASIC implementations is its minimal footprint. Each iteration requires only 2N + 1 multiplications and 2N additions, where N is the number of coefficients. There are no matrix inversions or correlation matrix calculations, making it ideal for high-bandwidth, resource-constrained DPD applications.
The Learning Rate (μ) Trade-off
The step-size parameter μ governs the algorithm's dynamics:
- Large μ: Fast convergence but high steady-state misadjustment (excess error).
- Small μ: Low misadjustment but slow tracking of PA memory effects.
- Stability Bound: 0 < μ < 2 / (N * E[|x|²]), where E[|x|²] is the input signal power.
Normalized LMS (NLMS) Variant
Standard LMS is sensitive to input signal power fluctuations. Normalized LMS solves this by dividing the step size by the squared Euclidean norm of the input vector. This makes the convergence behavior independent of signal amplitude, a critical feature for modern communication signals with high Peak-to-Average Power Ratio (PAPR).
Misadjustment vs. Convergence
Unlike Recursive Least Squares (RLS), LMS does not converge to the Wiener solution. It wanders in a 'bowl' around the minimum, producing excess Mean Squared Error (MSE). The misadjustment M is proportional to μ * Tr[R], where R is the input correlation matrix. This inherent noise floor must be budgeted in the DPD link budget.
Hardware Implementation & Pipelining
LMS maps efficiently to hardware. The update equation w(n+1) = w(n) + μ * e(n) * x(n) can be heavily pipelined. However, delayed LMS must be analyzed carefully; a delay in the error feedback path can destabilize the loop if not compensated for in the step-size selection.
LMS vs. Other Adaptive Algorithms
Comparative analysis of Least Mean Squares against other core adaptive filtering algorithms used in online DPD coefficient estimation.
| Feature | LMS | NLMS | RLS |
|---|---|---|---|
Update Equation Complexity | O(N) | O(N) | O(N²) |
Convergence Rate | Slow | Moderate | Fast |
Steady-State Misadjustment | Higher | Moderate | Lower |
Sensitivity to Input Scaling | High | Low (Normalized) | Low |
Numerical Stability | High | High | Moderate (Requires Regularization) |
Memory Requirement | Low | Low | High |
Tracking of Non-Stationary Signals | Good | Good | Poor (Without Forgetting Factor) |
Typical Use Case in DPD | Initial coarse tuning | Stable background calibration | Rapid initial acquisition |
Frequently Asked Questions
Clear, technically precise answers to the most common questions about the Least Mean Squares algorithm, its role in adaptive filtering, and its application in real-time Digital Pre-Distortion systems.
The Least Mean Squares (LMS) algorithm is a stochastic gradient descent adaptive filter that iteratively updates its coefficients to minimize the instantaneous squared error between a desired signal and the filter's actual output. It operates by computing the gradient of the squared error with respect to the filter coefficients at each time step and adjusting the coefficients in the negative gradient direction. The update rule is w(n+1) = w(n) + μ * e(n) * x(n), where w(n) is the coefficient vector, μ is the learning rate (step size), e(n) is the instantaneous error signal, and x(n) is the input signal vector. Unlike block-based methods that require batch data, LMS processes samples one at a time, making it inherently suited for online training and real-time closed-loop DPD systems where coefficients must track time-varying power amplifier nonlinearities.
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.

Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.

Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
Related Terms
Core algorithms and concepts that form the mathematical foundation for online coefficient estimation in digital predistortion systems.
Normalized Least Mean Squares (NLMS)
A direct extension of LMS that normalizes the step size by the input signal power to eliminate gradient noise amplification. When the input signal exhibits significant power fluctuations—common in modern communication waveforms with high Peak-to-Average Power Ratio (PAPR)—standard LMS becomes unstable. NLMS divides the step size μ by the squared Euclidean norm of the input vector plus a small regularization constant, ensuring consistent convergence behavior regardless of signal amplitude. The update equation is: w(n+1) = w(n) + [μ / (||x(n)||² + δ)] · e(n) · x(n). This normalization makes NLMS the practical default choice for DPD applications where signal statistics vary over time.
Recursive Least Squares (RLS)
An adaptive algorithm that minimizes a weighted linear least squares cost function recursively, offering an order-of-magnitude faster convergence than LMS at the cost of O(N²) computational complexity. RLS maintains and updates the inverse of the input correlation matrix using the matrix inversion lemma, eliminating the need for explicit matrix inversion at each step. A forgetting factor λ (typically 0.95–0.999) exponentially weights past data, enabling the algorithm to track time-varying PA characteristics. In DPD applications, RLS is preferred when rapid adaptation to changing amplifier behavior is critical, such as during temperature transients or channel reconfiguration events.
Stochastic Gradient Descent (SGD)
The foundational optimization method from which LMS is derived. SGD updates model parameters using the gradient of a loss function computed on a single sample or small minibatch rather than the full dataset. Key properties relevant to DPD:
- Unbiased gradient estimates: The instantaneous gradient is a noisy but unbiased estimator of the true gradient
- Learning rate annealing: Gradually reducing the step size over time improves convergence to the optimal solution
- Momentum variants: Adding a velocity term (e.g., Adam, RMSprop) accelerates convergence in ravines and plateaus LMS is essentially SGD applied to a mean squared error cost function with an identity preconditioner.
Adaptive Filter
A self-adjusting digital filter that automatically modifies its transfer function according to an optimization algorithm driven by an error signal. In the DPD context, the adaptive filter implements the predistorter, with coefficients updated to invert the PA's nonlinear characteristics. Core components:
- Filter structure: FIR, IIR, or nonlinear architectures (Volterra, memory polynomial)
- Adaptation algorithm: LMS, NLMS, RLS, or their variants
- Error computation: Difference between desired linear output and observed PA output The adaptive filter operates in a closed loop, continuously monitoring the feedback receiver and adjusting coefficients to maintain linearity as amplifier conditions change.
Convergence Rate vs. Misadjustment
A fundamental trade-off in adaptive filtering that governs DPD performance. Convergence rate measures how quickly the algorithm reaches steady-state—critical for tracking rapid PA changes due to thermal effects or supply voltage variations. Misadjustment is the excess mean squared error above the theoretical Wiener optimum, caused by gradient noise from the stochastic updates.
- Large step size μ: Fast convergence, high misadjustment, potential instability
- Small step size μ: Slow convergence, low misadjustment, better steady-state accuracy The learning rate parameter directly controls this trade-off. Advanced techniques like variable step-size LMS dynamically adjust μ based on error signal statistics to balance both objectives.
Cost Function Design
The mathematical function that quantifies the aggregate error between desired and actual system output, which the adaptation algorithm minimizes. In standard LMS, the cost function is the instantaneous squared error: J(n) = |e(n)|² = |d(n) − w^H(n)x(n)|². For DPD applications, cost functions may be extended to include:
- Weighted error norms: Emphasizing in-band EVM vs. out-of-band ACLR
- Regularization terms: Adding ||w||² to prevent coefficient drift during low-signal periods
- Multi-objective formulations: Simultaneously optimizing multiple metrics like efficiency and linearity The choice of cost function directly shapes the predistorter's behavior and must align with system-level performance requirements.

About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us