Glossary

Least Squares (LS)

A batch estimation algorithm that finds model coefficients by minimizing the sum of squared errors between the model's prediction and the measured output in a single computation.

Get in touch Learn more

MLOps engineer reviewing model serving infrastructure on laptop, container orchestration visible, technical workspace.

BATCH ESTIMATION

What is Least Squares (LS)?

A foundational parameter estimation technique that computes optimal model coefficients by minimizing the sum of squared residuals in a single closed-form computation.

Least Squares (LS) is a batch estimation algorithm that determines the optimal coefficients of a behavioral model by minimizing the sum of squared errors between the model's predicted output and the measured power amplifier output in a single, non-iterative computation. The method solves an overdetermined system of equations by applying the Moore-Penrose pseudoinverse to the regression matrix, yielding the coefficient vector that provides the best linear unbiased estimate under the assumption of white Gaussian measurement noise.

In digital predistortion model extraction, LS estimation is prized for its deterministic speed and simplicity when processing a complete captured data record offline. However, the technique is highly susceptible to ill-conditioning when the basis function correlation matrix is nearly singular, often requiring regularization techniques such as ridge regression or dimensionality reduction via principal component analysis (PCA) to stabilize the matrix inversion and prevent coefficient explosion.

BATCH ESTIMATION

Key Characteristics of Least Squares

Least Squares (LS) is a foundational batch estimation algorithm that computes optimal model coefficients in a single computation by minimizing the sum of squared residuals between predicted and measured outputs.

The Closed-Form Solution

LS provides a one-shot analytical solution via the normal equations: θ̂ = (XᵀX)⁻¹Xᵀy. This computes the coefficient vector θ̂ that minimizes ||y - Xθ||² without iterative gradient descent. The solution requires computing the Moore-Penrose pseudoinverse of the regression matrix X, making it deterministic and repeatable—identical data always yields identical coefficients.

Single-pass

Computation Mode

The Squared Error Cost Function

LS minimizes the L2 norm of the residual vector: J(θ) = Σᵢ(yᵢ - ŷᵢ)². This quadratic cost surface has a single global minimum when X has full column rank, guaranteeing a unique solution. The squaring operation heavily penalizes large errors, making LS sensitive to outliers in measurement data. For PA modeling, this means clean, high-SNR captures are essential.

Quadratic

Cost Surface Shape

Overdetermined Systems Requirement

LS requires an overdetermined system where the number of measurement samples N significantly exceeds the number of model parameters M. This redundancy averages out measurement noise. A typical PA model extraction might use N = 10,000+ samples to estimate M = 50-200 coefficients, providing a high degree of noise immunity through statistical averaging.

N ≫ M

Required Condition

Numerical Stability and Ill-Conditioning

The (XᵀX) matrix must be invertible. When basis functions are highly correlated—common in polynomial PA models—the condition number skyrockets, causing ill-conditioning. This makes coefficient estimates wildly sensitive to minor measurement noise. Remedies include:

Ridge Regression: Adds an L2 penalty λ||θ||² to stabilize inversion
PCA-based basis reduction: Decorrelates the regression matrix
QR decomposition: Avoids explicitly forming XᵀX

κ(XᵀX)

Stability Metric

Batch vs. Recursive Trade-off

LS processes all data at once, unlike Recursive Least Squares (RLS) or LMS which update coefficients sample-by-sample. This makes LS ideal for offline model extraction from captured waveforms where computational latency is irrelevant. However, LS cannot track time-varying PA behavior—it produces a static model. For adaptive DPD, the batch LS solution often serves as the initial seed for recursive algorithms.

Offline

Primary Use Case

Static

Model Type

Gauss-Markov Assumptions

LS yields the Best Linear Unbiased Estimator (BLUE) when measurement noise satisfies the Gauss-Markov conditions:

Zero mean: E[ε] = 0
Homoscedasticity: Constant variance across samples
No autocorrelation: Errors are independent In PA measurements, violations occur due to quantization noise from ADCs and phase noise in the observation receiver. Weighted Least Squares (WLS) can compensate for non-constant error variance.

BLUE

Optimality Guarantee

LEAST SQUARES ESTIMATION

Frequently Asked Questions

Clear answers to common questions about the batch estimation workhorse used to extract power amplifier behavioral models from measured data.

The Least Squares (LS) algorithm is a batch estimation method that finds the optimal coefficients for a behavioral model by minimizing the sum of squared errors between the model's predicted output and the actual measured output of a power amplifier. It operates in a single computation by solving the normal equation w = (X^H X)^{-1} X^H y, where X is the regression matrix constructed from basis functions of the input signal, y is the vector of measured output samples, and w is the vector of model coefficients to be determined. The algorithm assumes that the error residuals are uncorrelated and have constant variance. Because it processes the entire dataset at once, LS provides the minimum-variance unbiased estimator under the Gauss-Markov theorem, making it the gold standard for offline model extraction when computational resources and time permit a full matrix inversion.

BATCH VS. RECURSIVE PARAMETER ESTIMATION

Least Squares vs. Adaptive Estimation Algorithms

Comparison of the batch Least Squares method with sample-by-sample adaptive algorithms used for power amplifier model coefficient extraction and real-time DPD adaptation.

Feature	Least Squares (LS)	Recursive Least Squares (RLS)	Least Mean Squares (LMS)
Computation Mode	Batch (offline)	Recursive (online)	Recursive (online)
Convergence Speed	Instantaneous (single solve)	Fast (typically 10-50 samples)	Slow (typically 100-1000+ samples)
Computational Complexity per Sample	O(N³) one-time cost	O(N²)	O(N)
Memory Requirement	High (stores entire data matrix)	Moderate (stores covariance matrix)	Low (stores coefficient vector only)
Steady-State MSE	Minimum achievable (optimal for stationary data)	Approaches LS optimum	Higher than LS and RLS
Tracking of Time-Varying Systems
Numerical Stability on Ill-Conditioned Data	Poor (requires regularization)	Moderate (divergence risk without reset)	Robust (inherently stable)
Typical Application	Initial DPD coefficient extraction from training waveform	Fast-adapting DPD for temperature drift	Continuous background tracking in stable operation

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

FOUNDATIONAL ESTIMATION CONCEPTS

Related Terms

Least Squares estimation is the cornerstone of batch model extraction. These related terms define the mathematical and algorithmic ecosystem required to compute stable, accurate power amplifier models from measured data.

Moore-Penrose Pseudoinverse

The computational engine behind the LS solution. When the regression matrix X is not square, the pseudoinverse X⁺ = (XᵀX)⁻¹Xᵀ provides the minimum-norm least-squares solution. In DPD extraction, this single matrix operation computes all model coefficients in one shot, making it the standard batch estimator for offline behavioral modeling.

O(n³)

Computational Complexity

Overdetermined System

The standard condition for robust model extraction where the number of measurement samples far exceeds the number of unknown coefficients. This redundancy averages out measurement noise. A typical DPD extraction might use 100,000+ samples to estimate 50–200 coefficients, creating a highly overdetermined system where the LS solution finds the best compromise across all data points.

Ill-Conditioning

A critical numerical pathology where the condition number of the correlation matrix XᵀX becomes extremely large. This occurs when basis functions are highly correlated, making the pseudoinverse hypersensitive to noise. In power amplifier modeling, closely spaced memory taps or redundant polynomial terms frequently cause ill-conditioning, requiring regularization to stabilize the solution.

Ridge Regression

A regularized extension of LS that adds an L2 penalty λ‖w‖² to the cost function. The solution becomes w = (XᵀX + λI)⁻¹Xᵀy, where λ shrinks coefficient magnitudes. This directly combats ill-conditioning by adding a stabilizing diagonal term to the correlation matrix. In DPD, ridge regression prevents wild coefficient swings when basis functions are nearly collinear.

Recursive Least Squares (RLS)

The adaptive counterpart to batch LS. Instead of computing the pseudoinverse on all data at once, RLS updates the coefficient vector w(n) and the inverse correlation matrix P(n) with each new sample. The forgetting factor λ (typically 0.95–0.999) weights recent data more heavily, enabling the model to track amplifier drift due to temperature changes or aging.

O(n²)

Per-Sample Complexity

Model Order Estimation

The process of selecting the optimal nonlinearity order and memory depth before applying LS. Too few terms cause underfitting and residual distortion; too many cause overfitting to measurement noise. Information-theoretic criteria like AIC and BIC penalize model complexity relative to fit quality, guiding the engineer to the most parsimonious structure that captures the amplifier's true behavior.

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.