Post-distortion error is the residual nonlinear distortion signal remaining after a digital predistorter (DPD) has been applied to a power amplifier (PA), calculated as the complex difference between the desired ideal linear output and the actual measured amplifier output. It serves as the primary cost function minimized during coefficient estimation and the definitive metric for validating linearization performance.
Glossary
Post-Distortion Error

What is Post-Distortion Error?
The fundamental metric for quantifying the effectiveness of a digital predistortion linearization system.
Minimizing this error directly reduces spectral regrowth and improves error vector magnitude (EVM). In an indirect learning architecture, the post-distortion error drives the adaptation of the post-distorter model, which is then copied to the predistorter. Persistent post-distortion error indicates model deficiencies, such as underfitting, ill-conditioning during extraction, or uncompensated thermal memory effects.
Key Characteristics of Post-Distortion Error
The post-distortion error is the critical feedback signal that quantifies the effectiveness of a digital predistortion (DPD) system. It represents the uncorrected nonlinear artifacts remaining at the power amplifier output after linearization.
Definition and Mathematical Formulation
Post-distortion error is the complex-valued difference between the desired ideal linear output and the actual measured amplifier output after predistortion is applied. Mathematically, it is expressed as e(n) = y_ideal(n) - y_measured(n), where y_ideal(n) is the linearly scaled input signal and y_measured(n) is the captured PA output. This error vector is the primary cost function minimized by adaptive DPD algorithms like the Indirect Learning Architecture and Direct Learning Architecture.
Spectral Interpretation: Residual Regrowth
In the frequency domain, post-distortion error manifests as residual spectral regrowth in adjacent channels. While a perfect DPD would eliminate all out-of-band emissions, practical systems leave a measurable error floor. Key metrics include:
- Adjacent Channel Leakage Ratio (ACLR) improvement limits
- Residual spurious emissions in the transmit noise floor
- In-band error vector magnitude (EVM) degradation This residual spectrum is the ultimate indicator of linearization quality for regulatory compliance.
Time-Domain Error Characteristics
The instantaneous error signal exhibits distinct temporal patterns tied to amplifier physics. Peak error events typically coincide with signal envelope peaks where the PA is driven deep into compression. Memory effects cause error persistence that spans multiple samples, visible as temporal correlation in the error sequence. Analyzing the amplitude-to-amplitude modulation (AM/AM) and amplitude-to-phase modulation (AM/PM) characteristics of the residual error reveals whether the DPD model order is sufficient to capture the PA's nonlinear dynamics.
Convergence Monitoring in Adaptive Systems
Post-distortion error serves as the real-time feedback mechanism for adaptive DPD coefficient updates. In Least Mean Squares (LMS) and Recursive Least Squares (RLS) algorithms, the error directly drives coefficient adjustment. Key monitoring behaviors include:
- Monotonic error power decrease during initial convergence
- Steady-state error floor indicating the best achievable linearization
- Error divergence signaling model instability or PA characteristic drift Tracking the error power over time is essential for robust field deployment.
Error Sources and Decomposition
Not all post-distortion error originates from PA nonlinearity alone. A comprehensive error budget includes:
- Modeling residual: Error from insufficient DPD model complexity or order
- Estimation noise: Coefficient inaccuracies from noisy observation receivers
- Time misalignment: Residual error from imperfect loop delay estimation
- IQ impairments: Uncorrected IQ imbalance in the modulator or demodulator
- Quantization noise: Finite precision effects in the digital predistortion path Isolating these components is critical for targeted system improvement.
Normalized Mean Squared Error (NMSE)
NMSE is the standard scalar metric for quantifying post-distortion error magnitude. It normalizes the mean squared error by the reference signal power, expressed in dB. Typical DPD systems target NMSE values below -35 dB to -45 dB for wideband signals. NMSE correlates strongly with ACLR improvement and EVM performance. It is computed over a validation dataset distinct from training data to assess generalization and detect overfitting in the extracted behavioral model.
Post-Distortion Error vs. Related Error Metrics
Comparison of post-distortion error with other key error metrics used in digital predistortion linearization and power amplifier behavioral modeling.
| Feature | Post-Distortion Error | Normalized Mean Squared Error (NMSE) | Adjacent Channel Leakage Ratio (ACLR) |
|---|---|---|---|
Definition | Residual nonlinear distortion measured after applying a predistorter, calculated as the difference between the ideal linear output and the actual amplifier output | Time-domain error between modeled and measured signals normalized by the signal power, expressed in dB | Ratio of power in adjacent frequency channels to power in the main channel, measured in dBc |
Domain | Time-domain | Time-domain | Frequency-domain |
Primary Use Case | Validating predistorter correction effectiveness and quantifying residual nonlinearity | Evaluating behavioral model accuracy during extraction and validation | Regulatory compliance testing and quantifying spectral regrowth |
Typical Target Value | < -50 dBc for commercial base stations | < -40 dB for high-fidelity models | < -45 dBc for 3GPP compliance |
Sensitivity to Time Alignment | High | High | Moderate |
Captures Memory Effects | |||
Directly Measures Linearization Performance | |||
Computational Complexity | Low | Low | Moderate |
Frequently Asked Questions
Clarifying the residual nonlinear distortion that remains after digital predistortion is applied, and how it is quantified to validate linearization performance.
Post-distortion error is the residual nonlinear distortion measured at the output of a power amplifier after a digital predistorter has been applied, calculated as the complex difference between the desired ideal linear output signal and the actual measured amplifier output. It represents the uncorrected distortion that the predistorter failed to compensate, serving as the primary cost function minimized during coefficient estimation. Mathematically, it is expressed as e(n) = y_ideal(n) - y_measured(n), where y_ideal is the linearly scaled input and y_measured is the captured PA output. This error signal drives adaptation in both Direct Learning Architecture and Indirect Learning Architecture implementations.
Enabling Efficiency, Speed & Accuracy
Intelligent Analysis, Decision & Execution
We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.
Talk to Us
Search across company data
Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.
Useful when people spend too long searching or get different answers from different systems.

Automate internal workflows
Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.
Useful when repetitive work moves across multiple tools and teams.

Add AI to products and internal tools
Build assistants, guided actions, or decision support into the software your team or customers already use.
Useful when AI needs to be part of the product, not a separate tool.
Related Terms
Understanding post-distortion error requires familiarity with the metrics, architectures, and phenomena that define residual nonlinearity after linearization.
Normalized Mean Squared Error (NMSE)
The primary figure of merit for quantifying post-distortion error. NMSE calculates the power of the residual error signal normalized by the power of the ideal reference signal, typically expressed in decibels (dB).
- Formula: 10*log10(mean(|y_ideal - y_measured|^2) / mean(|y_ideal|^2))
- A lower (more negative) NMSE indicates superior linearization performance.
- Values below -40 dB are generally considered excellent for wideband signals.
- NMSE is sensitive to both in-band distortion (EVM degradation) and out-of-band distortion (spectral regrowth).
Error Vector Magnitude (EVM)
A time-domain metric that directly measures the deviation of the actual transmitted symbol constellation from the ideal reference points. EVM is a direct consequence of residual post-distortion error corrupting the modulated signal.
- EVM is expressed as a percentage of the peak or average symbol power.
- Standards like 3GPP mandate strict EVM limits (e.g., 3.5% for 256-QAM in 5G NR).
- While NMSE captures total error power, EVM isolates the in-band distortion that degrades bit error rate (BER).
- Poor EVM despite good NMSE often indicates IQ imbalance or phase noise, not just nonlinearity.
Adjacent Channel Leakage Ratio (ACLR)
The ratio of power transmitted in an adjacent frequency channel to the power in the intended carrier channel. Post-distortion error manifests as residual spectral regrowth that elevates ACLR.
- Regulatory bodies like the FCC and ETSI impose strict ACLR masks to prevent interference.
- A well-tuned DPD system can improve ACLR by 15-25 dB compared to an unlinearized amplifier.
- ACLR is primarily sensitive to odd-order nonlinearities (IM3, IM5) in the PA.
- Monitoring ACLR alongside NMSE provides a complete picture of both in-band and out-of-band error.
Indirect Learning Architecture (ILA)
The dominant closed-loop architecture for DPD coefficient estimation. In ILA, a post-distorter model is trained to minimize the error between the PA output and the desired linear output, then the coefficients are copied to the predistorter.
- The key assumption is that the post-inverse equals the pre-inverse, which holds for systems with mild memory effects.
- Post-distortion error is the explicit training signal for the post-distorter block.
- ILA is computationally efficient because it avoids solving a direct inverse problem.
- However, measurement noise in the feedback path can bias the coefficient estimate, increasing residual error.
Direct Learning Architecture (DLA)
An adaptive architecture that directly minimizes the post-distortion error by iteratively updating predistorter coefficients based on the difference between the ideal input and the measured PA output.
- DLA does not rely on the post-inverse equals pre-inverse assumption, making it theoretically more robust than ILA.
- Requires a nonlinear optimization solver (e.g., Gauss-Newton or Levenberg-Marquardt) at each iteration.
- Better suited for strong memory effects and complex Doherty amplifier topologies.
- The convergence speed and final residual error depend heavily on the accuracy of the PA model used in the optimization loop.
Residual Nonlinearity Floor
The minimum achievable post-distortion error limited by physical phenomena that a DPD model cannot compensate for. This floor sets the ultimate performance boundary for any linearization system.
- Contributing factors:
- Thermal noise in the feedback receiver chain.
- Quantization noise from the ADC in the observation path.
- Unmodeled dynamics such as charge trapping in GaN HEMT devices.
- Modulator impairments (IQ imbalance, LO leakage) that are not corrected.
- Understanding this floor prevents over-engineering the DPD model complexity beyond what the hardware can exploit.

About the author
Prasad Kumkar
CEO & MD, Inference Systems
Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.
His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.
Partnered with leading AI, data, and software stack.
How We Work
Custom AI workflows for your Business
One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.
01
Review the use case
We understand the task, the users, and where AI can actually help.
Read more02
Pick the right approach
We define what needs search, automation, or product integration.
Read more03
Build the first useful version
We implement the part that proves the value first.
Read more04
Improve from there
We add the checks and visibility needed to keep it useful.
Read moreThe first call is a practical review of your use case and the right next step.
Talk to Us