Volterra Kernel Pruning

Each kernel in the Volterra series is assigned a significance score based on its contribution to modeling accuracy. Common metrics include:

• Coefficient magnitude: Kernels with near-zero weights are pruned first
• Mean squared error impact: Measuring the degradation when a kernel is removed
• Orthogonal matching pursuit (OMP): Greedy selection of most correlated basis functions
• Fisher information: Quantifying the statistical relevance of each term The pruning threshold is typically set to remove 60-90% of kernels while maintaining ACLR within 0.5 dB of the full model.

Volterra kernels exhibit a near-diagonal dominance property where kernels with indices clustered around the main diagonal contribute most to the nonlinear response. Pruning strategies exploit this by:

• Retaining kernels where index differences are small (|n₁ - n₂| ≤ D)
• Discarding far-off-diagonal terms that model weak higher-order mixing products
• Using radial pruning to keep kernels within a Manhattan distance from the diagonal This structural pruning reduces the O(N³) complexity of a full 3rd-order Volterra model to approximately O(N²) or lower.

In operational transmitters, the optimal set of active kernels shifts with signal statistics and operating conditions. Advanced pruning implementations support:

• Run-time kernel selection based on instantaneous PAPR and signal bandwidth
• Look-up table indexed pruning masks that switch between pre-computed sparse model configurations
• Online significance re-evaluation using recursive least squares (RLS) to track slowly varying PA characteristics This dynamic approach ensures the pruned model adapts to changing modulation schemes and traffic patterns without full model retraining.

Pruning transforms the dense Volterra kernel matrix into a sparse representation, yielding significant hardware implementation advantages:

• Reduced multiply-accumulate operations: Directly proportional to the number of retained kernels
• Lower memory bandwidth: Fewer coefficients to fetch from LUTs or BRAM on FPGAs
• Improved numerical conditioning: Eliminating correlated kernels reduces the condition number of the estimation matrix
• Faster coefficient updates: Smaller matrices accelerate LS and RLS estimation loops For a 5G NR 100 MHz signal, a pruned model may require only 50-200 active kernels versus thousands in the full series.

Generalized memory polynomial (GMP) and Volterra models contain cross-terms coupling the signal with lagging and leading envelope samples. Pruning these requires specialized strategies:

• Envelope lag-lead symmetry: Exploiting symmetry to prune redundant cross-term pairs
• Temporal decay weighting: Applying exponential decay to cross-term significance with increasing lag distance
• Bandwidth-dependent selection: Retaining more cross-terms for wider signal bandwidths where memory effects are pronounced
• PA-class-specific masks: Pre-defined pruning patterns optimized for Doherty, Class-AB, or GaN amplifier architectures Cross-term pruning often achieves 70-80% reduction with negligible linearization loss.

Pruned models must be rigorously validated to ensure they do not introduce spectral regrowth under worst-case conditions:

• Multi-tone testing: Verifying IMD suppression across the full band
• Modulated signal stress tests: Using 256-QAM and OFDM signals with peak PAPR
• Temperature and frequency sweeps: Confirming robustness across operating corners
• Guardband retention: Keeping 5-15% extra kernels beyond the strict significance threshold as a safety margin Validation ensures the pruned model meets regulatory ACLR and EVM requirements across all specified operating conditions.

The systematic process of decreasing the number of coefficients in a behavioral model to minimize computational load while preserving linearization performance. Kernel pruning is a primary technique within this broader discipline.

• Goal: Reduce multiply-accumulate operations (MACs) in FPGA fabric
• Methods: Pruning, sparse identification, PCA
• Metric: Normalized Mean Squared Error (NMSE) vs. coefficient count

A greedy sparse approximation algorithm that iteratively selects the most correlated basis function from a dictionary to build a compact, low-complexity predistorter model. OMP is a direct algorithmic competitor to significance-metric-based kernel pruning.

• Mechanism: Projects residual error onto candidate kernels
• Advantage: Guarantees a specific sparsity level (K-sparse)
• Trade-off: Greedy selection may miss globally optimal kernel subsets

An enhanced memory polynomial model that includes cross-terms between the signal and its lagging or leading envelope samples. The GMP is the most common dictionary from which kernels are pruned, as its full form contains hundreds of terms.

• Cross-term types: Lagging envelope, leading envelope
• Full GMP complexity: Often >200 coefficients for wideband signals
• Pruning target: Retain only 20-40 dominant GMP terms

A dimensionality reduction technique applied to the basis function matrix to identify and retain only the most significant principal components. Unlike kernel pruning, PCA transforms the basis rather than selecting a subset.

• Approach: Eigenvalue decomposition of the correlation matrix
• Result: Orthogonalized, reduced-rank basis set
• Key difference: PCA creates new composite basis functions; pruning selects original kernels

A regularized least squares estimation technique that adds an L2 penalty on coefficient magnitude to the cost function. Often used in conjunction with pruning to stabilize the estimation of the remaining significant kernels.

• Regularization parameter (λ): Controls bias-variance trade-off
• Benefit: Prevents overfitting when model order is aggressively reduced
• Synergy: Pruning selects structure; ridge regression ensures robust coefficient values

The systematic selection or pruning of cross-terms in a behavioral model to balance linearization accuracy against computational complexity. Kernel pruning is the algorithmic implementation of cross-term management strategy.

• Scope: Envelope memory terms, lagging/leading cross-terms
• Significance metrics: Correlation magnitude, MSE reduction contribution
• Outcome: A sparse model that captures essential nonlinear memory effects without redundancy