Neural Network Equalizer

Unlike linear minimum mean square error (MMSE) equalizers, neural network equalizers can model and invert non-linear channel impairments caused by power amplifier saturation, IQ imbalance, and phase noise. By employing activation functions like ReLU or tanh, these models learn complex decision boundaries that separate distorted symbol clusters in the I/Q plane, significantly reducing the error vector magnitude (EVM) in scenarios where traditional Volterra series methods become computationally intractable.

Neural equalizers can be trained end-to-end with the channel decoder, performing joint symbol recovery and error correction in a single unified model. This approach, often implemented with a bidirectional recurrent neural network (Bi-RNN) or a transformer, bypasses the information bottleneck created by hard-decision slicing. The model outputs soft log-likelihood ratios (LLRs) directly, maximizing bit-wise mutual information and approaching the performance of maximum-likelihood sequence estimation (MLSE) without its exponential complexity.

A hybrid approach that unrolls the iterations of a classical algorithm like the iterative shrinkage-thresholding algorithm (ISTA) or approximate message passing (AMP) into a neural network. Each layer corresponds to one iteration, with learnable parameters replacing hand-crafted step sizes and thresholds. This architecture, known as Learned ISTA (LISTA) , retains the interpretability and convergence guarantees of the model-based method while achieving the performance of a purely data-driven network with far fewer parameters and faster inference.

Recurrent architectures like Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRU) inherently capture temporal dependencies in the channel impulse response. By processing sequences of received symbols, these equalizers track rapid channel variations caused by Doppler spread in high-mobility scenarios (e.g., high-speed rail or V2X communication). Meta-learning frameworks further enable few-shot adaptation, where the model fine-tunes to a new channel environment using only a handful of pilot symbols without full retraining.

Neural network equalizers can operate in blind mode without explicit pilot sequences by exploiting the statistical properties of the transmitted signal, such as constant modulus or finite alphabet constraints. Autoencoder-based architectures learn to reconstruct the transmitted constellation directly from unlabeled received data. In semi-blind settings, a small number of pilots provide an initial coarse estimate, which the network refines using the payload data, dramatically improving spectral efficiency by reducing pilot overhead.

Dedicated complex-valued neural networks (CVNNs) process in-phase (I) and quadrature (Q) components as a single complex entity using Wirtinger calculus for backpropagation. This preserves the phase information critical for coherent detection and avoids the suboptimal independent processing of real and imaginary parts. CVNN-based equalizers naturally handle complex-valued modulation schemes like QPSK and 256-QAM, learning richer representations of the wireless channel's complex baseband response.

A paradigm that replaces the entire traditional receiver chain—channel estimation, equalization, and demodulation—with a single end-to-end trained neural network. Unlike a standalone neural equalizer that slots into an existing pipeline, DeepRx learns a joint mapping from raw I/Q samples directly to log-likelihood ratios or decoded bits. This eliminates the information bottleneck caused by modular, sub-optimal processing blocks. The architecture typically uses a convolutional neural network backbone to exploit time-frequency correlations, treating the received OFDM grid as a 2D image. Training requires diverse channel models and hardware impairments to ensure generalization.

A design methodology that bridges classical algorithms and deep learning, also known as deep unfolding. An iterative optimizer like ISTA is unrolled into a fixed number of neural network layers, where each layer corresponds to one algorithm iteration. Crucially, hand-crafted parameters (step sizes, thresholds) become learnable weights optimized via backpropagation. For equalization, this yields architectures like Learned ISTA or unfolded orthogonal approximate message passing (OAMP). The result is a model with the interpretability and efficiency of a classical solver but the performance of a data-driven method, requiring far fewer parameters than a black-box DNN.

Standard neural networks operate on real numbers, treating I and Q components as separate input channels. This discards the inherent phase structure of wireless signals. Complex-valued neural networks (CVNNs) use complex weights, biases, and activation functions, with backpropagation performed via Wirtinger calculus. This allows the network to naturally represent multiplication and phase rotation—operations fundamental to equalization. CVNNs often achieve superior performance with fewer parameters for tasks like channel equalization and beamforming because they learn representations that respect the underlying algebraic structure of the physical layer.

A hybrid architecture that integrates the structural flow of the classical Kalman filter with small neural networks. The Kalman filter's predict-update loop is preserved, but the components that require explicit system knowledge—the process noise covariance and measurement noise covariance—are replaced by learned neural modules. This allows KalmanNet to track rapidly time-varying channels without a mathematical model of the channel dynamics. It learns the statistics of temporal evolution directly from pilot sequences, making it highly effective for equalization in high-mobility scenarios like V2X communication where Doppler shifts are severe.

The self-attention mechanism from transformer architectures is being adapted for physical layer tasks. In the context of equalization, attention allows the model to dynamically weigh the importance of different time steps, frequency subcarriers, or propagation paths. For example, an attention-based equalizer can learn to focus on the most reliable received symbols when making a decision, effectively performing soft interference cancellation. This approach excels in doubly-selective channels where both time and frequency dispersion are severe, as the attention matrix can capture long-range dependencies that convolutional kernels might miss.

Denoising diffusion probabilistic models (DDPMs) are emerging as powerful generative priors for wireless channels. A diffusion model is trained to learn the complex, high-dimensional distribution of realistic channel responses. During inference, it can be used as a learned prior for maximum a posteriori (MAP) estimation, refining a coarse pilot-based estimate into a high-fidelity channel reconstruction. This provides a robust foundation for subsequent equalization. The iterative denoising process naturally handles noisy observations and can even generate plausible channel realizations from heavily corrupted or incomplete pilot data.