Waveform Learning

Unlike traditional systems that design transmit and receive filters independently, waveform learning treats the entire pulse shaping and matched filtering chain as a single differentiable neural network. The transmitter learns a complex baseband pulse, while the receiver learns a corresponding filter kernel. Both are updated via backpropagation through a stochastic channel model, ensuring the pair is globally optimal for the target metric—typically minimizing bit error rate (BER) or maximizing spectral efficiency. This co-design eliminates the mismatch loss inherent in modular architectures.

A critical advantage of learned waveforms is the ability to directly optimize for spectral containment. By adding a penalty term to the loss function that measures adjacent channel leakage ratio (ACLR) or power outside the allocated bandwidth, the neural network discovers pulse shapes that naturally suppress side lobes. This learned shaping often outperforms classical raised-cosine or root-raised-cosine filters, achieving steeper spectral roll-off without the need for additional digital pre-distortion or guard bands.

Waveform learning does not assume an ideal additive white Gaussian noise (AWGN) channel. The neural network is trained on real or simulated channel profiles—including multipath fading, Doppler shift, and non-linear hardware impairments. The resulting pulse shape is implicitly tuned to the channel's delay spread and frequency selectivity. For a frequency-selective channel, the learned waveform may naturally incorporate a form of implicit equalization, pre-distorting the pulse to counteract known dispersion.

End-to-end training requires a channel model that permits gradient flow from the receiver loss back to the transmitter parameters. This is achieved using a differentiable channel surrogate—either a mathematical approximation of the physical layer or a pre-trained neural channel model. The surrogate captures key impairments like phase noise, IQ imbalance, and power amplifier non-linearity. During inference, the learned waveform generalizes to the real hardware channel, having been optimized against a high-fidelity digital twin.

In multi-carrier systems like OFDM or filter bank multi-carrier (FBMC), waveform learning can replace the entire prototype filter design. A neural network learns the optimal time-frequency localization for each subcarrier, balancing the trade-off between inter-symbol interference (ISI) and inter-carrier interference (ICI). This is particularly valuable in 5G NR and future 6G systems where mixed numerologies and asynchronous transmissions demand flexible, non-orthogonal pulse shapes.

Learned waveforms can incorporate real-world power amplifier (PA) constraints directly into the optimization loop. By including a differentiable PA model—such as a Rapp or Saleh model—the network learns a pulse shape with a reduced peak-to-average power ratio (PAPR) and minimal distortion when operating near saturation. This hardware-in-the-loop approach produces waveforms that are not only spectrally efficient but also practical for energy-constrained transmitters, reducing the need for aggressive digital pre-distortion (DPD).

The overarching framework that jointly optimizes a transmitter and receiver as a single deep neural network. Waveform learning is a specific application of this paradigm, where the autoencoder discovers an optimal pulse shape and matched filter pair directly from data, replacing traditional root-raised cosine filters with learned, channel-specific waveforms that minimize inter-symbol interference (ISI).

A critical enabler for waveform learning that allows gradients to backpropagate from the receiver loss to the transmitter parameters. This can be a mathematical surrogate or a neural channel model trained on real measurements. Without a differentiable channel, the end-to-end optimization of a transmit pulse shape cannot be performed via standard gradient descent, making this a prerequisite for joint transceiver learning.

A complementary technique where a neural network optimizes the positions and probabilities of constellation points in the I/Q plane. While waveform learning focuses on the pulse shape, learned constellation design optimizes the symbol mapping. Together, they form a fully learned modulation scheme that maximizes throughput for a specific channel profile, moving beyond fixed QAM and RRC combinations.

An end-to-end learning framework that jointly optimizes the transmitter and receiver for an orthogonal frequency-division multiplexing system. In this context, waveform learning extends to optimizing the windowing function and cyclic prefix configuration, learning to mitigate inter-carrier interference and peak-to-average power ratio (PAPR) directly from data rather than relying on fixed filter designs.

A transceiver architecture that integrates known physical layer algorithmic structures as non-trainable layers within a neural network. For waveform learning, this means embedding the Fast Fourier Transform (FFT) or pulse shaping filter structure directly into the model, improving data efficiency and interpretability while still allowing the network to learn optimal filter coefficients for a given channel.

A neural network receiver that jointly performs channel equalization and symbol detection directly from a received signal sequence without requiring explicit pilot-based channel estimation. This approach implicitly learns a matched filter that is robust to unknown channel conditions, making it a practical deployment of waveform learning principles in scenarios where channel state information (CSI) is unavailable or rapidly changing.