Learned Constellation

The neural network directly optimizes the complex-valued positions of each constellation point in the I/Q plane. Unlike fixed QAM, points are not constrained to a rectangular grid. The optimizer places points to maximize mutual information under a specific signal-to-noise ratio (SNR) and channel model, often resulting in non-uniform, circular, or spiral arrangements that better match the channel's capacity-achieving distribution.

Rather than moving points, the network learns a non-uniform probability mass function over a fixed set of constellation points. A distribution matcher encodes bits into symbols such that low-energy inner points are transmitted more frequently than high-energy outer points. This creates a Gaussian-like symbol distribution, providing a shaping gain of up to 1.53 dB over uniform QAM at high spectral efficiencies.

The constellation is treated as a trainable parameter tensor within a larger channel autoencoder. During training, a differentiable channel model—either a mathematical approximation or a neural surrogate—allows gradients from the receiver's loss to flow back through the channel to the constellation points. This enables joint optimization of the constellation and the receiver's decision boundaries using standard stochastic gradient descent.

A single neural network can learn to output different constellations based on the current channel state information (CSI) or estimated SNR. The constellation becomes a function of the channel, morphing its geometry in real-time to maintain optimal throughput. For example, at low SNR, points cluster for robustness; at high SNR, they spread out to maximize rate.

When the differentiable channel model includes power amplifier non-linearity, phase noise, or I/Q imbalance, the learned constellation automatically pre-distorts to compensate. The resulting point cloud may appear asymmetric or warped, but it maximizes the receiver's ability to decode correctly after the real hardware impairments are applied, outperforming a standard QAM constellation followed by a separate pre-distorter.

The training objective is often a mutual information neural estimator (MINE) or a variational lower bound on the channel capacity. Rather than minimizing symbol error rate, the network directly maximizes the amount of information that can be reliably transmitted per channel use. This aligns the optimization with the fundamental Shannon limit, producing constellations that are provably capacity-approaching for non-AWGN channels.

An optimization technique that assigns a non-uniform probability distribution to constellation points to approach the Shannon capacity limit. Unlike geometric shaping, which moves point locations, probabilistic shaping uses a distribution matcher to transmit low-energy inner symbols more frequently than high-energy outer symbols.

• Achieves shaping gain of up to 1.53 dB on AWGN channels
• Implemented via constant composition distribution matching (CCDM)
• Often combined with learned constellations for joint geometric-probabilistic shaping
• Key enabler for flexible-rate optical and wireless systems

A mathematical or neural surrogate model that allows gradients to backpropagate from the receiver loss function through the channel to the transmitter parameters. This is the critical enabler for learning constellation geometries via gradient descent.

• Can be a statistical model (e.g., differentiable AWGN, Rayleigh fading)
• Can be a GAN-based channel surrogate for real-world hardware-in-the-loop training
• Must accurately capture non-linear impairments like PA distortion and phase noise
• The quality of the learned constellation depends directly on channel model fidelity

The direct optimization of constellation point positions in the I/Q plane to maximize mutual information for a specific signal-to-noise ratio and channel model. Unlike probabilistic shaping, geometric shaping modifies the location of points rather than their frequency of use.

• Produces non-regular lattice structures optimized for non-AWGN channels
• Can be combined with neural demappers for joint geometric-demapper optimization
• Particularly effective for non-linear fiber channels and satellite links
• Learned via gradient descent on a mutual information lower bound

A receiver component that uses a neural network to compute soft bit estimates (log-likelihood ratios) directly from received I/Q symbols. When paired with a learned constellation, the demapper learns a non-linear decision boundary that outperforms classical maximum-likelihood demapping in the presence of hardware impairments.

• Learns to compensate for residual phase noise and I/Q imbalance
• Can be trained jointly with the constellation geometry
• Replaces Euclidean distance-based demapping with learned manifolds
• Critical for extracting full performance from irregular constellations

A neural network trained to estimate the mutual information between channel input and output, serving as a differentiable optimization objective for constellation design. MINE provides a tractable lower bound on the achievable information rate without requiring closed-form capacity expressions.

• Uses a dual representation of KL-divergence (Donsker-Varadhan bound)
• Enables optimization of constellations for channels with unknown or intractable capacity
• Can incorporate practical constraints like peak-to-average power ratio (PAPR)
• Replaces heuristic shaping metrics with information-theoretic objectives