CSI Entropy Coding

CSI Entropy Coding operates on the quantized bit stream produced by a CSI compressor, guaranteeing perfect reconstruction of the quantized values. Unlike the lossy autoencoder stage that introduces distortion, entropy coding exploits statistical redundancies in the bit sequence—such as non-uniform symbol probabilities and inter-bit correlations—to achieve additional compression without any information loss. Common techniques include arithmetic coding, Huffman coding, and context-adaptive binary arithmetic coding (CABAC).

After quantization, CSI matrices in the angular-delay domain exhibit significant sparsity, with most elements concentrated near zero. This produces a highly skewed probability distribution where a small set of quantization indices dominate. Entropy coders leverage this by assigning shorter codewords to high-probability symbols and longer codewords to rare ones, achieving compression ratios proportional to the entropy of the source distribution.

Modern CSI entropy coding systems employ neural networks as probability estimators to predict the likelihood of each quantized CSI element conditioned on previously encoded values. Architectures include:

• PixelCNN-style autoregressive models for spatial context
• Transformer-based hyperpriors for global dependencies
• Factorized entropy models with learned cumulative distribution functions These learned models provide tighter bounds on the true entropy than hand-crafted statistical models.

In learned CSI compression pipelines, entropy coding is tightly integrated with the variational autoencoder (VAE) framework. The encoder outputs a latent representation that is quantized and then entropy-coded using a learned prior. The rate-distortion loss function explicitly includes the cross-entropy of the latent code as the rate term, enabling end-to-end optimization of both the compressor and the entropy model simultaneously.

CSI feedback occurs in a time-slotted manner, with successive channel snapshots exhibiting strong temporal correlation. Advanced entropy coders exploit this by conditioning probability estimates on previously transmitted CSI frames, using recurrent neural networks or temporal context models. This reduces the effective entropy of the current frame and yields additional compression gains in low-mobility scenarios where the channel coherence time spans multiple feedback intervals.

The 3GPP 5G NR standard specifies entropy coding as part of the CSI report encoding chain for Type-II codebook feedback. The quantized coefficients undergo run-length encoding followed by Huffman coding to compress the sparse coefficient maps. For AI/ML-enhanced CSI feedback under 3GPP Release 18 and beyond, learned entropy models are being evaluated to replace traditional codebook-based compression, with performance measured by NMSE vs. feedback bit count trade-offs.

The discretization bridge between continuous-valued compression and binary transmission. Quantization maps floating-point latent vectors to a finite set of discrete symbols, introducing irreducible quantization error.

• Uniform quantization: Equal-width bins, simple but suboptimal for non-uniform distributions
• Non-uniform quantization: Lloyd-Max algorithm, minimizes mean squared error
• Vector quantization: Jointly quantizes blocks of values, approaching rate-distortion bounds
• Deep quantizers: Learned codebooks trained end-to-end with the autoencoder

The information-theoretic foundation that defines the theoretical lower bound for lossless compression. Shannon's theorem states that the minimum average codeword length is the entropy of the source distribution.

• Entropy H(X): The irreducible bit cost for a given symbol distribution
• Huffman coding: Optimal when symbol probabilities are powers of 1/2
• Arithmetic coding: Approaches entropy bound for any distribution
• Asymmetric Numeral Systems (ANS): Modern alternative combining Huffman speed with arithmetic precision

The state-of-the-art entropy coder used in H.264/H.265 video compression, directly applicable to CSI bitstreams. CABAC adapts probability models based on previously encoded symbols, exploiting spatial and statistical context.

• Binarization: Maps non-binary symbols to binary sequences
• Context modeling: Selects probability model based on neighboring syntax elements
• Adaptive update: Probabilities evolve after each encoded bit
• CSI application: Models angular-delay domain sparsity patterns for superior compression

An end-to-end paradigm that collapses compression, quantization, and entropy coding into a single neural network trained to map CSI directly to channel input symbols. This bypasses the traditional modular pipeline entirely.

• Unified objective: Minimize reconstruction error under channel capacity constraints
• Learned constellation: Network discovers optimal modulation scheme
• No explicit entropy model: Compression emerges from bottleneck architecture
• Advantage: Outperforms separate source-channel coding at low SNR

The fundamental trade-off framework governing CSI feedback design. Rate-distortion theory quantifies the minimum bit rate R required to achieve a given reconstruction quality D, defining the Pareto frontier for compression systems.

• R(D) function: Theoretical lower bound on bit rate for distortion D
• Lagrangian optimization: Minimize D + λR to sweep the rate-distortion curve
• CSI metric: NMSE or cosine similarity as distortion measure
• Practical impact: Guides codebook size selection and quantization granularity