Semantic Autoencoder

Unlike traditional separate source and channel coding, the semantic autoencoder jointly optimizes the encoder and decoder as a single neural network. The bottleneck layer forces the model to discard task-irrelevant information, learning a maximally compressed semantic feature vector. This is trained via backpropagation to minimize a loss function that balances reconstruction fidelity with the compression rate, often using a Variational Information Bottleneck (VIB) objective.

The bottleneck latent space is engineered to capture only the features relevant to a specific downstream task, not pixel-perfect reconstruction. For example, in image transmission for classification, the bottleneck preserves features like object shape and texture while discarding background noise.

• Classification tasks: Bottleneck retains categorical features.
• Segmentation tasks: Bottleneck preserves spatial boundaries.
• Control tasks: Bottleneck encodes actionable state representations.

Semantic autoencoders can be trained to map symbols directly to channel inputs, making them inherently robust to thermal noise, fading, and interference. By modeling the channel as a non-trainable stochastic layer during training, the decoder learns to interpret corrupted latent vectors. This replaces explicit Forward Error Correction (FEC) with learned resilience, maintaining high task accuracy even at low Signal-to-Noise Ratios (SNR).

A semantic autoencoder implements Deep JSCC by collapsing the traditional modular pipeline into a single model. The encoder directly maps source pixels or waveforms to complex-valued channel symbols, bypassing separate compression and modulation blocks.

• Bandwidth Adaptation: The bottleneck dimension directly controls the transmission rate.
• Graceful Degradation: Performance degrades smoothly with worsening channel conditions, avoiding the cliff effect of digital systems.

Advanced semantic autoencoders learn disentangled latent spaces where individual dimensions correspond to independent, human-interpretable generative factors. This allows for semantic manipulation—modifying a single attribute of the transmitted meaning without affecting others. Techniques like β-VAE or FactorVAE enforce this statistical independence, enabling robust communication even when the transmitter and receiver have mismatched background knowledge.

The architecture can be extended to fuse heterogeneous data streams—such as LiDAR point clouds, camera images, and radar signatures—into a unified semantic latent vector. A shared bottleneck forces the network to learn a modality-invariant representation of the scene's meaning. This is critical for autonomous systems where transmitting a single, compact semantic state is far more efficient than streaming raw sensor data from multiple sources.