Semantic Encoder

Unlike a traditional source encoder that aims for bit-level fidelity, the semantic encoder performs lossy compression with a purpose. It discards all information irrelevant to the receiver's specific task. For example, when transmitting an image for an object detection task, the encoder will preserve features like object shape and class while completely discarding background texture and lighting details that are not needed for identification. This is often formalized using the Variational Information Bottleneck (VIB) principle, which learns a latent representation Z that is maximally predictive of the task Y while minimizing mutual information with the input X.

To achieve extreme compression, the encoder relies on a shared Semantic Knowledge Base (SKB) with the receiver. It does not need to transmit common-sense facts or redundant context. Instead, it transmits only the novel, salient semantic features. The encoder leverages this shared ontology to resolve ambiguity at the source.

• Shared Context: Both sides know 'a car has four wheels'.
• Efficiency: Only the car's unique color and position are encoded.
• Disambiguation: Uses the SKB to select the correct meaning of polysemous words before transmission.

To interface with digital hardware, the continuous-valued latent vector produced by the neural network must be discretized into a finite set of symbols. Standard quantization is a non-differentiable step that blocks gradient flow during backpropagation. Semantic encoders overcome this using soft quantization or stochastic discretization techniques during training.

• Straight-Through Estimator (STE): Approximates the gradient during the backward pass.
• Softmax Relaxation: Replaces hard argmax with a temperature-controlled softmax.
• Vector Quantization (VQ): Learns a discrete codebook of semantic primitives.

The encoder must be resilient to semantic noise, which corrupts meaning rather than bits. A small perturbation to the input (e.g., a tiny sticker on a stop sign) should not cause the encoder to produce a latent code that the decoder interprets as a 'yield' sign. Training often involves adversarial learning, where the encoder is hardened against worst-case input perturbations designed to maximize semantic distortion. This ensures the extracted meaning is stable under minor physical or environmental variations.

The architecture of a semantic encoder is heavily biased by the modality of the source data. It uses domain-specific inductive biases to extract high-level features efficiently.

• Text: Uses Transformer blocks with multi-head self-attention to capture long-range syntactic and semantic dependencies.
• Images/Videos: Employs Convolutional Neural Networks (CNNs) or Vision Transformers (ViTs) to build hierarchical feature maps from edges to objects.
• Radio Frequency (RF): Processes raw IQ samples using complex-valued 1D convolutions to learn modulation-agnostic features directly from the waveform.