Semantic Distortion

Semantic distortion is not a universal metric—it is defined relative to a specific receiver task. A distortion that ruins an image classifier's accuracy may be irrelevant for an object detector.

• Measured in the latent feature space of a task-specific neural network
• A distorted signal may have a high bit-error rate but zero semantic distortion if the meaning is preserved
• Example: A blurry stop sign image with 30% pixel corruption may still be classified correctly, yielding low semantic distortion

Traditional distortion metrics like Mean Squared Error (MSE) or Bit Error Rate (BER) operate at the symbol level. Semantic distortion operates at the interpretation level, making it fundamentally different.

• A perfectly received bit sequence can carry high semantic distortion if the context is misinterpreted
• Conversely, a heavily corrupted signal can carry zero semantic distortion if the receiver correctly infers the intended meaning
• This decoupling from physical-layer metrics is the core innovation of semantic communication systems

Semantic distortion is often formalized as the distance between latent representations in a shared semantic space. Common formulations include:

• Cosine distance between the transmitted semantic vector z and the received vector ẑ
• Kullback-Leibler (KL) divergence when representations are probabilistic distributions
• Task-loss delta: The increase in the receiver's task-specific loss function caused by channel impairments
• These metrics are differentiable, enabling end-to-end optimization of the entire communication pipeline

Multiple factors contribute to semantic distortion across the communication pipeline:

• Channel noise: Physical-layer impairments that corrupt transmitted semantic symbols
• Mismatched knowledge bases: Transmitter and receiver using different ontologies or background knowledge for interpretation
• Compression artifacts: Information loss from aggressive semantic encoding that discards contextually relevant features
• Adversarial perturbations: Maliciously crafted noise designed to cause specific misinterpretations at the receiver
• Domain shift: Deployment in environments with statistical properties different from the training distribution

A critical distinction exists between perceptual distortion (how a signal looks or sounds to a human) and semantic distortion (what it means to a machine).

• Generative models can reconstruct perceptually flawless images that are semantically wrong—e.g., a realistic face with incorrect identity
• Semantic communication systems optimize for semantic fidelity, not perceptual quality
• This distinction is why traditional image quality metrics like PSNR and SSIM are inadequate for evaluating semantic systems

Semantic communication introduces a new distortion-rate tradeoff distinct from Shannon's classical formulation. The goal is to minimize semantic distortion given a channel capacity constraint.

• The semantic rate-distortion function defines the minimum rate required to achieve a target semantic distortion level
• This function is always lower than or equal to the classical rate-distortion function, as semantic encoding discards task-irrelevant information
• Practical systems approach this bound using variational information bottleneck methods and learned compression