Bottleneck Layer

The bottleneck must aggressively reduce the dimensionality of the feature map while preserving the semantic content necessary for the downstream task. This is achieved through a drastic reduction in the number of channels or hidden units.

• Goal: Minimize the bitrate required for transmission over the wireless link.
• Mechanism: Often implemented via a 1x1 convolution or a linear projection layer.
• Trade-off: Too much compression strips essential features, causing a drop in the tail model's accuracy.

The bottleneck should be placed at the partition point where low-level spatial features have been abstracted into high-level, compact semantic concepts. This is typically after the feature extractor but before the classifier head.

• Rationale: Transmitting raw pixels or low-level edges is bandwidth-prohibitive.
• Ideal State: The activations at this layer represent a latent code that is robust to input noise but cheap to transmit.
• Example: In a ResNet-50, the output of the conv4 block often serves as an effective bottleneck.

The activation distribution at the bottleneck must be amenable to post-training quantization (e.g., INT8) without significant accuracy degradation. This allows the compressed feature tensor to be represented with fewer bits.

• Requirement: A narrow, well-behaved dynamic range with minimal outliers.
• Benefit: Reduces transmission size by up to 4x compared to FP32.
• Technique: Applying entropy coding on top of quantized latents can further approach the theoretical compression limit.

An ideal bottleneck produces a general-purpose representation that can be reused by multiple downstream heads without retraining the on-device front-end. This supports multi-task learning at the edge.

• Strategy: Train the head (on-device) to produce a universal feature space.
• Validation: The same bottleneck output should enable object detection, segmentation, and depth estimation simultaneously.
• Contrast: A task-specific bottleneck limits the utility of the split architecture to a single function.

The bottleneck must enforce a heavy front-end and a lightweight tail. The majority of the computational load (FLOPs) should reside on the edge server, while the on-device head remains extremely lean.

• Metric: The head model should consume < 10% of the total model FLOPs.
• Purpose: Enables inference on severely resource-constrained IoT sensors.
• Implementation: Use depthwise separable convolutions or MobileNet blocks for the head, reserving dense layers for the server-side tail.

The bottleneck serves as a privacy and security barrier. By transmitting only intermediate activations (smashed data) rather than raw input, it prevents the server from reconstructing the original user data with high fidelity.

• Privacy Mechanism: The compressed latent representation obscures low-level details.
• Security Note: While not cryptographically secure, it raises the bar against inversion attacks compared to raw data offloading.
• Enhancement: Can be combined with differential privacy noise injection at the cut layer.