DNN Splitting

The fundamental architecture of DNN splitting divides the model into two distinct segments. The head executes on the local device, processing raw sensor data through initial feature extraction layers. The tail executes on a powerful edge server, completing the computationally intensive deeper layers. This division ensures that raw, potentially sensitive data never leaves the device; only an abstract, compressed feature representation is transmitted over the network.

The partition point is not arbitrary. It is chosen at a bottleneck layer—an intermediate layer with a highly compressed feature map. Key selection criteria include:

• Minimizing transmission size: The output tensor at this layer must be significantly smaller than the raw input.
• Preserving information entropy: The compressed features must retain sufficient discriminative information for the server-side tail to achieve high accuracy.
• Balancing compute: The computational load is distributed such that the device handles only what its silicon allows within the power budget.

A primary driver for DNN splitting is data privacy. By executing the initial layers on-device, the system ensures that raw user data—such as images, audio streams, or location data—is never transmitted to an external server. The server only receives an obfuscated, high-dimensional feature vector. This makes reverse-engineering the original input computationally infeasible, providing a robust technical safeguard for compliance with regulations like GDPR and HIPAA.

DNN splitting strategies are categorized by their adaptability:

• Static Partitioning: The split point is fixed at design time based on a known device profile. This is simple but brittle under variable network conditions.
• Dynamic Partitioning: A scheduler monitors real-time metrics—such as channel state information, device CPU load, and battery level—and shifts the partition point adaptively. This ensures the system meets a strict latency budget even as conditions fluctuate.

To further reduce the latency overhead of transmitting the bottleneck activations, lossy and lossless compression techniques are applied directly to the feature tensor. Common methods include:

• Quantization: Reducing the precision of activations from FP32 to INT8 or lower.
• Entropy coding: Applying Huffman or arithmetic coding to the serialized tensor.
• Dimensionality reduction: Using PCA or an autoencoder sub-layer to project features into an even lower-dimensional space before transmission.

DNN splitting is a specific implementation of the broader split computing paradigm. While split computing refers to any arbitrary distribution of a model graph, DNN splitting specifically targets a single, clean cut at a bottleneck. This contrasts with early exit architectures, where the device runs the full model but can short-circuit execution, and federated learning, where training—not inference—is distributed. DNN splitting is purely an inference-time optimization for latency and privacy.