Split Learning

The neural network is divided into a client-side segment and a server-side segment. The client processes raw data through initial layers, producing smashed data (intermediate activations). Only these activations—not the raw input—are transmitted to the server. The server completes the forward pass, computes the loss, and backpropagates gradients to the cut layer. The client then completes backpropagation locally. This architecture ensures the server never accesses raw data and the client never sees the full model.

For label privacy, split learning can be extended to a U-shaped architecture where the network is partitioned across three entities: the client holds the initial and final layers, while the server holds the middle layers. The client sends activations to the server, which processes them and returns intermediate gradients. The client then completes the final layers where labels reside. This prevents the server from accessing both raw inputs and corresponding labels, a critical requirement in vertically partitioned data scenarios.

Unlike federated learning, where raw gradients are shared, split learning transmits only intermediate activations and gradients at the cut layer. This significantly reduces the attack surface for gradient inversion attacks. An adversary intercepting cut-layer gradients must invert through multiple unknown server layers to reconstruct inputs. However, recent research shows that cut-layer activations can still leak information—defenses like differential privacy or activation perturbation may be applied to the smashed data for stronger guarantees.

Split learning operates sequentially: each client completes its forward pass, the server processes activations, and gradients flow back before the next client begins. This lock-step execution eliminates the need for synchronous parallel updates but introduces idle time for the server. Variants like SplitFed hybridize split learning with federated averaging to enable parallel client training while retaining privacy benefits. This trade-off between privacy and throughput is a central design consideration.

By transmitting only smashed data rather than full model gradients, split learning can reduce communication overhead compared to federated learning for deep networks. The cut layer's activation dimensions are typically smaller than the total parameter count. However, the sequential nature requires multiple rounds of communication per batch. Techniques like activation compression and quantization further reduce bandwidth, making split learning viable for resource-constrained edge devices with limited uplink capacity.

Split learning naturally accommodates heterogeneous client capabilities. The computationally intensive server-side layers can run on powerful cloud infrastructure, while lightweight client-side layers execute on edge devices. This enables training of large models that would exceed individual client memory or compute budgets. The cut layer can be dynamically adjusted based on client resources—deeper cuts shift more computation to the server, enabling participation from low-power IoT devices.

A mathematical framework providing provable privacy guarantees by injecting calibrated noise into computations. The privacy budget (epsilon) quantifies the maximum information leakage, with smaller values enforcing stronger guarantees. In the context of distributed training, DP-SGD clips per-sample gradients and adds Gaussian noise before sharing. Split learning can be combined with differential privacy by applying noise to the smashed data or gradients at the cut layer, offering a composable privacy guarantee that complements its architectural separation.