Split Learning

The model is physically split into a client-side segment and a server-side segment. The client processes raw data through initial layers and outputs a compressed intermediate representation called smashed data. Only this activation tensor, not the original input, ever leaves the client device. The server completes the forward pass, computes the loss, and backpropagates gradients only through its own layers. The client receives gradients for its segment alone, never exposing full end-to-end gradient paths that could be exploited by gradient inversion attacks.

The smashed data is the intermediate activation tensor transmitted from client to server at the cut layer. Key properties:

• Dimensionality reduction: Typically much smaller than raw input, creating a natural information bottleneck
• Lossy compression: Fine-grained features necessary for input reconstruction are discarded
• No label exposure: Labels remain on the server side, preventing the client from accessing ground truth This design directly counters feature reconstruction and deep leakage from gradients (DLG) attacks by ensuring the transmitted tensor lacks sufficient mutual information with the original input.

In a U-shaped split learning setup, the model is partitioned across three segments: an initial client encoder, a server core, and a final client decoder. The data flows:

1. Client encodes input → sends smashed data to server
2. Server processes through deep layers → sends intermediate back to client
3. Client decodes final output This configuration is particularly effective for autoencoder-based defenses and tasks where both input and output must remain private on the client. The server never sees raw inputs or final predictions, only abstract intermediate representations.

Unlike federated learning, where labels reside on the client, split learning keeps labels exclusively on the server. The client never receives ground truth labels, only the gradients necessary to update its local segment. This prevents:

• Membership inference attacks that exploit label availability
• Model inversion attacks that use label-conditioned gradient analysis
• Property inference from label distributions For supervised learning tasks in regulated industries, this architectural separation provides a critical privacy guarantee that complements differential privacy mechanisms.

A key limitation of vanilla split learning is sequential client-server communication. Each training step requires:

• Client forward pass → send smashed data
• Server forward + backward pass → send client gradients
• Client backward pass and weight update This round-trip latency makes split learning slower than federated learning for large client populations. Mitigations include SplitFed (parallelizing client updates) and overlap techniques that pipeline computation with communication. For latency-sensitive deployments, this overhead must be weighed against the superior privacy guarantees.

Split learning inherently resists gradient inversion attacks because:

• The server never receives raw gradients from the client's data—only smashed activations
• The client never receives the server's full gradient path—only truncated gradients for its own layers
• The cut layer acts as an information bottleneck, discarding pixel-level details Combined with per-sample gradient clipping and DP-SGD on the server side, split learning provides layered defenses. Research shows that even advanced gradient matching techniques fail to reconstruct recognizable inputs from smashed data alone, making this architecture a robust choice for privacy-preserving collaborative training.