Cut Layer

The cut layer is the specific, pre-defined layer in a neural network where the model is split between participants in a VFL system. This division creates distinct local sub-models:

• Bottom models reside on feature owner devices, processing raw, private feature data.
• The top model resides with the label owner, which holds the target values. The selection of this layer is a critical design decision, balancing privacy, communication cost, and model performance.

The output of the cut layer—the intermediate output or embedding—is the only data communicated from a feature owner to the label owner during forward propagation. This design enforces data minimization.

• The intermediate output is a transformed representation, not raw data.
• It is often protected via homomorphic encryption or secure multi-party computation (MPC) protocols before transmission.
• This interface prevents direct exposure of the original private features while allowing collaborative learning.

The depth of the cut layer directly influences the privacy-utility trade-off.

• A shallow cut (early in the network) means less complex computation is done locally. The transmitted intermediate outputs may retain more information about the raw input, potentially increasing privacy risk but often yielding higher model accuracy due to more centralized learning.
• A deep cut (later in the network) means more nonlinear transformation occurs on the feature owner's device. The transmitted data is a more abstract, less invertible representation, enhancing privacy but may require more on-device compute and can sometimes hinder final model convergence.

During backpropagation, the cut layer is the point where gradients are partitioned and routed back to the respective model owners.

• The label owner computes gradients for its top model and the gradient with respect to the cut layer's output.
• This gradient is sent back (often securely) to the feature owner.
• The feature owner uses this received gradient to perform backpropagation through its local bottom model and update its parameters. This process enables vertical backpropagation without any party accessing another's full model or raw data.

The cut layer's properties dictate key system costs in VFL:

• Communication Overhead: The size and frequency of the intermediate outputs exchanged define bandwidth consumption. Larger embedding dimensions from later layers increase cost.
• Computation Overhead: Feature owners must have sufficient compute to run their portion of the model up to the cut layer. Cryptographic operations for securing the exchange add significant processing latency.
• Synchronization Point: All feature owners must compute and transmit their intermediate outputs before the label owner can proceed, making the cut layer a potential bottleneck.

Selecting and tuning the cut layer is a core optimization problem in VFL system design. Strategies include:

• Automated Layer Search: Using neural architecture search techniques to find the optimal split point for a given privacy budget and accuracy target.
• Adaptive Cutting: Dynamically adjusting the cut point based on network conditions or data distribution.
• Hybrid Splits: Employing multiple cut layers for scenarios with more than two parties, creating a chain of computation. The goal is to maximize learning efficacy while adhering to strict privacy and efficiency constraints.

A split neural network is the model architecture used in VFL where the network is divided into multiple segments, each residing on a different party. The cut layer defines the precise boundary of this split.

• Local Bottom Models: Feature owners train these on their private data.
• Top Model: The label owner (or a coordinator) typically holds this segment.
• Forward/Backward Pass: Computation flows vertically across the split, requiring secure exchange of intermediate outputs and gradients.

An intermediate output (or embedding) is the tensor produced by the forward pass up to the cut layer. This is the only data a feature owner shares during training.

• Privacy Vector: It contains a transformed, abstract representation of the raw input features.
• Communication Payload: The size and content of this output directly impact vertical communication overhead.
• Security Critical: This vector must be protected via encryption or perturbation to prevent model inversion attacks that could reconstruct raw data.

These are the distributed versions of standard neural network training passes, split at the cut layer.

• Vertical Forward Propagation: Each feature owner computes locally up to the cut layer, sends the intermediate output to the label owner, who completes the forward pass.
• Vertical Backpropagation: The label owner computes gradients for the top model and the gradient with respect to the intermediate output. This gradient is sent back to feature owners, who use it to compute gradients for their local bottom models via vertical gradient computation.

Vertical secure aggregation is a cryptographic protocol for combining model updates (e.g., gradients) from multiple feature owners without revealing any individual contribution. It operates on the vectors passed across the cut layer.

• Purpose: Prevents the label owner from inferring a single party's data from their update.
• Techniques: Often employs Secure Multi-Party Computation (MPC) or homomorphic encryption.
• Output: The label owner receives only the aggregated update, enabling safe vertical federated averaging (VFA).

A vertical training protocol is the defined sequence of steps all parties follow for one training round, with the cut layer as the coordination point.

1. Entity Alignment & Forward Pass: Align samples, compute to cut layer, share intermediate outputs.
2. Loss & Gradient Calculation: Label owner computes loss and top-model gradients.
3. Backward Pass & Secure Update: Gradients are passed back; feature owners compute local updates, which may be securely aggregated.
4. Model Update: Each party updates its segment of the split neural network. This protocol defines the vertical communication overhead and computation overhead.

This encompasses all techniques to prevent data leakage in VFL, with the cut layer as a primary attack surface. Defenses are applied to the data crossing this boundary.

• Cryptographic Protections: Homomorphic encryption allows computation on encrypted intermediate outputs. Vertical MPC enables joint computation without revealing private inputs.
• Mathematical Protections: Differential privacy adds calibrated noise to intermediate outputs or gradients before sharing.
• Goal: Ensure secure entity resolution and training reveals nothing beyond the final model's intended predictions.