Vertical Training Protocol

The initial, privacy-preserving step where parties identify the overlapping set of entities (e.g., customer IDs) across their vertically partitioned datasets without revealing non-intersecting records. This is typically achieved using a Private Set Intersection (PSI) protocol. The output is a cryptographically secure, aligned sample index used for all subsequent training computations.

The distributed forward pass of a split neural network. Each feature owner computes the network's forward pass on their local features up to a predefined cut layer. The output of this layer, called the intermediate output or embeddings, is then securely transmitted (often encrypted) to the label owner or the next party in the chain to complete the forward pass and compute the loss.

The label owner computes the initial loss gradient and propagates it backward to the cut layer. The gradient with respect to the intermediate output is then sent back to each feature owner. Each party uses this received gradient to compute gradients for their own portion of the model (their local layers) and update their local model parameters. This step ensures the label owner never sees raw features, and feature owners never see raw labels.

A cryptographic step to combine model updates (e.g., gradients or updated intermediate parameters) from multiple parties without exposing any single party's contribution. Techniques include:

• Secure Multi-Party Computation (MPC)
• Homomorphic Encryption
• Differential Privacy noise addition This prevents inference attacks and provides formal privacy guarantees, forming the core of a privacy-preserving vertical FL system.

The protocol must define the message-passing sequence, failure handling, and state synchronization between heterogeneous parties. This manages vertical communication overhead and vertical computation overhead. Key considerations include:

• Handling dropped or slow clients
• Version control for model parts
• Ensuring all parties are aligned on the current global model state before proceeding to the next round.

The protocol for making predictions after training. A vertical inference protocol requires the same secure collaboration: feature owners compute their portion of the forward pass on new data and send the encrypted intermediate result to the label owner, who completes the computation and returns the final prediction. This ensures data privacy is maintained during model usage.

This is the foundational architectural pattern for VFL. The model is vertically split at a designated cut layer. Feature owners compute the forward pass up to this layer, generating intermediate outputs (embeddings). These outputs are securely sent to the label owner, who completes the forward pass, calculates the loss, and initiates backpropagation. The protocol defines the exact sequence for exchanging these intermediate results and gradients.

• Example: A bank (label owner with loan default labels) collaborates with a retailer (feature owner with purchase history). The retailer's local model processes transaction data and sends encrypted embeddings to the bank, which combines them with credit data to predict risk.

A protocol pattern where cryptographic privacy is enforced. Feature owners encrypt their intermediate outputs or gradients using homomorphic encryption before sending them to the label owner or an aggregator. Computations (like summation for aggregation) are performed directly on the ciphertext. This prevents any party from inspecting the raw data contributions of others.

• Key Mechanism: The label owner can compute the aggregated gradient needed for its model portion without decrypting any single client's update.
• Use Case: Essential in highly regulated scenarios (e.g., healthcare VFL) where even intermediate feature representations must remain confidential.

A pattern that distributes the training computation itself across parties using secret sharing. Instead of sending intermediate outputs, the data and model weights are secret-shared among participants. The protocol coordinates a series of local computations and secure exchanges that collectively compute the forward pass, loss, and gradients, with no single party ever reconstructing another's raw input.

• Contrast with HE: Often more communication-efficient than pure homomorphic encryption but involves more complex coordination.
• Implementation: Common frameworks like PySyft and TF Encrypted provide MPC backends for implementing such VFL protocols.

A critical pre-training protocol phase. Before model updates begin, parties must identify their common samples (entities) without revealing their full datasets. This is achieved via a Private Set Intersection (PSI) protocol.

• Process: Parties (e.g., a hospital and an insurance provider) encode their private user IDs. Through a cryptographic protocol, they learn only the intersection of IDs (the aligned entities) and nothing about non-matching IDs.
• Foundation: This aligned ID list becomes the basis for all subsequent sample-wise operations in the training protocol, ensuring all parties compute on the same batch of entities.

A pattern that adds formal, mathematical privacy guarantees by integrating differential privacy (DP) mechanisms into the training loop. Feature owners or the label owner add carefully calibrated noise to their shared quantities (gradients or intermediate outputs).

• Protocol Step: After computing a local update, a party adds noise from a distribution (e.g., Gaussian) before sharing it. The protocol specifies the privacy budget (epsilon) and the noise scale.
• Trade-off: Provides strong privacy against inference attacks but can slow model convergence and reduce final accuracy—a managed trade-off defined in the protocol parameters.

A common pattern for vertical federated learning with tree-based models (e.g., Gradient Boosting Decision Trees). The protocol coordinates the secure construction of a global tree by finding the best split points across federated features.

• Mechanism: The label owner calculates gradients and sends them to feature owners. Each feature owner computes potential split gains for its features locally. Using a secure aggregation sub-protocol (like MPC), parties compare these encrypted gains to jointly decide the globally best split without revealing individual feature histograms.
• Framework Example: FederatedForest and SecureBoost are implementations of this protocol pattern.

The foundational model architecture for Vertical Federated Learning (VFL). The neural network is vertically partitioned into segments, with each segment residing on a different data-owning party.

• Local Bottom Models: Feature owners hold the initial layers that process their private feature subsets.
• Top Model: The label owner (or a coordinator) typically holds the final layers that consume combined intermediate outputs to produce a prediction.
• This split dictates the flow of data and gradients during the Vertical Training Protocol.

The specific layer in a split neural network where the model is divided between parties. It is a critical hyperparameter in the training protocol.

• Defines the intermediate output that must be communicated from a feature owner to the label owner.
• A deeper cut (later layer) increases computation locally but reduces the dimensionality of data exchanged, potentially lowering communication overhead.
• A shallower cut (earlier layer) does the opposite, exchanging more raw feature information.

The prerequisite step that enables VFL by identifying overlapping entities (e.g., customer IDs) across different parties' datasets without revealing non-overlapping records.

• Without alignment, parties cannot know which samples correspond for collaborative training.
• Typically achieved via cryptographic protocols like Private Set Intersection (PSI).
• The output is a secure, aligned list of common IDs that the Vertical Training Protocol uses to coordinate sample batches.

The distributed execution of the forward and backward passes that constitute a single training step in the protocol.

• Forward Pass: Feature owners compute local activations up to the cut layer, sending the intermediate output to the label owner, who completes the forward pass to compute loss.
• Backward Pass: The label owner computes gradients for its top model and the gradient w.r.t. the received intermediate output. This gradient is sent back to feature owners, who use it to compute gradients for their local model segments via chain rule.
• This dual-pass communication is the core computational loop of the protocol.

Cryptographic techniques used to combine model updates (e.g., gradients or intermediate outputs) from multiple feature owners without revealing any single owner's contribution.

• Protects against privacy leakage that could reverse-engineer features from shared updates.
• Common techniques include Secure Multi-Party Computation (MPC) and masking schemes.
• A non-cryptographic baseline is simple aggregation by a coordinator, which offers less privacy.

The operational protocol for making predictions after the model is trained. It defines how feature owners securely provide input to the deployed split neural network.

• For a new sample, each feature owner computes its local forward pass to the cut layer.
• The intermediate outputs are sent (often encrypted) to the label owner or inference service.
• The top model completes the forward pass to generate the final prediction, which may be returned to a requesting party.