Vertical Federated Learning

In VFL, data is partitioned vertically or by feature. Different parties (e.g., a bank and an e-commerce platform) hold different attributes (features) for the same set of users (entities or sample IDs).

• Bank: Holds credit score, transaction history.
• E-commerce: Holds purchase history, browsing behavior.

The goal is to train a model that utilizes this combined feature space without any party exposing its raw feature columns. This contrasts with Horizontal Federated Learning (HFL), where parties have the same feature set but different samples.

A prerequisite for VFL is Private Set Intersection (PSI) to securely identify the common set of entities across parties without revealing non-overlapping samples.

• Process: Parties use cryptographic protocols (e.g., based on Diffie-Hellman, oblivious transfer) to compute the intersection of their ID lists.
• Output: Only the aligned, overlapping samples are used for training. The protocol ensures no party learns the full ID list of another.

This step is computationally intensive but critical for privacy and model validity, preventing training on misaligned data.

The model architecture is physically split across participants. A typical setup involves:

• Bottom Models: Each party holds a local model (e.g., a few neural network layers) that processes its private features.
• Interactive Layer: The outputs (embeddings or smashed data) from all bottom models are sent to a guest party or a neutral coordinator.
• Top Model: The guest/coordinator aggregates these intermediate outputs and runs the remaining layers of the network to produce the final prediction.

During backward propagation, gradients flow back through the top model to each party's bottom model for local updates, without exposing raw features.

VFL typically involves asymmetric participant roles, unlike the symmetric client-server model of HFL.

• Guest Party: The party that holds the labels (Y) for the aligned samples and usually hosts the top model. It initiates the training task and computes the final loss.
• Host Party(ies): Parties that hold only features (X) and host bottom models. They contribute feature representations but do not possess labels.

This role distinction fundamentally shapes the protocol's communication pattern, incentive structure, and privacy considerations.

The forward pass is designed to prevent leakage of private features. The key mechanism is the transmission of encrypted or homomorphically encrypted intermediate results.

• Plaintext Embeddings: In basic setups, hosts send plaintext embeddings (smashed data) to the guest. This reveals some information but not raw features.
• Enhanced Privacy: For stronger guarantees, hosts encrypt their embeddings using Homomorphic Encryption (HE) or Secure Multi-Party Computation (SMPC). The guest can perform computations on these encrypted values to continue the forward pass without decryption.

This ensures the guest cannot directly invert the embeddings to reconstruct host features.

The backward pass must also protect sensitive information. Gradients can leak information about the underlying training data.

• Gradient Protection: Hosts receive gradients for their bottom models from the guest. To prevent label leakage from the guest to the hosts, these gradients may be obfuscated or computed using cryptographic techniques.
• Aggregation without Disclosure: Protocols like Secure Aggregation can be extended to VFL, allowing the coordinator to compute the necessary aggregated gradient information without learning any party's individual contribution.

This secure exchange is crucial for maintaining the privacy guarantee for all parties throughout the training lifecycle.

The more common counterpart to VFL, where participating parties hold different data samples (e.g., different users) but with the same feature space. For example, two hospitals train a model on their separate patient records, which all contain the same clinical features. The core challenge is data distribution heterogeneity across clients.

• Key Difference: HFL partitions data by rows (samples); VFL partitions by columns (features).
• Typical Use: Cross-device learning on smartphones or IoT sensors.

A distributed technique where a neural network is vertically split between parties. The client holding the raw data computes the initial layers and sends the intermediate outputs (smashed data or activations) to another party (e.g., a server) which completes the forward and backward pass. This allows collaborative training without sharing raw input data.

• Relation to VFL: Often used as the underlying training protocol for VFL, especially when one party holds the labels. The model architecture itself is federated.
• Privacy Risk: Smashed data can potentially leak information, requiring privacy safeguards.

A cryptographic framework that enables multiple parties to jointly compute a function (like model training or inference) over their private inputs while revealing only the final output. No party learns anything about the others' raw data beyond what is implied by the result.

• Role in VFL: Used to securely compute aggregated statistics, gradients, or loss calculations during the collaborative training process. For example, securely computing the sum of gradients from different feature holders.
• Common Protocols: Garbled Circuits, Secret Sharing.

The prerequisite step in VFL where parties must securely identify overlapping samples (entities) across their disjoint feature sets without exposing their entire datasets. This is necessary because VFL assumes collaboration on the same set of entities (e.g., the same customers).

• Core Challenge: Performing alignment with privacy guarantees, often using techniques like Private Set Intersection (PSI).
• Without Alignment: Parties would be training on misaligned data, rendering the model useless. This step is unique to the vertical data partition setting.

A form of encryption that allows computation on ciphertexts. An operation performed on encrypted data produces an encrypted result which, when decrypted, matches the result of the operation as if it had been performed on the plaintext.

• Role in VFL: Enables a party (e.g., the label holder) to perform computations on encrypted intermediate results from other feature holders. This allows forward/backward propagation without decrypting sensitive features.
• Trade-off: Provides strong cryptographic security but introduces significant computational overhead.

A federated learning setting characterized by a small number of reliable, organizational participants (e.g., 2-100 banks or hospitals). Participants have significant computational resources and stable connectivity. This is the primary deployment scenario for VFL.

• Contrast with Cross-Device FL: Cross-device involves millions of unstable edge devices (phones). VFL's need for entity alignment and complex cryptographic protocols makes it practically suited only for the cross-silo context.
• Trust Model: Parties are often known but mutually distrustful regarding their proprietary data assets.