Label Space

The label space is exclusively held by a single party, known as the label owner or active party. This centralizes orchestration and loss calculation. All other participants are feature owners (passive parties) who hold subsets of the input features. This asymmetry creates a client-server-like topology where the label owner typically acts as the server, coordinating the training rounds and aggregating updates from feature owners.

The physical location of the label space determines the cut layer in a split neural network. The model is divided so that:

• Feature owners compute the forward pass up to the cut layer on their local features, producing intermediate outputs (embeddings).
• These outputs are sent to the label owner, which holds the latter layers of the model that consume the embeddings and compute the final prediction against the local labels. This split ensures raw features and raw labels are never exchanged, only mathematically transformed intermediate values.

The label space is a high-value target and a potential leakage point. Adversaries may attempt:

• Label Inference Attacks: A malicious feature owner could use repeated intermediate outputs and the resulting gradient updates from the label owner to infer the private labels.
• Property Inference Attacks: To deduce statistical properties of the label distribution. Defenses are architected around the label owner, employing homomorphic encryption for received embeddings or differential privacy by adding noise to its gradient broadcasts to protect the label space.

The label owner executes the core vertical training protocol:

1. Initiation: Broadcasts the initial model architecture and public keys (if using encryption).
2. Coordination: Manages the synchronized forward and backward passes across parties.
3. Loss & Gradient Calculation: Computes the loss function using its local labels and calculates gradients for its portion of the model.
4. Aggregation & Update: Securely aggregates gradients from feature owners (e.g., via secure aggregation) and updates the global model. This central role is distinct from Horizontal FL, where the server typically holds no data.

The nature of the label space defines the machine learning task for the entire VFL system:

• Binary Classification: Label space is {0, 1}.
• Multi-class Classification: Label space is a set of C discrete classes.
• Regression: Label space is a continuous range R. The choice of final activation function (e.g., sigmoid, softmax, linear) and loss function (e.g., BCE, cross-entropy, MSE) is implemented by the label owner, directly impacting the gradients propagated back to feature owners.

Before training begins, the aligned entity space must be established. The label owner's sample IDs (e.g., user IDs) form the basis for this alignment. Using protocols like Private Set Intersection (PSI), the label owner and each feature owner confidentially discover their common samples without revealing their full ID lists. Only data for these intersected entities participates in training. The size and quality of this intersection directly limit the effective training dataset size.

The foundational data structure for VFL, where different features (columns) describing the same set of entities (rows) are held by separate parties. This is distinct from horizontal partitioning, where different parties hold different rows with the same features.

• Example: Hospital A holds patient lab results, while Hospital B holds the same patients' imaging data and an insurer holds the billing codes.

The privacy-preserving process of identifying matching records (e.g., the same customer or device) across the vertically partitioned datasets held by different parties. This is a critical prerequisite for VFL, as training requires aligned samples.

• Primary Method: Private Set Intersection (PSI) protocols allow parties to compute the overlap of their private ID sets without revealing non-matching IDs.

The model architecture used in VFL, where a single neural network is divided into segments distributed across parties. The cut layer defines the boundary.

• Forward Pass: A feature owner computes an intermediate output up to the cut layer and sends it to the label owner, who completes the forward pass and calculates the loss.
• Backward Pass: Gradients are propagated backwards through the network segments to update each party's local model parameters.

Cryptographic methods for combining model updates (e.g., gradients or intermediate outputs) from multiple parties without revealing any individual contribution. This protects against inference attacks that could reconstruct private data from a single party's update.

• Common Techniques: Utilize secure multi-party computation (MPC) or homomorphic encryption to perform computations on encrypted data.

The defined sequence of communication and computation steps that coordinating parties follow to execute a training round. It orchestrates the entire lifecycle:

• Alignment Phase: Securely align entity IDs using PSI.
• Training Loop: Iterate through distributed forward/backward propagation and secure aggregation.
• Aggregation & Update: The central server (often the label owner) aggregates updates and broadcasts the new global model state.

The overarching discipline applying cryptographic and statistical techniques to VFL to prevent data leakage. It addresses risks beyond raw data exposure, such as inferring features from gradients or intermediate outputs.

• Key Techniques:
  • Differential Privacy (DP): Adds calibrated noise to updates.
  • Homomorphic Encryption (HE): Allows computation on ciphertext.
  • Multi-Party Computation (MPC): Enables joint computation without revealing inputs.