Feature Space

The defining characteristic in VFL is that the complete feature space is not held by a single entity. Instead, different features (columns) for the same set of entities (rows) are owned by different parties.

• Example: A bank holds a customer's credit history (features: credit score, loan history), while an e-commerce platform holds their purchase behavior (features: avg. cart value, product categories).
• Implication: No single party has a complete feature vector for any entity, making centralized training impossible and necessitating collaborative protocols.

The aggregate feature space across all parties is often extremely high-dimensional, as it combines attributes from diverse domains. However, the local feature space held by any single party is a lower-dimensional, dense subset.

• This creates a global sparsity pattern: for any given sample, most feature values are zero from the perspective of a single owner, because they belong to other parties.
• Engineering Challenge: Protocols must efficiently handle operations on these conceptually complete but physically distributed high-dimensional vectors.

The feature distributions held by different owners are inherently non-Identically Distributed (non-IID). Each party's local feature subspace comes from a different data generating process and operational context.

• Example: Hospital medical images (pixel data) vs. pharmacy prescription records (categorical codes). Their statistical properties (mean, variance, modality) are entirely different.
• Consequence: Standard federated averaging (FedAvg) from horizontal FL fails. VFL requires specialized optimization that accounts for this cross-silo feature heterogeneity.

For VFL to function, the distributed feature spaces must be aligned sample-wise. Parties must identify which of their local records correspond to the same real-world entity (e.g., the same customer) without exposing their full datasets.

• This is solved using cryptographic protocols like Private Set Intersection (PSI) for entity alignment.
• Critical Precondition: The aligned intersection set defines the effective training dataset. Its size and quality directly limit model performance.

The partitioned nature of the feature space dictates the split neural network architecture used in VFL. The model is divided at a cut layer, with each party hosting a bottom model that processes its local features.

• Forward Pass: Each party computes an intermediate output (embedding) from its local features. These are sent to a coordinator (often the label owner) to compute the top model and final prediction.
• Architectural Constraint: The split must be designed considering feature semantics, privacy leakage from intermediates, and communication costs.

In VFL, the feature space partition is the root cause of communication overhead. Unlike horizontal FL where entire model weights are exchanged, VFL requires transmitting intermediate outputs and gradients for every sample in every training batch.

• Bandwidth Cost: Scales with batch size * embedding dimension * number of rounds, not model size.
• Optimization Focus: A major research area is designing communication-efficient VFL protocols using compression, selective synchronization, and in-network computation.

A vertical data partition is a dataset split where different features (columns) pertaining to the same set of entities (rows) are held by different, non-overlapping parties. This structure is the foundational data assumption for VFL.

• Key Distinction: Contrasts with horizontal partitioning, where different parties hold different rows (samples) with the same features.
• Example: In a financial-medical collaboration, Bank A holds a customer's transaction history (features), while Hospital B holds the same customer's diagnostic codes (different features).

Entity alignment is the privacy-preserving process of identifying the overlapping set of entities (e.g., users, devices) across the vertically partitioned datasets held by different parties. It is a critical prerequisite for VFL, as training can only occur on the aligned samples.

• Core Challenge: Must be performed without revealing the full list of entities from any party.
• Primary Tool: Typically achieved using cryptographic protocols like Private Set Intersection (PSI).

A split neural network is the standard model architecture for VFL, where the neural network is divided into multiple segments. Each segment resides on a different party that holds the corresponding subset of the total features.

• Typical Setup: Feature owners hold the bottom models (initial layers) that process their local features. A label owner holds the top model (later layers) that consumes combined intermediate outputs to produce a prediction.
• Cut Layer: The specific layer where the model is split, defining the boundary of computation and communication.

A vertical training protocol defines the exact sequence of secure communication and computation steps that coordinating parties follow to execute a single training round in a VFL system.

• Standard Steps: 1) Feature owners compute forward passes to a cut layer. 2) They send encrypted intermediate outputs to the label owner. 3) The label owner completes the forward pass, computes loss, and backpropagates gradients. 4) Relevant gradients are sent back to feature owners for their local model updates.
• Governs: The flow of data, encryption methods, and aggregation routines.

Vertical secure aggregation refers to cryptographic protocols used to combine model updates (e.g., gradients or intermediate outputs) from multiple feature owners without revealing any single party's contribution. This protects against privacy leakage and inference attacks.

• Mechanisms: Often employs Secure Multi-Party Computation (MPC) or Homomorphic Encryption (HE).
• Purpose: Ensures the label owner (or aggregator) only sees the summed update, not individual data.

Vertical communication overhead is the significant bandwidth cost and latency introduced by the need to exchange intermediate outputs and gradients between parties during every forward and backward pass of training. This is a primary systems challenge in VFL.

• Scalability Bottleneck: Overhead grows with the number of aligned samples, the dimensionality of the cut layer, and the training frequency.
• Optimization Focus: Drives research into communication compression, selective participant updates, and efficient cryptographic protocols.