Vertical Federated Averaging (VFA)

VFA operates on a split neural network where the model is partitioned at a designated cut layer. Feature owners compute the forward pass on their local features up to this layer, producing intermediate outputs (embeddings). These outputs are securely sent to the label owner, who completes the forward pass, computes the loss, and initiates vertical backpropagation. This architecture ensures raw feature data never leaves its owner's device.

Unlike horizontal FL which averages complete model weights, VFA aggregates updates from vertically partitioned model segments. This involves securely combining:

• Intermediate outputs from feature owners for the forward pass.
• Gradients w.r.t. these intermediate outputs during backpropagation. Aggregation uses cryptographic techniques like homomorphic encryption or secure multi-party computation (MPC) to prevent the label owner from inferring raw feature values from the individual updates, a process known as vertical secure aggregation.

A foundational requirement for VFA is entity alignment—identifying the overlapping set of samples (e.g., the same customers) across the vertically partitioned datasets held by different parties. This is typically achieved using privacy-preserving protocols like Private Set Intersection (PSI) or secure entity resolution. Training only proceeds on this aligned subset, making the efficiency and privacy of the alignment protocol critical to overall system performance.

VFA defines distinct, asymmetric roles for participants:

• Feature Owners: Hold subsets of features for aligned entities. They compute the first segment of the model and are typically passive participants receiving gradients.
• Label Owner: Holds the target values (labels) and often acts as the coordinating server. It computes the final layers, loss, and orchestrates the training protocol. This asymmetry differentiates VFA from horizontal FL and influences protocol design, trust models, and incentive structures.

VFA incurs a unique vertical communication overhead. Each training round requires multiple exchanges per sample/batch:

1. Feature owners → Label Owner: Intermediate outputs.
2. Label Owner → Feature Owners: Gradients w.r.t. those outputs. This pattern is more frequent and fine-grained than the single weight exchange in horizontal Federated Averaging. The overhead is a key bottleneck, driving research into compression and efficient encryption for intermediate results.

The core computation in VFA is inherently privacy-preserving by design, but often augmented with additional guarantees. The base protocol prevents direct data sharing. To defend against potential inference from gradients or intermediate outputs, VFA systems commonly integrate:

• Differential Privacy (DP): Adding calibrated noise to gradients or embeddings.
• Homomorphic Encryption (HE): Performing computations on encrypted intermediate outputs.
• Multi-Party Computation (MPC): Cryptographically splitting computations. This layered approach provides defense-in-depth for sensitive vertical data.

A vertical data partition is a dataset split where different features (columns) describing the same set of entities (rows) are held by different, non-overlapping parties. This is the foundational data structure for VFL and VFA.

• Example: Hospital A holds patient lab results, Hospital B holds medical imaging features, and an insurer holds the diagnostic labels—all for the same cohort of patients.
• This structure contrasts with horizontal partitioning, where different parties hold different samples with the same feature set.

Entity alignment is the privacy-preserving process of identifying the overlapping set of entities (e.g., user IDs, patient records) across the vertically partitioned datasets held by different parties. It is a critical prerequisite for VFA, as training can only occur on the aligned subset.

• This is typically achieved using cryptographic protocols like Private Set Intersection (PSI).
• Without secure alignment, parties would risk exposing their entire user base or dataset membership.

A split neural network is the model architecture used in VFA and VFL, where the neural network is divided into multiple segments. Each segment resides on a different party that holds the corresponding subset of features.

• The cut layer defines the boundary where the model is split.
• Feature owners compute forward passes up to the cut layer, producing intermediate outputs.
• The label owner (or a coordinating server) typically holds the subsequent layers and the loss function, completing the forward and backward passes.

Secure aggregation refers to cryptographic protocols that allow a central server (or coordinator) to compute the sum of model updates from multiple clients without being able to inspect any individual client's contribution. In the context of VFA, this principle is adapted for vertical secure aggregation of gradients or intermediate results.

• This protects against privacy leakage from a curious aggregator.
• Common techniques include secure multi-party computation (MPC) and masking with secret shares.

Homomorphic Encryption (HE) is a form of encryption that allows computations to be performed directly on ciphertext. In VFA, HE can enable the label owner to perform computations on encrypted intermediate outputs from feature owners.

• This allows the training process to continue without ever decrypting sensitive feature data.
• While powerful, HE introduces significant computation and communication overhead, making efficiency a key research challenge.

A vertical training protocol is the defined sequence of steps that coordinating parties follow to execute a single training round in a VFL system like VFA. It formalizes the interaction pattern.

• Typical Steps: 1) Entity Alignment, 2) Feature owners compute intermediate outputs, 3) Secure transmission/aggregation, 4) Label owner computes loss and gradients, 5) Gradients are propagated back to feature owners, 6) All parties update their local model segments.
• The protocol must explicitly handle failure, dropout, and malicious behavior.