Vertical Secure Aggregation

Secure Multi-Party Computation (MPC) is a foundational cryptographic paradigm that enables multiple parties to jointly compute a function over their private inputs while keeping those inputs concealed. In VFL, MPC protocols are used to perform operations like gradient averaging or loss calculation.

• Core Principle: No party learns anything beyond the final aggregated output and what can be inferred from its own input and output.
• VFL Application: Parties can collaboratively compute the sum of their individual gradient vectors without any single party seeing another's vector. Common protocols include Garbled Circuits and Secret Sharing-based approaches.

Homomorphic Encryption (HE) is a form of encryption that allows specific types of computations to be performed directly on ciphertext, generating an encrypted result that, when decrypted, matches the result of the operations as if they had been performed on the plaintext.

• Partial vs. Fully HE: Partially Homomorphic Encryption (e.g., Paillier) supports only addition or multiplication. Fully Homomorphic Encryption (FHE) supports both but is computationally intensive.
• VFL Workflow: A feature owner encrypts its local gradient update using the aggregator's public key. The aggregator can sum these encrypted vectors without decrypting them, then decrypt only the final aggregated result.

Differential Privacy (DP) provides a rigorous, mathematical framework for quantifying and limiting privacy loss. When integrated with secure aggregation in VFL, it adds a layer of protection against inference attacks on the final aggregated model.

• Local DP: Each party adds calibrated noise (e.g., from a Gaussian or Laplacian distribution) to its update before secure aggregation. This guarantees privacy even if the aggregation protocol is compromised.
• Global DP: Noise is added to the aggregated model update. This requires a trusted aggregator but often provides better utility for the same privacy budget (epsilon, δ).

Secret Sharing is a method for distributing a secret (e.g., a gradient vector) among a group of parties, where only a specified subset (the threshold) can reconstruct the secret. It's a core building block for many MPC protocols used in aggregation.

• Shamir's Secret Sharing: A threshold scheme where a secret is split into n shares. Any t shares can reconstruct the secret, but any t-1 shares reveal zero information.
• VFL Process: Each party splits its local update into secret shares and distributes them among other participants or non-colluding servers. The aggregation (summation) is performed directly on the shares, and only the final summed shares are combined to reveal the aggregated update.

Verifiable Secret Sharing (VSS) extends standard secret sharing by allowing parties to verify that their shares are consistent and correctly formed, even if the dealer (the party creating the shares) is malicious. This is critical for robust VFL systems.

• Byzantine Fault Tolerance: VSS ensures the aggregation protocol remains correct in the presence of Byzantine clients who may send corrupted shares to disrupt training.
• Mechanism: Uses cryptographic commitments (like Pedersen commitments) to allow share recipients to verify the integrity of their share without learning the secret or other shares.

The design of a vertical secure aggregation protocol is dictated by the assumed threat model and communication topology.

• Threat Models:
  • Semi-Honest (Honest-but-Curious): Parties follow the protocol but try to learn extra information from received messages. Most basic protocols target this.
  • Malicious (Byzantine): Parties may arbitrarily deviate from the protocol. Defenses require VSS, zero-knowledge proofs, and robust aggregation.
• Communication Topologies:
  • Centralized: A central server (aggregator) coordinates all parties. Most common but introduces a single point of coordination.
  • Peer-to-Peer (Decentralized): Parties communicate directly. Increases robustness but complicates protocol design and synchronization.

Secure aggregation protocols, especially those based on Multi-Party Computation (MPC) or Homomorphic Encryption (HE), require multiple rounds of communication per training step. Unlike horizontal FL, where clients send a single model update, VFL often involves exchanging intermediate outputs and gradients for each aligned sample in a mini-batch. This can lead to a quadratic or cubic increase in bandwidth relative to the number of participating parties and the size of the aligned dataset. The trade-off is between the strength of cryptographic guarantees and the practical feasibility of training, often requiring aggressive compression of transmitted tensors.

The cryptographic primitives that enable privacy are computationally intensive. Homomorphic encryption operations on high-dimensional intermediate tensors can be orders of magnitude slower than plaintext operations. Secure Multi-Party Computation protocols involve continuous secret sharing and reconstruction. This computation overhead directly translates to longer training times and increased energy consumption on participant hardware. The engineering trade-off involves selecting cryptographic building blocks (e.g., additive secret sharing vs. fully homomorphic encryption) that provide the necessary security level without making training computationally prohibitive.

VFL's sequential dependency—where the label owner must wait for all feature owners' encrypted intermediate outputs to proceed—creates a synchronization bottleneck. Stragglers (slow parties due to weak hardware or network) delay the entire training round. This is exacerbated by secure aggregation protocols that are often all-or-nothing; the aggregation cannot proceed without all participants. Mitigations like asynchronous protocols or dropout tolerance are complex to design cryptographically. The trade-off is between robust privacy guarantees that require full participation and training efficiency that must tolerate real-world system variability.

Absolute privacy often comes at the cost of model utility. Techniques applied for secure aggregation can degrade learning:

• Differential Privacy (DP) Noise: Adding DP noise to gradients or outputs to formally guarantee privacy directly reduces the signal-to-noise ratio in updates, slowing convergence and potentially lowering final model accuracy.
• Quantization & Compression: Applying lossy compression to encrypted messages to reduce communication overhead can discard meaningful information.
• Approximate MPC: Using faster, less secure MPC variants may leak more information. The core challenge is quantifying and configuring this trade-off to meet specific regulatory and business requirements.

Vertical secure aggregation is not a simple averaging of parameters as in horizontal FL. The aggregation logic is tightly coupled with the split neural network architecture and the training protocol. It must correctly combine:

• Encrypted intermediate embeddings from feature owners.
• Gradients back-propagated from the label owner.
• Partial model updates to different segments of the model held by different parties. This requires a deeply integrated cryptographic protocol within the training loop, increasing the complexity of the vertical FL framework and the risk of implementation errors that could compromise privacy.

Many classical secure aggregation protocols scale poorly with the number of participants (n). Communication complexity often grows O(n²). In a vertical setting where each party holds a unique feature set, adding more valuable data partners directly increases coordination complexity. While client-side secure aggregation in horizontal FL can handle thousands of devices, party-side secure aggregation in VFL may become impractical beyond a few dozen participants. The trade-off forces a choice between a broader data consortium and the performance/feasibility of the cryptographic protocol, pushing designs toward hierarchical or hybrid aggregation topologies.

A model architecture where the neural network is divided into multiple segments, with each segment residing on a different party in the VFL system. The division point is defined by the cut layer.

• Local Bottom Models: Feature owners hold the initial layers that process their local features.
• Central Top Model: The label owner (or a coordinator) typically holds the layers from the cut layer to the output.
• Forward Pass: Feature owners compute intermediate outputs up to the cut layer and send them securely to the party holding the next segment.

The defined sequence of communication and computation steps that all participating parties follow to execute a complete training round in a VFL system. It orchestrates forward propagation, loss calculation, backpropagation, and secure update aggregation.

• Typical Steps:
  1. Entity Alignment (via PSI).
  2. Vertical Forward Propagation: Feature owners compute and send encrypted intermediate outputs.
  3. Loss & Gradient Calculation: Performed by the label owner.
  4. Vertical Backpropagation: Gradients are sent back securely.
  5. Secure Aggregation & Update: Local model segments are updated, often using a secure aggregation protocol.
• Guarantees: Ensures the process is correct, private, and efficient.