Glossary

Vertical Federated Learning

A federated learning paradigm where datasets share the same sample space but differ in feature space, enabling collaboration between institutions holding different attributes for the same patients.

Get in touch Learn more

Data engineer managing feature store on laptop, feature definitions visible, casual data engineering session.

FEATURE-PARTITIONED COLLABORATION

What is Vertical Federated Learning?

A privacy-preserving machine learning paradigm where participating entities hold different attributes for the same set of overlapping samples, enabling collaborative model training without sharing raw data.

Vertical Federated Learning (VFL) is a federated learning topology where datasets share the same sample space but differ in their feature space. Unlike horizontal federated learning, where different hospitals hold the same clinical measurements for different patients, VFL applies when institutions possess different attributes for the same individuals. A canonical example is a hospital holding medical images and lab results for a patient cohort, while a pharmaceutical company holds genomic sequences for that identical cohort. VFL enables these entities to jointly train a machine learning model without exposing their respective feature sets to one another, preserving the privacy of both clinical and genomic data.

The primary technical challenge in VFL is secure entity alignment—identifying overlapping samples across institutions without revealing non-matching records—typically accomplished through Private Set Intersection (PSI) protocols. Once aligned, training proceeds by partitioning the neural network across participants: each party maintains its own bottom model for local feature extraction, while a central server or trusted third party aggregates intermediate representations to compute the global loss. This architecture requires rigorous cryptographic safeguards, as the exchange of embeddings and gradients can inadvertently leak sensitive information, necessitating techniques like homomorphic encryption or differential privacy to protect against inference attacks during the collaborative training process.

ARCHITECTURAL PRIMITIVES

Key Features of Vertical Federated Learning

Vertical Federated Learning (VFL) addresses the scenario where institutions hold different attributes for the same patients. Unlike horizontal FL, VFL requires entity alignment and privacy-preserving feature intersection before training can begin.

Entity Alignment via Private Set Intersection

The foundational prerequisite for VFL. Before any model training, participating institutions must identify overlapping samples (e.g., the same patients) without revealing non-overlapping identities. Cryptographic Private Set Intersection (PSI) protocols enable two or more parties to compute the intersection of their sample IDs while revealing nothing about samples outside the intersection. This step is legally critical in healthcare, where revealing that a patient is not at another hospital can itself be a privacy violation.

Zero

Non-overlapping IDs revealed

Split Neural Network Architecture

VFL typically employs a split learning paradigm where the neural network is partitioned across participants. Each institution holds a bottom model that processes its local features into intermediate representations (embeddings). These embeddings are exchanged and fed into a top model hosted by a central server or a designated lead institution. Only the cut-layer activations and gradients—not raw features—are transmitted, preserving the confidentiality of each party's proprietary feature space.

Asymmetric Feature Ownership

The defining characteristic of VFL is that no single party holds the complete feature vector for any sample. For example:

Hospital A holds demographics and lab results
Hospital B holds genomic sequences and imaging data
Insurance Provider C holds claims history and medication adherence

All three hold records for overlapping patients. VFL enables joint model training across these complementary views without any party gaining access to another's raw data columns.

Secure Gradient Aggregation with Homomorphic Encryption

During backpropagation, the top model computes gradients that must be transmitted back to each bottom model. To prevent gradient leakage—which can reveal sensitive feature information—VFL implementations often employ additively homomorphic encryption (HE). This allows the server to aggregate encrypted gradients without decrypting individual contributions. Combined with differential privacy noise injection, this creates a robust defense against gradient inversion attacks in healthcare multi-institutional settings.

Label Asymmetry and Federated Semi-Supervised Learning

In many healthcare VFL scenarios, only one institution holds the ground-truth labels (e.g., the hospital with confirmed diagnoses). This creates a label asymmetry challenge. VFL frameworks address this through:

Federated semi-supervised learning where unlabeled parties contribute feature representations
Label propagation techniques that distribute soft pseudo-labels to non-label-holding parties
Mutual knowledge distillation where each party's bottom model learns from the ensemble's consensus predictions

Inference-Time Coordination Protocol

Unlike horizontal FL where each client can use the global model independently, VFL requires runtime coordination for every inference. All participating institutions must be online to compute their respective bottom-model embeddings and transmit them to the top model. This introduces latency and availability requirements. Production healthcare VFL systems implement service-level agreements (SLAs) for inference response times and fallback mechanisms for when a participating node is temporarily unavailable.

VERTICAL FEDERATED LEARNING

Frequently Asked Questions

Clear, technically precise answers to the most common questions about the architecture, security, and implementation of Vertical Federated Learning in healthcare networks.

Vertical Federated Learning (VFL) is a privacy-preserving machine learning paradigm where two or more institutions hold different features (columns) for the same set of entities (rows). Unlike Horizontal Federated Learning where datasets share the same schema, VFL addresses scenarios where, for example, a hospital holds a patient's lab results and imaging data while a pharmacy holds that same patient's prescription history and genomic markers. The training process uses Entity Alignment—a privacy-preserving protocol, often based on Private Set Intersection (PSI)—to match overlapping samples without revealing non-overlapping identities. Once aligned, the parties collaboratively train a split model: each institution maintains a local bottom model that processes its own features, and the intermediate representations (embeddings) are exchanged to compute a combined loss. A neutral third-party server typically computes the gradients and updates, ensuring no raw features are ever shared. This architecture enables the joint model to learn from a complete feature vector that no single party possesses, unlocking predictive power impossible in isolated silos.

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

Federated Learning Topologies

Related Terms

Explore the structural paradigms that define how data and computation are distributed across participants in a federated network. Understanding these topologies is essential for architects designing multi-institutional healthcare AI systems.

Horizontal Federated Learning

The most common paradigm where datasets share the same feature space but contain different sample populations. In healthcare, this occurs when multiple hospitals record the same clinical measurements (e.g., blood pressure, lab results) for entirely different patients. The global model benefits from increased sample diversity without seeing any individual's data.

Same features, different samples

Core Principle

Federated Transfer Learning

A hybrid technique that bridges scenarios where parties have neither overlapping samples nor overlapping features. A pre-trained model is adapted using a small, overlapping dataset or public data to align feature spaces. This is critical for rare disease research where a small hospital may have unique genomic markers but limited patient overlap with larger institutions.

Split Learning

A privacy-preserving architecture where a deep neural network is physically partitioned between a client and a server. The client processes data through initial layers and sends only intermediate activations (smashed data) to the server, which completes the forward pass. Gradients flow backward similarly. No raw data or complete model is ever shared, making it ideal for resource-constrained medical devices.

Federated Secure Aggregation

A cryptographic protocol ensuring the central server can only compute the sum of client model updates without inspecting any individual contribution. This prevents the server from reverse-engineering a single hospital's patient data from its gradient update. Essential for multi-institutional consortia where trust is limited and regulatory compliance demands mathematical privacy guarantees.

Federated Non-IID Data

The defining challenge of real-world federated learning where local client datasets are not independently and identically distributed. In healthcare, this manifests as:

Label distribution skew: One hospital specializes in cardiology, another in oncology.
Feature distribution skew: Different imaging equipment produces varying pixel intensities.
Concept drift: The same diagnosis criteria may vary regionally.

Federated Model Divergence

The tendency of locally trained models to drift apart from the global optimum due to statistical heterogeneity. When a hospital's model overfits to its local patient demographics, averaging its weights with others can degrade global performance. Mitigation strategies include proximal regularization (FedProx) and variance reduction techniques that constrain local updates.

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.