Federated Hypernetwork

A central hypernetwork takes a client descriptor vector as input and outputs the full set of weights for that client's personalized model. The descriptor encodes site-specific characteristics—such as patient demographics, imaging protocols, or disease prevalence—allowing the hypernetwork to conditionally generate parameters tailored to each institution's data distribution. This eliminates the need for clients to share model updates directly; instead, they only communicate loss gradients back to the hypernetwork for training.

Client descriptors serve as a structured interface for personalization. These vectors can encode:

• Statistical metadata: label distributions, feature means, covariate shifts
• Clinical context: hospital type (tertiary vs. community), imaging device manufacturer
• Learned embeddings: latent representations optimized jointly with the hypernetwork

By conditioning on known sources of heterogeneity, the hypernetwork disentangles shared knowledge (encoded in its own weights) from site-specific variation (encoded in the descriptor), enabling principled adaptation without ad-hoc fine-tuning.

Unlike standard federated averaging, which requires transmitting full model weights or gradients proportional to model size, federated hypernetworks communicate only hypernetwork gradients and client descriptors. The hypernetwork is typically designed to be significantly smaller than the target model it generates, resulting in orders-of-magnitude reduction in uplink bandwidth. This makes the approach viable for cross-silo deployments where institutions face strict data egress limitations.

Once trained, the hypernetwork can generate model weights for previously unseen clients by conditioning on their descriptor vectors. A new hospital joining the federation provides only its metadata descriptor—no local training required—and receives a fully personalized model. This zero-shot personalization capability is critical for rapid clinical deployment across heterogeneous sites without the onboarding latency of traditional federated fine-tuning.

By generating weights through a shared hypernetwork, all client models are constrained to lie on a low-dimensional manifold in parameter space. This acts as a powerful regularizer, preventing individual client models from overfitting to small local datasets—a common failure mode in personalized federated learning. The hypernetwork learns a compressed representation of valid model configurations, effectively sharing statistical strength across clients while preserving local adaptation.

Federated hypernetworks provide an inherent privacy advantage: client data never leaves the local institution, and model weights are never transmitted. The hypernetwork receives only aggregated gradient information about its own parameters, not about client-specific model weights. This architecture naturally resists model inversion attacks and membership inference, as attackers cannot access individual client models. When combined with differential privacy noise on hypernetwork gradients, formal privacy guarantees can be established.