Federated Model Personalization

A methodology where a foundation model pre-trained on a large public corpus is distributed to institutions. Each site then fine-tunes only the final layers on its private clinical data. Only these task-specific, shallow updates are aggregated centrally, preserving the bulk of the model's general knowledge while adapting to local patient demographics. This is highly effective when local datasets are small but statistically distinct from the global average.

An architecture that treats each hospital's local optimization as a separate but related task. The global model learns a shared representation that captures common clinical patterns, while site-specific layers branch off to handle local data distributions. This prevents the model from overfitting to the dominant data distribution in the network and explicitly models inter-institutional variability in patient populations and clinical practices.

A 'learning to learn' approach where a model is trained across diverse clinical tasks from multiple institutions to find an optimal initialization. This initialization can be rapidly adapted to a new task at a new hospital with very few local data points. The goal is not a single global model, but a model that is explicitly optimized for fast personalization, often using algorithms like Model-Agnostic Meta-Learning (MAML) adapted for federated settings.

A model compression and personalization technique where a large, powerful global 'teacher' model's knowledge is transferred to smaller, site-specific 'student' models. Instead of sharing private model gradients, institutions exchange only the teacher's output logits on a public or synthetically generated dataset. This allows each hospital to train a compact, personalized model that mimics the global model's behavior without direct weight sharing.

A post-training personalization technique that creates a custom model for a specific site by computing a weighted average between the global model and a model fine-tuned purely on local data. The interpolation weight controls the trade-off: a higher weight on the local model increases personalization but risks overfitting, while a higher weight on the global model improves generalization. This provides a simple, computationally cheap personalization lever.

A technique that adds a penalty to the local training objective to prevent the feature representations learned at one institution from diverging too far from the global consensus. This ensures a semantically consistent embedding space across the network while still allowing for local adaptation. It is critical for tasks like federated patient similarity search, where a consistent notion of 'similar' must be maintained across all sites.