Federated Self-Supervised Learning

The core mechanism enabling learning without labels. Clients train on synthetic tasks derived from the data itself:

• Contrastive Learning: Pull augmented views of the same sample together while pushing apart different samples (e.g., SimCLR, MoCo)
• Masked Autoencoding: Reconstruct intentionally corrupted portions of input data (e.g., MAE for images, BERT-style masking for EHR text)
• Rotation Prediction: Classify the rotation angle applied to an image patch
• Jigsaw Puzzles: Predict the correct spatial arrangement of shuffled image patches

These tasks force the model to learn semantically meaningful features that transfer well to downstream clinical tasks.

A widely adopted federated self-supervised framework adapting momentum contrastive learning to decentralized settings:

• Each client maintains a query encoder (actively updated) and a momentum encoder (slowly updated via exponential moving average)
• A dynamic dictionary of negative samples is maintained, either locally or via a federated queue shared across clients
• The query encoder learns to match augmented views against the momentum encoder's output while discriminating from negatives
• Federated aggregation combines only the query encoder weights, while momentum encoders evolve locally

This architecture is particularly effective for medical imaging where inter-institutional negatives improve representation quality.

A self-supervised approach that eliminates the need for negative samples, simplifying federated deployment:

• Each client trains two networks: an online network and a target network
• The online network predicts the target network's representation of an augmented view
• The target network is updated via exponential moving average of the online weights
• Only the online network parameters are shared with the federated server for aggregation

Key advantage: BYOL avoids the negative pair collapse problem without requiring large batch sizes or memory banks, making it suitable for clients with limited compute resources.

A critical challenge unique to federated SSL: ensuring that representations learned independently across clients inhabit a shared latent space:

• Prototype Exchange: Clients share abstract class prototypes—representative embedding centroids—rather than raw gradients
• Federated Contrastive Loss: An additional loss term that pulls representations of semantically similar samples across different clients closer together
• Representation Normalization: Applying consistent normalization layers (e.g., L2 normalization) across all clients before aggregation
• Periodic Global Prototype Broadcast: The server computes and distributes global prototypes to align local learning trajectories

Without alignment, representations from different hospitals may encode the same clinical concept in incompatible vector spaces.

Self-supervised learning demonstrates inherent resilience to non-IID data distributions common in healthcare federated networks:

• Unlike supervised learning, SSL does not depend on label distribution alignment across clients
• Pretext tasks operate on data structure rather than semantic categories, reducing sensitivity to class imbalance
• Local augmentation policies can be tailored to each institution's data characteristics without harming global convergence
• Techniques like Federated Spectral Clustering can group clients with similar data distributions to form sub-federations for more stable SSL training

This property makes federated SSL particularly valuable for rare disease detection where labeled examples are scarce and unevenly distributed.

After federated SSL pre-training, the learned representations are adapted to specific clinical tasks with minimal labeled data:

• Linear Probing: Freeze the pre-trained encoder and train only a linear classifier on top—requires as few as 1-5% labeled samples
• Fine-Tuning: Unfreeze and update all layers with a small learning rate on labeled downstream data
• Federated Few-Shot Learning: Combine SSL pre-training with meta-learning to adapt to new classes from only 1-5 examples per class
• Multi-Task Heads: Attach multiple task-specific heads (diagnosis, segmentation, prognosis) to a single shared SSL backbone

This workflow enables hospitals to collaboratively build a universal medical foundation model that each site can cheaply adapt to local clinical needs.

A self-supervised federated framework where local models are trained to pull representations of augmented views of the same sample together while pushing apart representations of different samples. This eliminates the need for manual labels at any client site.

• Uses SimCLR or MoCo style instance discrimination as the pretext task
• Local clients generate positive pairs via data augmentation before sharing only the representation gradients
• Critical for learning from unlabeled medical imaging where annotation is prohibitively expensive

A communication-efficient paradigm where clients share abstract class prototypes—representative embedding vectors of each category—instead of raw model weights or gradients.

• Prototypes are averaged globally to form a shared latent space without exposing individual samples
• Naturally aligns with self-supervised clustering objectives like SwAV or DeepCluster
• Reduces bandwidth by orders of magnitude compared to transmitting full model updates

The engineering challenge of training robust representations when local client datasets exhibit statistical heterogeneity—different label distributions, feature shifts, or domain gaps across hospitals.

• Self-supervised methods are inherently more resilient to label distribution skew since they do not rely on labels
• Techniques include FedProx (proximal regularization) and SCAFFOLD (variance reduction) to stabilize convergence
• Critical in healthcare where rare diseases appear at only a subset of institutions

The process of adapting a model pre-trained on a large centralized dataset to a decentralized setting where target data is distributed across silos with different feature or label spaces.

• Self-supervised pre-training on public data (e.g., ImageNet, PubMedBERT) provides a universal initialization
• Local clients then fine-tune on their private unlabeled data using federated self-supervised objectives
• Enables knowledge transfer without violating data sovereignty

A training objective that aims to produce a model that generalizes to entirely unseen target domains without requiring any data from them during training.

• Self-supervised learning encourages learning domain-invariant features by focusing on inherent data structure rather than superficial correlations
• Techniques like FedDG use continuous frequency space interpolation to simulate unseen distributions
• Essential for deploying models across hospitals with different scanner vendors and imaging protocols

Strategies for minimizing bandwidth overhead when transmitting model updates between distributed clients and the central aggregation server.

• Gradient compression via sparsification or quantization reduces update size by 100x or more
• FetchSGD uses count sketches to compress gradients with provable error bounds
• Self-supervised training often requires more communication rounds, making efficiency techniques critical for practical deployment