Federated Semi-Supervised Learning

FSSL operates under the fundamental assumption that labeled data is not uniformly available across the network. One common scenario is the server-labeled setting, where the central server holds a curated, labeled dataset while clients possess only unlabeled local data. The inverse—where clients hold sparse labels and the server has none—is also valid. This asymmetry mirrors real-world clinical deployments where a research hospital may have annotated ground truth while community clinics only have raw, unlabeled patient records.

The core mechanism for propagating label information to unlabeled clients. The principle enforces that a model should produce similar predictions for an unlabeled data point even when it is perturbed by different augmentations or noise. Key techniques include:

• FixMatch: Generates pseudo-labels on weakly augmented data and uses them as targets for strongly augmented versions
• Virtual Adversarial Training (VAT): Applies small adversarial perturbations that maximize the change in model output distribution
• Mean Teacher: Maintains an exponential moving average of model weights to generate stable targets for unlabeled data

Pseudo-labeling assigns synthetic labels to unlabeled data based on the model's highest-confidence predictions. In FSSL, this process must be carefully managed to prevent confirmation bias—where the model reinforces its own errors. Federated pseudo-labeling typically involves:

• Confidence thresholding to only retain high-probability predictions
• Cross-client validation where pseudo-labels from one client are verified against the aggregated global model
• Gradual unfreezing of pseudo-labeled samples as training stabilizes This technique is particularly effective when combined with Federated Averaging to smooth out noisy individual client predictions.

An interpolation-based regularization strategy adapted for decentralized settings. Mixup creates synthetic training examples by linearly combining pairs of data points and their labels. In FSSL, this extends to:

• Intra-client Mixup: Blending labeled and unlabeled samples within a single client to smooth decision boundaries
• Inter-client Mixup: Sharing interpolated representations (not raw data) between clients to bridge distribution gaps
• Manifold Mixup: Applying the interpolation in the model's hidden representation space rather than input space This technique reduces overfitting on sparse labels and improves generalization across heterogeneous client distributions.

A principle borrowed from semi-supervised learning that encourages the model to make low-entropy, high-confidence predictions on unlabeled data. In federated settings, this acts as an implicit regularizer that pushes decision boundaries away from high-density regions of the data distribution. The technique:

• Penalizes uncertain predictions on unlabeled client data
• Works synergistically with consistency regularization to sharpen pseudo-labels
• Must be balanced against the risk of overconfidence on out-of-distribution samples from heterogeneous clients Entropy minimization is often implemented as an additional loss term in the local client objective function.

An iterative process where the global model generates pseudo-labels for unlabeled client data, clients train on these expanded datasets, and the server aggregates the improved models. The cycle repeats with progressively refined pseudo-labels. Critical design choices include:

• Selection criteria: Only pseudo-labels exceeding a confidence threshold are retained
• Curriculum pacing: Gradually increasing the proportion of pseudo-labeled data as model accuracy improves
• Aggregation weighting: Giving higher weight to clients with more reliable (higher-confidence) pseudo-labels during federated averaging This approach effectively converts an unlabeled federated dataset into a progressively labeled one without manual annotation effort.

A paradigm that leverages unlabeled local data to learn robust representations in a decentralized manner before fine-tuning on limited labeled data. This directly addresses the label scarcity problem by using pretext tasks—such as rotation prediction or contrastive instance discrimination—to extract meaningful features from raw patient records without requiring manual annotation. The learned representations serve as a powerful initialization for downstream semi-supervised fine-tuning.

A self-supervised approach that aligns the representation spaces of different clients by maximizing agreement between similar instances and minimizing it for dissimilar ones without sharing labels. In a semi-supervised context, this technique can propagate label information by pulling representations of unlabeled samples toward labeled prototypes of the same class, effectively performing consistency regularization across the federated network.

A method that aggregates abstract class representations from local clients to form global prototypes, which are then redistributed to regularize local training and correct label distribution skew. In semi-supervised settings, unlabeled client data can be assigned pseudo-labels based on proximity to these global prototypes, enabling the server to guide local learning even when clients possess no ground-truth annotations.

A technique that applies knowledge learned from a source domain to a target domain within a federated network, addressing label scarcity and feature space misalignment. When the server holds labeled data and clients possess only unlabeled samples, FTL can align feature distributions and transfer supervisory signals across the network, enabling effective semi-supervised training across heterogeneous clinical environments.

A collaborative training paradigm where a cohort of local models learns simultaneously by teaching each other through shared predictions. In semi-supervised scenarios, models trained on labeled server data can generate high-quality pseudo-labels for unlabeled client data, while client models provide diverse perspectives that prevent the global model from overfitting to the server's labeled distribution.

A core semi-supervised technique that enforces prediction stability under input perturbations. In federated settings, this principle is applied by ensuring that a client's model produces consistent predictions for unlabeled data even when the data is augmented or when the model receives slightly different global parameters. This smoothness constraint propagates label information from the server's labeled set to the clients' unlabeled distributions.