Federated Multi-Task Learning

FMTL architectures learn a common feature extractor that captures patterns useful across all tasks, while maintaining task-specific heads for individual predictions. This parameter sharing enables knowledge transfer between tasks—for example, a shared encoder trained on both diabetic retinopathy detection and glaucoma classification learns richer retinal features than either task alone. The shared layers benefit from aggregated gradients across all tasks and clients, improving sample efficiency in data-scarce scenarios.

FMTL explicitly models task relatedness to determine which tasks should share parameters and to what degree. Common approaches include:

• Hard parameter sharing: All tasks use identical hidden layers with separate output heads
• Soft parameter sharing: Each task has its own parameters with regularization penalties encouraging similarity
• Task clustering: Dynamically grouping related tasks based on statistical similarity metrics This prevents negative transfer, where dissimilar tasks degrade each other's performance when forced to share representations.

Unlike standard federated learning that optimizes a single global objective, FMTL allows each client to maintain local task importance weights. A hospital specializing in cardiac care can prioritize cardiovascular risk prediction over other tasks, while a neurology clinic emphasizes neurodegenerative disease classification. The global model aggregates these heterogeneous objectives using multi-objective optimization, producing a shared representation that accommodates diverse clinical priorities without forcing uniform task relevance across sites.

FMTL reduces communication overhead by decomposing model updates into shared components (transmitted to the server) and task-specific components (retained locally). The shared representation gradients are aggregated across all clients, while task-specific parameters update only on clients performing those tasks. This selective gradient sharing minimizes bandwidth consumption—critical in healthcare settings where institutions may have limited network infrastructure—while preserving the benefits of collaborative learning across related clinical prediction tasks.

FMTL provides natural resilience to non-IID data distributions by framing client-specific data biases as distinct but related tasks. When a rural clinic's patient demographics differ from an urban hospital's, FMTL treats each as a separate task within a related task group rather than forcing a single global model to accommodate conflicting distributions. This task-based decomposition of statistical heterogeneity improves convergence stability and final model performance compared to standard FedAvg in highly heterogeneous clinical environments.

FMTL enables multi-institutional clinical models that simultaneously predict multiple related outcomes from shared patient representations. Example deployments include:

• Joint prediction of hospital readmission risk, length of stay, and mortality from EHR data
• Simultaneous tumor segmentation and survival prognosis from distributed pathology images
• Multi-task drug response prediction across pharmacogenomic datasets at different research hospitals Each institution contributes to and benefits from the shared knowledge while maintaining data sovereignty.