Federated Learning (FL)

The foundational principle of FL is that raw data never leaves the client device. The model travels to the data, not the other way around. Training computation is performed locally on edge devices, on-premise servers, or mobile phones, ensuring that sensitive information remains within the organization's or user's direct control. This directly addresses data residency and privacy regulations.

Instead of centralizing data, FL centralizes model updates. Each client computes a local update—typically gradients or weight deltas—on its private dataset. These updates are then sent to a central coordinating server. The server's primary role is to aggregate these disparate updates using algorithms like Federated Averaging (FedAvg) to produce a new, improved global model, which is then redistributed for the next round.

Unlike a curated central database, data on FL clients is almost never independently and identically distributed (non-IID). A user's photo library differs drastically from another's; a regional hospital's patient demographics differ from a national average. This statistical heterogeneity is a core challenge, requiring specialized algorithms that can converge despite skewed, unbalanced, and non-representative local datasets.

Federated networks often consist of millions of devices with constrained, intermittent connectivity. The communication of model updates is a primary bottleneck. FL systems employ compression techniques like gradient quantization, sparsification, and structured updates to reduce bandwidth usage. The goal is to minimize the number of communication rounds and the size of each transmitted message to achieve convergence.

While data locality provides a baseline of privacy, it is insufficient against advanced inference attacks. FL is therefore combined with formal privacy guarantees. Differential Privacy (DP) is applied at the client level, where noise is added to updates before transmission. Secure Aggregation protocols using multi-party computation ensure the server can only decrypt the sum of updates, not any individual client's contribution.

The client population in an FL system is massively diverse in hardware, network, and power constraints. A flagship smartphone and a low-end IoT sensor cannot be expected to perform the same amount of computation. Structured updates and resource-aware scheduling allow devices to train on sub-parts of a model or perform variable amounts of local work, ensuring stragglers do not halt the entire training round.