Federated Learning

The foundational principle of Federated Learning is that raw data never leaves the client device. Training data remains on the user's smartphone, hospital server, or IoT sensor. The model travels to the data, not the other way around. This directly addresses data residency regulations and minimizes the attack surface for data breaches. Only model updates—gradients or weight deltas—are transmitted to the central server.

The canonical aggregation algorithm. The central server distributes a global model to a subset of clients. Each client trains locally on its own data for a few epochs. The server then collects the resulting model weights and computes a weighted average to create a new, improved global model. This process repeats for multiple communication rounds. Key parameters include:

• C-fraction: The fraction of clients selected per round.
• E: The number of local epochs.
• B: The local batch size.

Unlike centralized training where data is shuffled, client data in Federated Learning is almost always non-Independently and Identically Distributed (non-IID). A user's photo library is not a uniform sample of all photos. This statistical heterogeneity can cause local models to diverge, slowing convergence or degrading the final global model's accuracy. Solutions include FedProx (adding a proximal term to the local objective) and SCAFFOLD (using control variates to correct client drift).

Uploading full model weights from millions of devices is a bandwidth bottleneck. Federated Learning employs compression techniques to reduce the bits transmitted per round:

• Gradient Quantization: Reducing the precision of gradients (e.g., from 32-bit floats to 2-bit integers).
• Gradient Sparsification: Sending only the top-k largest gradient values and masking the rest.
• Local Steps: Performing multiple local updates before communicating, trading compute for communication.

A cryptographic protocol that ensures the central server can only see the sum of client updates, not any individual client's contribution. Clients mask their updates with pairwise secrets and self-masking noise. When summed, the masks cancel out, revealing only the aggregate. This prevents gradient leakage attacks where an honest-but-curious server could reconstruct private training images from individual gradients.

Federated Learning alone does not provide formal privacy guarantees; model updates can still leak information. DP-FedAvg integrates differential privacy by clipping per-client updates to a fixed L2 norm and adding calibrated Gaussian noise to the aggregate before applying it to the global model. This bounds the influence of any single client and provides a measurable privacy budget (ε, δ) against membership inference and data reconstruction attacks.

A defense mechanism that truncates or suppresses the full prediction vector returned by a model's inference API. Instead of returning logits for all classes, the server returns only the top-k predicted labels or a simple class label. This prevents attackers from exploiting fine-grained confidence values to perform model inversion or membership inference attacks against the federated global model. The trade-off is reduced interpretability for legitimate downstream consumers of the API.