Communication Rounds

The round begins with the central server selecting a cohort of available clients and transmitting the current global model parameters to them. This broadcast is a one-to-many distribution, often optimized for bandwidth. The selection strategy can be random, based on device capability, or designed to maximize statistical diversity. In cross-device FL, this phase must handle intermittent connectivity and device churn.

Each selected client performs local stochastic gradient descent (Local SGD) on its private, on-device dataset. This involves multiple training epochs over the local data. The key hyperparameter is the number of local steps or epochs before communication. Performing more local computation reduces communication frequency but can lead to client drift, where local models diverge due to non-IID data distributions. Algorithms like FedProx add a proximal term to the local loss to constrain this drift.

After local training, the client computes an update. This is typically the difference between the initialized and final model weights (delta) or the computed gradients. To preserve privacy, this update may be protected before transmission:

• Differential Privacy: Adding calibrated noise to the update.
• Secure Aggregation Preparation: Using cryptographic masks so the server can only decrypt the sum of all updates, not individual contributions.
• Compression: Applying techniques like quantization or sparsification to reduce upload payload size.

The server collects updates from participating clients. The core algorithmic step is model aggregation, most commonly Federated Averaging (FedAvg), which computes a weighted average of client updates. For security:

• Secure Aggregation protocols ensure the server learns only the aggregated sum, not individual updates.
• Byzantine Robust aggregation rules (e.g., median, trimmed mean) are used to filter out malicious updates from model poisoning attacks. This phase transforms many local insights into a single, improved global model.

The aggregated update is applied to the global model, creating a new model version for the next round. The server then evaluates the new global model's performance, typically on a held-out validation set. Key metrics tracked across rounds include global accuracy, loss convergence, and fairness across client distributions. This evaluation informs decisions like adjusting client selection or learning rates for subsequent rounds.

The round concludes with the server preparing for the next iteration. In synchronous FL, the server waits for all selected clients to respond or times out, creating a consistent update cycle. In asynchronous FL, clients update the global model as soon as they finish, improving efficiency at the cost of potential staleness. The total number of communication rounds is a primary determinant of the training time and the final model quality, directly impacting the communication-computation trade-off.

The foundational aggregation algorithm that defines a standard communication round. The server broadcasts the global model, clients perform local SGD, and the server computes a weighted average of the returned model updates. Its simplicity makes it the baseline, but its performance degrades significantly under non-IID data and partial client participation.

A critical phenomenon where local client models, optimized on their heterogeneous data, diverge from the global objective during multiple local update steps. This divergence accumulates over communication rounds, slowing convergence and reducing final accuracy. Algorithms like FedProx and SCAFFOLD are designed explicitly to mitigate client drift by constraining or correcting local updates.

The defining characteristic of federated data where the data distribution varies significantly across clients (non-IID). This is the root cause of client drift and makes aggregation challenging. Heterogeneity necessitates:

• Robust aggregation rules (e.g., median-based, trimmed mean).
• Personalization techniques to adapt the global model locally.
• Careful client selection strategies per round.

A cryptographic protocol that ensures privacy within a communication round. It allows the server to compute the sum of client updates without being able to inspect any individual client's contribution. This protects against a curious server reconstructing raw data from gradients. It adds computational overhead but is essential for high-assurance privacy in cross-device settings.

The practical reality in cross-device FL where only a subset of available clients participates in each communication round, due to device availability, connectivity, or sampling strategies. This requires:

• Unbiased client sampling to maintain statistical representativeness.
• Aggregation algorithms robust to missing updates.
• Asynchronous or semi-asynchronous protocols to avoid stragglers.

A primary optimization goal for federated learning, as communication is often the bottleneck compared to local computation. Techniques to reduce rounds or data per round include:

• Local steps: Performing multiple SGD steps per round.
• Compression: Sending sparsified, quantized, or sketched updates.
• Adaptive client selection: Choosing clients with the most informative updates. The trade-off is between round efficiency and final model accuracy.