Model Broadcast

In federated learning, the downlink (server-to-client) bandwidth is often more constrained than the uplink, especially in mobile or satellite networks. Transmitting a full model (e.g., a 100+ MB neural network) to thousands of clients each round creates a significant communication bottleneck. This makes optimizing the broadcast phase critical for overall system efficiency and training time.

The payload size is determined by the global model's architecture and parameter precision. For example:

• A ResNet-50 model has ~25 million parameters.
• Transmitting at 32-bit full precision requires ~100 MB per client.
• Techniques like model pruning and quantization (e.g., to 8-bit) are applied before broadcast to reduce this size, directly lowering bandwidth consumption and client download time.

The server does not broadcast to all available clients in each round. Instead, it uses a client selection strategy to choose a cohort. Selection is based on:

• System criteria: Device connectivity, battery level, and available compute.
• Statistical criteria: Data distribution or utility to improve model convergence.
• Adaptive strategies: Dynamically selecting clients to maximize learning progress per communication cost. This selective broadcast is a primary method for managing total downlink volume.

The broadcast acts as the synchronization barrier for a federated learning round. It ensures all participating clients in a given round start local training from the identical global model state. This synchronization is crucial for the convergence of synchronous aggregation algorithms like Federated Averaging (FedAvg). Asynchronous protocols modify this to allow stale model broadcasts, trading synchronization for efficiency.

To reduce broadcast cost, the model is often compressed. Common techniques include:

• Quantization: Reducing parameter precision from 32-bit floats to lower bitwidths (e.g., 8-bit).
• Pruning: Broadcasting only the weights of a sparsified sub-network.
• Delta Encoding: Transmitting only the difference from a previously known model state, if applicable.
• Entropy Coding: Applying lossless compression (e.g., Huffman coding) to the quantized/ pruned weights. The chosen method creates a trade-off between compression ratio and potential fidelity loss.

Broadcast over unreliable edge networks (e.g., cellular, IoT) requires robustness. Strategies include:

• Forward Error Correction (FEC): Adding redundant data to allow clients to recover from packet loss without re-transmission.
• Erasure Coding: Splitting the model into fragments so the full model can be reconstructed from a subset.
• Progressive Transmission: Sending the most critical parameters (e.g., lower layers) first to allow partial training if a connection drops. These mechanisms ensure training continuity despite network volatility.

Uplink communication is the transmission of compressed model updates (e.g., gradients, parameters) from the distributed clients back to the central server. This phase is typically the primary bandwidth bottleneck in federated systems due to:

• Asymmetric network links (slower upload vs. download speeds).
• The massive aggregate data volume from thousands of clients.
• It is the primary target for compression techniques like gradient sparsification and quantization to reduce total communication cost.

A communication round is the fundamental iterative cycle in federated learning. Minimizing the total number of rounds is a primary goal of communication-efficient algorithms. Each round consists of three sequential phases:

1. Model Broadcast: Server sends the global model to a selected client cohort.
2. Local Training: Clients perform Stochastic Gradient Descent on their private data.
3. Uplink Aggregation: Clients send updates; server aggregates them (e.g., via Federated Averaging). The round complexity—the number of rounds needed to reach target accuracy—directly determines system latency and total bandwidth consumption.

Partial participation is a system constraint where only a subset of available clients is selected for each training round. It is a fundamental necessity—not just an optimization—in large-scale federated learning due to:

• Stragglers: Devices that are offline, slow, or have poor connectivity.
• System Heterogeneity: Variations in compute power, battery life, and data plans.
• Scalability: Server cannot handle simultaneous connections from millions of devices. This constraint inherently limits per-round communication volume but introduces statistical challenges, as the aggregated update is based on a non-representative sample.

Client drift is a convergence-harming phenomenon where local client models diverge from the global objective. It is caused by performing multiple steps of local Stochastic Gradient Descent on statistically heterogeneous (non-IID) data. Drift manifests as:

• Noisy or biased aggregate updates sent to the server.
• Slower convergence, requiring more communication rounds.
• Potential convergence to a sub-optimal global model. Algorithms like FedProx (using a proximal term) and SCAFFOLD (using control variates) are explicitly designed to correct for client drift, making communication more efficient by ensuring each round provides a more accurate update direction.

Hierarchical Federated Learning is a multi-tier communication architecture that introduces intermediate aggregators (e.g., edge servers, cluster heads) between end devices and the central cloud server. This topology directly addresses model broadcast and uplink inefficiencies by:

• Reducing Long-Haul Communication: The edge server performs local aggregation of updates from its cell of clients, then sends a single consolidated update to the cloud.
• Localized Model Broadcast: The edge server can re-broadcast the global model to its clients, reducing load on the central server.
• Enabling cross-silo federated learning between organizations or data centers.

Asynchronous Federated Learning is a communication protocol where the central server aggregates client updates as soon as they arrive, without waiting for a synchronized round to finish. This contrasts with the synchronous, round-based paradigm of standard model broadcast. Key characteristics:

• Eliminates Straggler Delays: Faster clients do not wait for slower ones, improving hardware utilization.
• Introduces Staleness: Updates are computed from an outdated global model. Staleness-aware aggregation techniques (e.g., weighting updates based on their age) are required to maintain convergence stability.
• Complicates the model broadcast phase, as the server may need to maintain multiple model versions or send targeted updates.