Global Model

The global model's parameters are the single source of truth, maintained and versioned by a central coordinating server. This server is responsible for the critical aggregation step, combining client updates (model deltas) to produce a new, improved global state. The model architecture—layer types, sizes, and connectivity—is defined centrally and is identical for all participating clients, ensuring compatibility for aggregation.

The global model does not train on a centralized dataset. Instead, it improves through an iterative federated training loop:

• Broadcast: The server sends the current global model to a selected subset of clients.
• Local Training: Each client trains the model on its private data for several local epochs.
• Aggregation: The server collects the resulting model deltas and applies a weighted average (e.g., Federated Averaging) to update the global parameters. This cycle repeats for hundreds or thousands of communication rounds, enabling collaborative learning from distributed data silos.

A core characteristic is that the global model is trained without ever seeing raw client data. Clients only share parameter updates (gradients or weights), not their private datasets. This architecture provides a foundational privacy guarantee. For stronger, formal guarantees, this process can be augmented with secure aggregation protocols (which hide individual updates) and differential privacy mechanisms (which add calibrated noise to updates), making it suitable for regulated industries like healthcare and finance.

A well-converged global model represents a consensus solution that performs reasonably well on the aggregate data distribution across all clients. The standard weighted averaging aggregation scheme, where updates are weighted by client dataset size, explicitly optimizes for this population-level objective. However, its performance on any single client's unique data distribution (which is often non-IID) may be suboptimal, leading to the need for personalized federated learning techniques that adapt the global model locally.

The global model must be robust to extreme practical constraints. System heterogeneity—variations in client hardware, connectivity, and availability—is the norm. Algorithms like FedAvg are designed for partial client participation, where only a fraction of clients are active per round. Techniques for straggler mitigation (e.g., asynchronous updates, deadlines) and update compression (e.g., quantization) are essential to ensure the global model can converge efficiently in real-world, unreliable edge networks.

While the global model itself is a single, shared entity, it often serves as the starting point or foundational base for generating personalized models for individual clients. Approaches include:

• FedPer: Keeping the global base layers shared while learning local, personalized head layers.
• Fine-tuning: Taking the converged global model and performing additional local training on a client's specific data.
• Meta-learning: Using the global model as a meta-initialization that can be rapidly adapted to new clients. Thus, the global model's quality directly enables effective downstream personalization.

Federated Averaging (FedAvg) is the foundational iterative algorithm that creates the global model. The server coordinates multiple clients: it broadcasts the current global model, clients perform local training for several epochs, and the server aggregates their model updates via a weighted average to produce the next global model iteration.

• Core Mechanism: Iterative averaging of client model parameters.
• Key Innovation: Enables collaborative training without raw data exchange.
• Standard Baseline: The reference algorithm against which all federated optimization variants are compared.

A local model is a client-specific instance of the global model. At the start of a communication round, a client downloads the current global model. It then performs local training on its private dataset for a specified number of local epochs. The resulting trained model is used to compute a model delta (the update) which is sent back to the server. The local model is ephemeral; its sole purpose is to generate an update to improve the shared global model.

Update aggregation is the server-side process that combines individual client updates into a new global model. The standard method is weighted averaging, where each client's model delta is scaled by the size of its local dataset. This ensures the global objective aligns with the overall data distribution.

• Primary Method: Weighted average of model parameters.
• Protocols: Can be synchronous (wait for all clients) or asynchronous (update upon receipt).
• Advanced Forms: Algorithms like FedProx or FedOpt modify the aggregation logic to improve convergence.

Personalized Federated Learning refers to techniques that produce models tailored to individual clients' data distributions, rather than a single global model. This addresses the core challenge of statistical heterogeneity (non-IID data).

• Common Approaches:
  • FedPer: Keeps final model layers (the "head") local and personal.
  • FedRep: Learns a shared representation globally with local heads.
  • Meta-Learning: Uses the federated process to learn a model that can be quickly adapted per client.
• Goal: Balance shared knowledge with local specialization.

A Federated Learning Orchestrator is the central server software platform that manages the entire lifecycle of the global model. It is responsible for:

• Client Selection & Management: Choosing which devices participate in each round.
• Model Distribution: Broadcasting the global model to selected clients.
• Update Aggregation: Applying algorithms like FedAvg to combine received updates.
• Convergence Monitoring: Tracking global model performance and training metrics.
• Security & Privacy: Enforcing protocols for secure aggregation and differential privacy.

Frameworks like TensorFlow Federated, Flower, and PySyft provide orchestrator capabilities.

Statistical Heterogeneity, or non-IID data, is the defining characteristic of federated learning where the data distribution differs significantly across clients. This is the primary challenge for global model convergence.

• Causes: User behavior, geography, device type, and time create unique local datasets.
• Consequence: Leads to client drift, where local models diverge from the global objective.
• Solutions: Algorithms like FedProx, SCAFFOLD, and FedBN are explicitly designed to mitigate the effects of heterogeneity on the global model.