Local Model

A local model is a temporary copy of the global model downloaded by a client (e.g., a smartphone or edge device) at the start of a communication round. It exists only for the duration of local training. Its lifecycle is:

• Instantiated from the latest global model parameters.
• Trained for a specified number of local epochs.
• Used to compute an update (the model delta).
• Discarded after the update is sent, preserving client memory. The learned knowledge is captured solely in the transmitted update.

The defining characteristic of a local model is that it trains exclusively on the client's private local dataset, which never leaves the device. This enforces a strict privacy-by-design architecture.

Key implications:

• Raw Data Isolation: Sensitive user data (e.g., personal messages, health metrics, on-device photos) is never transmitted or centralized.
• Regulatory Compliance: This architecture directly supports compliance with frameworks like GDPR and HIPAA by minimizing data movement.
• Trust Model: The server only ever sees mathematical updates (gradients or parameter deltas), not the underlying data, establishing a foundational trust boundary.

The local model's objective is to minimize loss on its local data distribution. It performs multiple steps of stochastic gradient descent (SGD) or a variant, acting as a local optimizer. This creates a fundamental tension:

• Goal: Improve performance on the client's specific data.
• Constraint: Updates must also be useful for improving the shared global model.

Under statistical heterogeneity (non-IID data), this local optimization causes client drift, where models diverge. Algorithms like FedProx add a proximal term to the local loss to anchor updates closer to the global model.

The primary output of a local model is a model update (or delta). This is the mathematical difference between the model's parameters before and after local training.

Update Calculation: Δ_i = θ_local^(t+1) - θ_global^t Where Δ_i is the update from client i, θ_global^t is the initial global model, and θ_local^(t+1) is the locally trained model.

This delta is then processed (e.g., compressed, clipped for differential privacy) and sent to the server for aggregation. The local model's role is complete once this transmission occurs.

Local models must operate within the highly variable constraints of edge devices, making them an interface for system heterogeneity.

Key variations they must accommodate:

• Compute Power: From powerful GPUs to microcontrollers.
• Memory: Limits model size and batch processing.
• Connectivity: Intermittent or slow networks affect download/upload of models and updates.
• Availability: Devices are only available for training when idle, plugged in, and on Wi-Fi.

Frameworks handle this via partial participation and straggler mitigation, where only a subset of available clients with capable local models complete a round.

While the standard local model is transient, its architecture is the foundation for personalized federated learning. Techniques modify the local model's structure to create persistent, user-specific intelligence.

Common Approaches:

• FedPer: Keeps the final layers (the "head") of the model local and personal, while collaboratively training shared base layers.
• FedRep: Learns a shared representation globally, while clients train unique local heads.
• FedBN: Keeps Batch Normalization layer statistics local to each client, accounting for feature distribution shift without adding persistent parameters.

In these paradigms, the local model evolves from a temporary worker into a lasting, tailored asset.

Federated Averaging (FedAvg) is the foundational iterative algorithm that orchestrates the training of a local model. The server coordinates multiple clients in a series of communication rounds, where each client trains its local model and sends back an update. The server aggregates these updates, typically via a weighted average, to form a new global model. FedAvg's efficiency stems from performing multiple local epochs of training per communication round, drastically reducing the required server-client exchanges compared to a naive federated SGD approach.

The global model is the central, shared neural network whose parameters are maintained by the server. It is the target of the federated learning process. In each round:

• The server broadcasts the current global model to selected clients.
• Each client uses it to initialize their local model.
• After local training, client updates (model deltas) are sent back.
• The server performs update aggregation to produce a new, improved global model. The global model represents the collaborative knowledge learned from all participating devices' data, without that data ever leaving its source.

Client drift is a critical optimization challenge caused by statistical heterogeneity (non-IID data) across clients. It occurs when local models, trained on diverse data distributions, diverge from the global objective. This divergence hinders convergence and degrades final model performance. Mitigation strategies include:

• FedProx: Adds a proximal term to the local loss function to constrain updates.
• SCAFFOLD: Uses control variates to correct for update variance.
• Gradient Clipping: Caps update norms to limit the influence of outliers. Managing client drift is essential for stable federated training.

Update aggregation is the server-side process that combines the model deltas from participating clients to update the global model. The standard method is weighted averaging, where each client's update is scaled by the size of its local dataset. This ensures the global objective aligns with the overall data distribution. Advanced aggregation techniques include:

• FedOpt Framework: Applies adaptive optimizers (e.g., Adam) to aggregated updates.
• FedAvgM: Incorporates server-side momentum for accelerated convergence.
• Secure Aggregation: Cryptographically combines updates without revealing individual contributions.

Personalized FL techniques adapt the federated paradigm to produce models tailored to individual clients' data, addressing the limitations of a single global model under high data heterogeneity. Key approaches that modify the local model paradigm include:

• FedPer: Keeps the final model layers (the "head") local and personal, while collaboratively learning the base layers.
• FedRep: Clients learn a shared representation (model body) but train unique local heads.
• FedBN: Localizes batch normalization statistics, allowing the model to adapt to local feature distribution shifts. These methods balance shared knowledge with local specialization.

This suite of techniques reduces the bandwidth and frequency of communication between clients and the server, a major bottleneck. It directly impacts how local model updates are transmitted. Core methods include:

• Update Compression: Applying quantization (reducing numerical precision) and sparsification (sending only the largest values) to shrink update size.
• Partial Client Participation: Selecting only a subset of available clients per round.
• Local Epochs: Performing multiple local training passes per round amortizes communication cost.
• Straggler Mitigation: Using asynchronous protocols or deadlines to prevent slow clients from delaying aggregation.