Inferensys

Glossary

Federated Averaging (FedAvg)

Federated Averaging (FedAvg) is the foundational optimization algorithm for federated learning that combines locally trained model updates from distributed clients by computing a weighted average to produce an improved global model without centralizing raw data.
Data scientist building training data pipeline on laptop, data preprocessing visible, technical workspace.
OPTIMIZATION ALGORITHM

What is Federated Averaging (FedAvg)?

The foundational optimization algorithm for federated learning that combines locally trained model updates from distributed clients by computing a weighted average to produce an improved global model without centralizing raw data.

Federated Averaging (FedAvg) is a distributed optimization algorithm that trains a shared global model by averaging the stochastic gradient descent (SGD) updates computed independently on decentralized client datasets. Instead of sending raw training data to a central server, each client downloads the current global model, computes an update using its local data, and transmits only the resulting model weights or gradients back to the aggregation server.

The server then constructs a new global model by computing a weighted average of these client updates, typically weighted by the number of local training samples each client processed. This iterative process repeats over multiple communication rounds, enabling collaborative learning while preserving data sovereignty and addressing the statistical challenges of non-IID data distributions across heterogeneous clients.

Algorithmic Foundations

Key Characteristics of FedAvg

The core mechanisms that make Federated Averaging the dominant optimization strategy for privacy-preserving, distributed model training across decentralized telecom infrastructure.

01

Local Stochastic Gradient Descent (SGD)

Each participating client performs multiple steps of stochastic gradient descent on its local data partition before communicating. This contrasts with traditional distributed training that requires gradient synchronization after every batch. By executing E local epochs, FedAvg significantly reduces communication rounds—a critical advantage in bandwidth-constrained radio access networks. The local update rule follows: w_k = w_k - η∇L(w_k; batch) where η is the learning rate and L is the local loss function.

02

Weighted Model Averaging

The central server aggregates client updates using a weighted average proportional to each client's dataset size n_k. The global update rule is: w_global = Σ (n_k / n_total) * w_k. This weighting mechanism ensures that clients with more representative data exert proportionally greater influence on the global model, while preventing any single large dataset from dominating convergence. The approach naturally handles statistical heterogeneity across base stations with varying traffic patterns.

03

Communication Round Structure

FedAvg operates in discrete synchronized rounds:

  • Round 1: Server broadcasts current global model w_t to selected clients
  • Round 2: Each client k initializes with w_t, trains for E local epochs on private data D_k, producing updated weights w_k
  • Round 3: Clients transmit only model deltas Δw_k = w_k - w_t back to the server
  • Round 4: Server computes weighted average of deltas and updates global model This structure minimizes uplink bandwidth consumption—critical for edge inference offloading scenarios.
04

Convergence Under Non-IID Data

A defining challenge for FedAvg is statistical heterogeneity where local datasets are non-IID. In telecom deployments, different base stations observe fundamentally different traffic distributions—urban macrocells see dense daytime demand while rural cells exhibit sparse, bursty patterns. This causes client drift: local models diverge toward their own local optima, and naive averaging can produce a global model that performs poorly on all distributions. The weight divergence is proportional to E * η * (gradient dissimilarity), requiring careful tuning of local epochs.

05

Client Selection and Straggler Resilience

Each round, the server selects a fraction C of available clients for participation. This client selection strategy balances model convergence speed against communication overhead. FedAvg is inherently vulnerable to stragglers—slow devices that delay round completion. Production deployments implement timeout thresholds: if a client fails to report within T_max, its update is discarded for that round. This trades off some statistical accuracy for predictable latency, essential for zero-touch network provisioning workflows.

06

Privacy-Preserving Foundation

FedAvg provides a baseline privacy guarantee: raw training data never leaves the base station. Only mathematical weight updates are transmitted. However, these updates are not inherently cryptographically private—gradients can leak information through model inversion attacks. Production telecom deployments layer FedAvg with:

  • Secure Aggregation to prevent the server from inspecting individual updates
  • Differential Privacy via Gaussian noise injection before transmission
  • Gradient clipping to bound per-example sensitivity This layered approach satisfies data sovereignty requirements for cross-jurisdiction deployments.
FEDERATED AVERAGING

Frequently Asked Questions

Clear, technically precise answers to the most common questions about the foundational algorithm that powers privacy-preserving distributed model training across telecom infrastructure.

Federated Averaging (FedAvg) is the foundational optimization algorithm for federated learning that constructs a global model by computing a weighted average of locally trained model updates from distributed clients, without ever centralizing raw data. The algorithm operates in synchronized communication rounds. In each round, a central server distributes the current global model parameters to a selected subset of clients. Each client then performs multiple steps of stochastic gradient descent (SGD) on its local private dataset, producing an updated model. The server collects these client updates and aggregates them by taking a weighted average, where each client's weight is typically proportional to the size of its local training dataset. This aggregated update becomes the new global model, and the process repeats. The key insight is that averaging model parameters is mathematically equivalent to averaging gradients when clients start from the same initialization, enabling the global model to learn from decentralized data while preserving privacy. FedAvg dramatically reduces communication rounds compared to earlier federated SGD approaches by allowing clients to perform multiple local epochs before uploading, making it practical for bandwidth-constrained telecom environments where base stations train on sensitive user data.

Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.