Federated Averaging (FedAvg)

The core mechanism of FedAvg involves each client performing multiple steps of Stochastic Gradient Descent on its local data partition before communicating. Instead of sending raw gradients, the client transmits an updated local model state. This contrasts with FedSGD, where clients compute a single gradient step and send it immediately. By performing multiple local epochs, FedAvg drastically reduces the number of communication rounds required for convergence, making it practical for bandwidth-constrained factory networks.

The 'averaging' step occurs on a central orchestration server. The server collects the model weights from all participating clients in a given round and computes a weighted average to produce the new global model. The standard weighting scheme is proportional to the size of each client's local dataset:

• Weighted Update: w_global = Σ (n_k / n) * w_k
• Equal Contribution: A simple arithmetic mean when dataset sizes are unknown or intentionally ignored. This step is the only point of model fusion and does not require the server to inspect raw production data.

FedAvg is explicitly designed to be communication-bound, not computation-bound. By increasing the number of local training epochs (E) and adjusting the local batch size (B), the algorithm trades local compute cycles for a significant reduction in network I/O. This is critical for factory fleets where uplink bandwidth may be limited or intermittent. Key tuning parameters include:

• Local Epochs (E): More epochs mean fewer communication rounds.
• Client Fraction (C): Only a random subset of clients participates in each round, preventing server bottlenecks.

A defining challenge for FedAvg is statistical heterogeneity, where local factory datasets are non-identically distributed (Non-IID). Unlike data-center training with shuffled data, one factory's sensor logs may represent normal operations while another's contain rare failure modes. While standard FedAvg can diverge under extreme Non-IID conditions, its foundational structure allows for proximal modifications. The algorithm's robustness is directly tied to the data shift across the fleet, making it a baseline against which more advanced algorithms like FedProx are measured.

FedAvg provides a baseline layer of data locality by design. Raw production telemetry, proprietary process parameters, and defect imagery never leave the factory floor. Only model deltas—the mathematical differences between the trained local model and the received global model—are transmitted. However, these weight updates are not inherently cryptographically secure. In production, FedAvg is almost always paired with a Secure Aggregation protocol to prevent the server from inspecting individual client updates, which could otherwise be vulnerable to model inversion attacks.

The theoretical convergence of FedAvg relies on specific assumptions. Under conditions of convex loss functions and IID data distributions, the algorithm provably converges to the global optimum. In the non-convex, Non-IID reality of deep neural networks, convergence is empirical and sensitive to hyperparameter tuning. Key factors affecting stability include:

• Learning Rate Decay: A decaying schedule prevents oscillation.
• Client Drift: Occurs when local models diverge too far from the global model during local training.
• Number of Local Steps: Too many local steps on divergent data can prevent global convergence.

The primary statistical challenge that FedAvg must overcome. In factory fleets, Non-IID data means that the sensor distributions at Plant A (producing high-speed widgets) differ drastically from Plant B (producing heavy machinery). This violates the standard machine learning assumption of identical distribution.

• Label skew: Plant A sees mostly 'normal' parts, Plant B sees 'defective' parts.
• Feature skew: Vibration sensors differ by machine vintage.
• Impact: Naive FedAvg can diverge or fail to converge on Non-IID data, requiring robust variants like FedProx.

A cryptographic protocol that ensures the central server can only compute the sum of model updates without ever inspecting an individual factory's gradient. This prevents gradient leakage attacks where raw data could be reconstructed from model updates.

• Secret sharing: Each client masks its update with random noise that cancels out only when all updates are summed.
• Threshold security: The server learns nothing unless a minimum number of clients participate.
• Integration: Often paired with FedAvg to satisfy strict industrial data governance requirements.

A mathematical guarantee that the inclusion or exclusion of a single factory's data point does not significantly change the model's output. Applied to FedAvg, differential privacy clips and noises gradients before aggregation.

• Epsilon parameter: Quantifies the privacy loss budget; lower epsilon means stronger privacy.
• Central vs. Local DP: Central DP adds noise at the server; local DP adds noise at each client before transmission.
• Trade-off: Stronger privacy guarantees degrade model utility, requiring careful calibration for quality inspection tasks.

A communication efficiency technique that reduces the bandwidth cost of transmitting model updates from edge devices to the aggregation server. Essential for factory floors with constrained or intermittent connectivity.

• Sparsification: Transmit only the top-k largest gradient values, zeroing out the rest.
• Quantization: Reduce gradient precision from 32-bit floats to 8-bit integers or even 1-bit signs.
• Error feedback: Accumulates compression errors locally to prevent model divergence over multiple rounds.

The resilience property required when some factory nodes in the federation behave arbitrarily or maliciously. A single compromised edge device uploading poisoned updates can corrupt the global model in standard FedAvg.

• Attack vectors: Data poisoning, model replacement, and free-riding attacks.
• Defense mechanisms: Robust aggregation rules like Krum or median-based aggregation that filter out outlier updates.
• Relevance: Critical for multi-tenant industrial platforms where competitors may share a federation.