Asynchronous Aggregation (FedAsync)

The central server updates the global model immediately upon receiving a local update from any client, without waiting for a cohort of participants to finish training. This contrasts sharply with synchronous methods like FedAvg, where the server must wait for all selected clients to report back before performing aggregation. The update rule typically applies a staleness-weighted mixing coefficient to blend the incoming local model with the current global model, preventing a single fast client from dominating the optimization trajectory.

To prevent outdated updates from corrupting the global model, FedAsync applies a staleness function that decays the weight of a client's contribution based on the time elapsed since that client was selected. Common staleness functions include:

• Polynomial decay: weight = (τ + 1)^(-a), where τ is the staleness and a > 0 is a hyperparameter
• Hinge function: weight = 1 if τ ≤ threshold, else weight = 1/τ
• Constant staleness: weight = 1/(τ + 1) This mechanism ensures that slow clients still contribute, but their influence is appropriately discounted.

FedAsync employs a convex combination to mix the incoming local model with the current global model. The mixing coefficient α is a function of the staleness τ:

• Global model: W_global = (1 - α) * W_global + α * W_local
• When α is close to 1, the update aggressively incorporates the new local model
• When α is close to 0, the global model changes slowly, preserving stability This regularized mixing prevents the global model from oscillating wildly when updates arrive at unpredictable intervals, a critical property for convergence in non-IID healthcare environments.

In synchronous federated learning, a single slow device—a straggler—can delay the entire training round. FedAsync eliminates this bottleneck entirely. Clients train and transmit updates independently, and the server never blocks waiting for a specific node. This is critical in cross-device healthcare scenarios where edge devices like wearables or rural clinic servers have highly variable connectivity and compute capabilities. The system naturally accommodates devices that drop offline for extended periods without stalling global model improvement.

The FedAsync server operates on an event-driven architecture, processing updates asynchronously as they arrive. Key implementation considerations include:

• Concurrency control: The server must handle multiple simultaneous update requests, often using optimistic locking or atomic operations on the global model weights
• Update queuing: In high-throughput deployments, a message queue buffers incoming updates before sequential or batched application
• Version tracking: Each global model iteration is versioned, allowing the server to compute staleness τ as the difference between the current version and the version the client trained on

FedAsync introduces a staleness-variance tradeoff. While it eliminates synchronization overhead, the use of stale gradients introduces bias into the optimization. Theoretical analysis shows that with a properly decaying staleness function and learning rate, FedAsync converges to a stationary point under standard non-convex assumptions. Key factors affecting convergence:

• Staleness bound: The maximum allowable staleness must be bounded to guarantee convergence
• Mixing hyperparameter: The α function must satisfy α(τ) → 0 as τ → ∞ to prevent divergence
• Client selection: Uniform random selection with replacement ensures statistical consistency