Gossip Aggregation (GossipAvg)

Nodes in a gossip network communicate exclusively with a randomly selected subset of neighbors rather than a central coordinator. Each node maintains a local view of the network graph and initiates push-pull exchanges. This topology eliminates the single point of failure inherent in centralized federated averaging (FedAvg). The communication pattern follows an epidemic protocol, where information propagates through the network with a logarithmic spread rate, ensuring all nodes converge to a consistent global model state without requiring global synchronization barriers.

GossipAvg implements a distributed consensus mechanism where each node computes a pairwise weighted average of its local model with a neighbor's model upon contact. The update rule follows:

• Node i and Node j exchange their current parameter vectors w_i and w_j
• Both nodes update to (w_i + w_j) / 2 (or a weighted variant)
• Over multiple gossip rounds, all local models converge to the network-wide average

This process mathematically approximates the global Federated Averaging result without requiring any node to observe all individual updates, preserving privacy through transitive information diffusion.

Gossip protocols exhibit graceful degradation under adverse network conditions. Key resilience properties include:

• Straggler tolerance: Slow or temporarily disconnected nodes do not block global progress; the network continues averaging among available peers
• Dynamic membership: Nodes can join or leave the network at any time without reconfiguration, making the protocol ideal for cross-device federated learning on mobile or edge hardware
• Byzantine resilience: When combined with robust aggregation rules like Krum or Trimmed Mean, gossip topologies can filter out malicious updates without a trusted central authority

The lack of a synchronization barrier means the protocol naturally adapts to heterogeneous hardware capabilities and intermittent connectivity.

The convergence rate of GossipAvg depends on the spectral gap of the network's communication graph. For a connected graph with adjacency matrix A and degree matrix D, the mixing time is bounded by the second-largest eigenvalue of the normalized Laplacian. Key theoretical properties:

• Symmetric doubly-stochastic matrices: When averaging weights are symmetric, the network preserves the global sum invariant, ensuring unbiased convergence
• Exponential convergence: Under mild connectivity assumptions, the mean squared error between local models and the true average decreases exponentially with the number of gossip rounds
• Push-sum protocols: For directed or asymmetric graphs, push-sum gossip corrects for degree imbalances by tracking a scalar weight that compensates for non-doubly-stochastic exchanges

Gossip aggregation trades total communication volume for decentralized parallelism. Each node communicates with only O(log n) peers per round on well-connected graphs, compared to O(n) for a central server. Efficiency characteristics:

• Bandwidth distribution: Communication load is evenly distributed across all participants, avoiding server-side bottlenecks
• Overlap with computation: Nodes can perform local training while concurrently gossiping with neighbors, hiding communication latency behind computation
• Scalability: The protocol scales to thousands of nodes without requiring hierarchical aggregation tiers, though clustered gossip variants can further optimize for geographic locality

This makes gossip protocols particularly suitable for edge AI deployments where centralized infrastructure is unavailable or undesirable.

Gossip aggregation provides a distinct privacy profile compared to centralized secure aggregation. Since no single node observes all updates, differential privacy guarantees can be amplified through the iterative mixing process. Key considerations:

• Local differential privacy: Each node can inject noise before sharing, with the gossip process providing additional privacy amplification by iteration
• No trusted aggregator: The absence of a central server eliminates the need for complex secure multi-party computation or homomorphic encryption to protect against an honest-but-curious aggregator
• Information leakage trade-off: Intermediate model states are exposed to immediate neighbors, requiring careful consideration of neighborhood trust assumptions in highly sensitive healthcare deployments

The foundational centralized aggregation algorithm that GossipAvg seeks to decentralize. A coordinating server computes a weighted average of local model updates, where weights are proportional to local dataset sizes. While simple and effective for IID data, FedAvg introduces a single point of failure and communication bottleneck that gossip protocols eliminate by distributing the averaging process across peer-to-peer edges.

A hybrid topology bridging centralized and fully decentralized paradigms. Edge servers perform intermediate model averaging on client updates within local clusters before a central cloud server executes final global aggregation. This reduces latency compared to pure FedAvg but retains a hierarchical trust model. GossipAvg eliminates these intermediate aggregation points entirely, enabling direct peer-to-peer synchronization without infrastructure dependencies.

An aggregation scheme that updates the global model immediately upon receiving an update from any single client, eliminating synchronization barriers. This naturally handles straggler devices common in hospital networks. GossipAvg extends this asynchrony to a fully decentralized setting where nodes exchange updates with randomly selected neighbors at independent intervals, creating a robust, non-blocking diffusion process without any central model repository.

A class of robust aggregation rules ensuring correct convergence when malicious or corrupted nodes submit arbitrary updates. Techniques like Krum and Trimmed Mean filter outlier gradients before averaging. In gossip networks, BFT mechanisms must be adapted for local neighborhood validation—each node independently verifies received updates against statistical norms before incorporating them, preventing adversarial contamination from propagating through the peer-to-peer graph.

A cryptographic protocol enabling a server to compute the sum of encrypted client updates without inspecting individual contributions. In gossip-based systems, secure aggregation becomes a multi-party computation challenge where neighboring nodes must collaboratively compute pairwise averages without revealing their private model parameters. Techniques like pairwise masking and secret sharing are adapted to the peer-to-peer topology to maintain differential privacy guarantees.

A communication-efficient technique that stochastically quantizes gradient vectors to low-precision representations before transmission, drastically reducing bandwidth. In gossip networks where nodes exchange updates frequently with multiple peers, quantization is critical for scalability. Each gossip message carries compressed model deltas, enabling high-frequency synchronization over constrained hospital network links without saturating available bandwidth.