Robust Aggregation

A Byzantine-resilient aggregation rule that selects a single gradient vector from the pool of client updates. Krum operates by computing the sum of squared Euclidean distances from each candidate vector to its n - b - 2 closest neighbors, where n is the total number of clients and b is the estimated number of Byzantine nodes. The vector with the minimal distance score is chosen as the global update, effectively isolating outliers. This method is computationally efficient but discards information from the majority of honest clients, making it most suitable when the adversary count is small and precisely bounded.

A coordinate-wise robust statistical aggregation technique that mitigates the influence of extreme outlier gradients. For each dimension of the gradient vector, the algorithm sorts the values submitted by all clients, discards the largest and smallest k fractions, and computes the arithmetic mean of the remaining coordinates. This approach assumes that Byzantine nodes will generate values at the distributional extremes. It is highly effective against sign-flipping attacks and additive noise, but can be vulnerable to more subtle, coordinated perturbations that fall within the trimmed range.

A foundational robust aggregation operator that replaces the mean with the coordinate-wise median of all client updates. The median's breakdown point is 50%, meaning an adversary must control more than half of the clients to arbitrarily corrupt the aggregated result. While offering strong theoretical guarantees against omniscient adversaries, median aggregation is statistically inefficient in high dimensions and can significantly slow convergence when the data distribution across honest clients is heterogeneous (non-IID). It serves as a critical baseline for evaluating more sophisticated defenses.

An extension of the Krum algorithm designed to improve statistical efficiency by selecting multiple honest gradients instead of a single vector. Multi-Krum iteratively applies the Krum scoring function, selects the best candidate, removes it from the pool, and repeats the process to build a set of trusted updates. The final global model is the average of this selected set. This approach leverages more information from the honest majority, accelerating convergence compared to single-vector Krum while maintaining the same Byzantine tolerance threshold of b < (n - 2)/2.

A meta-aggregation algorithm that combines iterative outlier rejection with a trimmed mean post-processing step. Bulyan first applies a variant of Multi-Krum to select a candidate set of gradients that are mutually close in vector space. It then performs a coordinate-wise trimmed mean on this pre-filtered set to neutralize any remaining adversarial influence that survived the initial selection. This two-stage pipeline provides a strong defense against a wide spectrum of attacks, including convergence-poisoning and omniscient Byzantine strategies, at the cost of higher computational overhead.

A defensive technique that bounds the influence of individual client updates by projecting them onto a ball of fixed radius centered around a reference point, typically the current global model or a robust estimate of the honest mean. If a client's gradient norm exceeds the clipping radius, it is scaled down to lie on the sphere's surface. This prevents malicious nodes from sending unbounded gradient explosions that dominate the aggregate. The clipping radius is a critical hyperparameter, often tuned dynamically based on the empirical distribution of historical honest gradient norms.

The foundational property that robust aggregation algorithms aim to achieve. A distributed system exhibits Byzantine resilience if it can reach a correct consensus even when an arbitrary subset of worker nodes behaves adversarially, sending arbitrary or malicious updates. This concept originates from the Byzantine Generals Problem and is the theoretical backbone of secure federated learning.

A coordinate-wise robust statistical technique that discards a fixed percentage of the most extreme values before averaging. For each model parameter coordinate, the Trimmed Mean algorithm sorts the values received from all clients, removes the largest and smallest β fraction, and computes the mean of the remainder. This simple yet effective method mitigates the impact of outlier gradients without complex distance calculations.

A defensive technique that caps the L2-norm of individual gradient vectors during training. By enforcing a maximum threshold on gradient magnitude, clipping prevents maliciously large updates from dominating the learning process and destabilizing the model. While primarily a stability tool, it serves as a lightweight first line of defense against model poisoning in distributed settings.

The application of data sanitization and anomaly detection techniques at the edge or client level in a federated network. Rather than relying solely on server-side robust aggregation, federated sanitization filters suspicious local data before training begins, preventing poisoned samples from ever contributing to local model updates. This defense-in-depth strategy combines client-side integrity checks with server-side resilience.