Byzantine-Resilient Aggregation

A distance-based aggregation rule that selects a single local model update as the global update. Krum computes the sum of squared Euclidean distances from each candidate update to its n - b - 2 closest neighbors, where b is the estimated number of Byzantine clients. The update with the minimal sum is chosen.

• Byzantine Tolerance: Tolerates up to b Byzantine clients out of n total clients, provided 2b + 2 < n.
• Selection Mechanism: Discards updates that are geometric outliers, assuming honest updates cluster together in parameter space.
• Computational Complexity: O(n^2 * d) where d is the number of model parameters, making it expensive for high-dimensional models.
• Variant: Multi-Krum averages the top-m scoring updates instead of selecting a single one, improving convergence stability.

A coordinate-wise aggregation strategy that sorts each model parameter independently across all client updates, discards the largest and smallest b values, and computes the mean of the remaining values.

• Mechanism: For each weight in the model, the top b and bottom b values are removed before averaging.
• Byzantine Tolerance: Effective against sign-flipping attacks and additive noise, but vulnerable to more subtle, coordinated perturbations.
• Assumption: Relies on the distribution of honest updates being symmetric and concentrated around the true mean.
• Practical Use: Often combined with differential privacy noise addition to provide a dual privacy-robustness guarantee.

The simplest robust aggregation operator. For each model parameter coordinate, the server computes the median of the values received from all clients, replacing the arithmetic mean entirely.

• Breakdown Point: Theoretically achieves the optimal breakdown point of 50%, meaning it can tolerate up to just under half of clients being Byzantine.
• Limitation: In high-dimensional, non-i.i.d. settings, the coordinate-wise median does not guarantee convergence because it ignores the joint distribution of parameters.
• Variant: Geometric Median computes a single vector that minimizes the sum of Euclidean distances to all client updates, preserving the joint structure. This is typically solved via Weiszfeld's algorithm.

A two-phase meta-aggregator designed to combine the strong outlier resistance of Krum with the variance reduction of Trimmed Mean. It specifically defends against a high-dimensional attack that can defeat coordinate-wise methods.

• Phase 1: Iteratively applies Krum to select a subset of theta candidate updates, where theta is chosen such that the subset is guaranteed to contain a majority of honest updates.
• Phase 2: Applies a variant of Trimmed Mean to the selected subset, discarding the extreme values per coordinate.
• Guarantee: Provides provable convergence under a stronger Byzantine resilience model than either Krum or Trimmed Mean alone.
• Cost: High computational overhead due to the iterative application of Krum.

A regularization-based approach that frames robust aggregation as a penalized optimization problem. Instead of filtering updates, the server adds a penalty term to the aggregation objective that is small for honest updates and large for outliers.

• Mechanism: Uses robust loss functions like Huber loss or Tukey's biweight during the aggregation step to down-weight the influence of extreme gradient values.
• Advantage: Does not require pre-specifying the number of Byzantine clients b.
• Integration: Can be implemented as a drop-in replacement for the mean in standard FedAvg.
• Trade-off: The choice of regularization hyperparameter lambda is critical and often requires federated hyperparameter tuning.

A server-side defense that maintains a momentum-based estimate of the true global gradient direction and rejects client updates that deviate significantly from this historical trajectory.

• Core Idea: Byzantine updates are likely to be inconsistent with the recent history of honest gradient directions.
• Detection: Computes the cosine similarity between each incoming update and the server's running gradient estimate. Updates below a threshold are discarded.
• Resilience: Effective against model replacement and backdoor attacks that attempt to steer the model toward a malicious objective.
• Statefulness: Requires the server to maintain persistent state across rounds, adding a minor storage overhead.

The foundational system property that Byzantine-resilient aggregation aims to achieve. It describes a distributed system's ability to reach consensus and continue operating correctly even when some components fail arbitrarily or act maliciously. In federated learning, this means the global model must converge despite clients sending corrupted gradients, random noise, or coordinated adversarial updates designed to poison the model.

A coordinate-wise robust aggregation strategy that sorts each dimension of the received gradient vectors independently and discards the largest and smallest k values before averaging the remainder. This simple yet effective defense assumes that malicious updates will manifest as extreme values. It is computationally efficient but can struggle against omniscient adversaries who craft Byzantine updates that fall within the benign distribution's range while still corrupting the model.

A highly robust, coordinate-wise aggregation rule that replaces the weighted mean with the coordinate-wise median of all client updates. It is provably resilient to arbitrary corruption from up to 50% of clients minus one. While offering strong theoretical guarantees, Median aggregation can significantly slow convergence in high-dimensional, non-IID settings because it ignores the magnitude of updates and only considers their central rank, discarding useful gradient information from honest outliers.

The primary threat vector that Byzantine-resilient aggregation defends against. In a model poisoning attack, an adversary controls one or more clients and manipulates their local training data or directly crafts malicious model updates to corrupt the global model. Goals include:

• Targeted poisoning: Causing misclassification on a specific trigger (backdoor).
• Untargeted poisoning: Maximizing global model error.
• Gradient scaling: Amplifying malicious updates to overpower honest contributions.

A complementary cryptographic protocol often confused with Byzantine resilience. Secure Aggregation (SecAgg) uses secret sharing to ensure the central server can only compute the sum of client updates without inspecting individual contributions, protecting privacy. However, SecAgg provides no defense against poisoning; a malicious client can still submit a corrupted update, and the server cannot audit it. Production systems often layer Byzantine-resilient rules on top of SecAgg for combined privacy and robustness.