Byzantine Robust Aggregation

A robust statistical aggregator that discards extreme values. For each model parameter (coordinate), the server sorts all client update values, removes a predefined fraction (β) of the highest and lowest values, and computes the mean of the remaining values.

• Key Property: Highly effective against large-magnitude outlier attacks.
• Parameter: The trim fraction β must be set greater than the estimated fraction of Byzantine clients.
• Example: With β=0.2 and 100 clients, the 20 highest and 20 lowest values for each parameter are discarded before averaging the middle 60 values.

This method computes the median value for each individual model parameter across all client updates to form the new global update.

• Key Property: Extreme resilience to outlier values, as the median is unaffected by the magnitude of outliers, only their count.
• Limitation: Can converge more slowly than the mean under benign conditions due to higher variance.
• Use Case: Preferred in high-security scenarios where the threat model includes powerful adversaries capable of submitting updates with unbounded magnitude.

A geometric, non-coordinate-wise method. Krum selects the single client update vector that is closest to its nearest neighbors in Euclidean space. Multi-Krum extends this by selecting a subset of such candidate updates and averaging them.

• Mechanism: For each client's update, it calculates the sum of squared distances to its n - f - 2 closest neighbors (where f is the max Byzantine count). The update with the smallest sum is chosen.
• Strength: Defends against both large-magnitude and small, coordinated sign-flipping attacks that trimmed mean may not catch.
• Drawback: Computationally expensive O(n²) pairwise distance calculations.

A meta-aggregation defense that combines a robust selector with a robust aggregator. First, it uses a robust rule (like Krum or Multi-Krum) to select a candidate set of updates. Then, it applies coordinate-wise trimmed mean to this filtered set to produce the final update.

• Key Insight: Provides a second layer of filtering, making it robust against attacks specifically designed to fool a single robust rule.
• Guarantee: Proven to be effective under the assumption of a limited fraction of Byzantine clients.
• Result: Often achieves higher accuracy than Krum or trimmed mean alone under sophisticated attacks.

An algorithm designed to detect and mitigate Sybil attacks, where a single adversary controls many fake clients. It operates on the principle that honest clients have diverse, non-IID data, leading to divergent update directions, while Sybils will submit highly similar updates.

• Mechanism: Computes a pairwise similarity score (cosine similarity) between client update histories. Clients with highly similar update patterns receive down-weighted contributions.
• Strength: Does not require an a priori estimate of the number of malicious clients.
• Scenario: Particularly effective in cross-silo federated learning where Sybil attacks are a primary concern.

Choosing an algorithm depends on the threat model, computational budget, and performance tolerance.

• Statistical Methods (Trimmed Mean, Median): Fast, simple, excellent against large-value outliers. Weak against coordinated small-magnitude attacks.
• Geometric Methods (Krum, Bulyan): Stronger against diverse attack vectors but computationally heavier. Bulyan offers layered defense.
• Similarity-Based (FoolsGold): Specialized for Sybil/collusion detection. Lower overhead than Krum.

Trade-off: Increased Byzantine robustness often comes at the cost of slower convergence and reduced final accuracy under purely benign conditions, known as the robustness-accuracy trade-off.

Trust Scoring is a dynamic reputation system that assigns a credibility weight to each federated client based on the historical consistency and quality of their contributions. These scores are then used to weight client updates during aggregation, diminishing the influence of unreliable or malicious participants.

• Mechanism: Scores are updated each round based on metrics like:
  • Update similarity to a robust aggregate (e.g., cosine similarity with the trimmed mean).
  • Contribution to global model improvement (validation accuracy gain).
  • Behavioral consistency over time.
• Advantage over Static Rules: Adapts to changing client behavior, potentially detecting slowly deteriorating or strategic adversaries.
• Relation to Robust Aggregation: Trust scoring can be seen as a soft, weighted version of robust aggregation, whereas methods like Krum perform hard selection. They can be combined, e.g., using trust scores to pre-filter clients before applying a robust rule.

Gradient Inspection refers to server-side analysis of the statistical properties of submitted model updates (gradients/weights) to identify anomalies indicative of malicious behavior, faulty hardware, or non-IID data extremes.

• Common Inspection Techniques:
  • Norm Bounding: Clipping updates with extremely large L2 norms.
  • Distribution Analysis: Checking if a client's update vector follows an expected distribution (e.g., near other updates).
  • Geometric Analysis: Methods like Multi-Krum examine the spatial clustering of updates in high-dimensional parameter space.
• Role in the Defense Stack: Gradient inspection acts as a pre-filter or diagnostic tool. It provides the signals (e.g., "this update is an outlier") that robust aggregation rules like trimmed mean or median use to make their final aggregation decision. It's the analysis phase before the mitigation phase.