Trimmed Mean Aggregation

Unlike geometric median approaches that operate on entire update vectors, trimmed mean processes each parameter independently. For every coordinate j in the model, the server collects the j-th value from all n clients, sorts them, discards the k largest and k smallest values, and computes the mean of the remaining n-2k values. This granularity ensures that an adversary poisoning only a subset of parameters cannot corrupt the entire update.

Trimmed mean provides deterministic resilience against up to k Byzantine clients when the total number of malicious nodes is bounded. The theoretical guarantee holds under the assumption that benign client updates are drawn from a symmetric, unimodal distribution around the true gradient. Key properties:

• Breakdown point: The fraction of corrupted values that can be tolerated before the estimator fails is k/n
• Optimal trimming: Setting k proportional to the expected number of adversaries maximizes robustness
• No cryptographic overhead: Unlike secure aggregation, trimmed mean operates on plaintext updates

A critical limitation arises when Byzantine nodes submit colluding adversarial updates. If all malicious clients shift a specific coordinate in the same direction, they can form a new mode in the distribution, causing the trimmed mean to converge toward the attackers' objective. This is known as the 'matching pennies' or sign-flipping attack. Mitigation strategies include:

• Combining trimmed mean with dimensionality reduction via random projection
• Using median-based alternatives like coordinate-wise median for stronger breakdown points
• Applying pre-aggregation clustering to detect and isolate coordinated adversarial groups

Trimming discards potentially valid information from the tails of the distribution, introducing a bias-variance trade-off. For normally distributed benign updates:

• No trimming (k=0): Minimum variance, zero bias, but zero Byzantine resilience
• Moderate trimming: Small efficiency loss with substantial robustness gains
• Aggressive trimming: High bias, reduced variance, but may discard legitimate high-magnitude gradients from clients with unique data distributions In non-IID federated settings common in healthcare, excessive trimming can inadvertently remove updates from outlier but valid clinical sites with rare patient populations.

Trimmed mean aggregation imposes minimal additional overhead compared to standard Federated Averaging:

• Server-side computation: O(n log n) per coordinate for sorting, where n is the number of clients
• Client-side: No modification to local training; clients transmit standard model updates
• Communication: Identical to FedAvg — no extra bits transmitted
• Memory: The server must buffer all n updates before aggregation, requiring O(n × d) storage for d-dimensional models This makes trimmed mean particularly attractive for resource-constrained edge deployments where complex cryptographic protocols are infeasible.

While both trimmed mean and Krum provide Byzantine resilience, they differ fundamentally in approach:

• Trimmed Mean: Coordinate-wise operation, discards extremes per parameter, retains majority values
• Krum: Vector-wise operation, selects the single update closest to its n-k-2 nearest neighbors
• Multi-Krum: Extends Krum by selecting multiple candidates and averaging them Trimmed mean excels when adversarial corruption is sparse across coordinates, while Krum is more robust to omniscient adversaries that corrupt all coordinates simultaneously. In practice, hybrid approaches applying trimmed mean after Krum-based client selection often yield superior results.