Byzantine Resilience

The system withstands Byzantine faults, the most severe failure class, where faulty nodes can behave arbitrarily—sending contradictory messages, colluding, or acting maliciously. Unlike crash faults where a node simply stops, Byzantine nodes actively try to corrupt the computation. A Byzantine-resilient system guarantees safety and liveness as long as fewer than one-third of participants are faulty.

Standard Federated Averaging (FedAvg) fails under Byzantine attacks because a single malicious update can arbitrarily skew the mean. Byzantine-resilient systems replace the mean with robust statistical estimators:

• Coordinate-wise Median: Replaces each weight with the median of client updates
• Trimmed Mean: Discards the largest and smallest values before averaging
• Krum: Selects the single update closest to its neighbors in vector space
• Multi-Krum: Averages the n most mutually similar updates

Byzantine resilience relies on spatial and temporal redundancy. The system assigns identical computation tasks to multiple independent clients and compares outputs. Diversity in hardware, software stacks, and data sources prevents a single vulnerability from compromising the majority. In cross-silo federated learning, this means engaging institutions with non-overlapping failure domains.

Secure aggregation protocols ensure the server can only decrypt the final aggregated result, never individual updates. Combined with zero-knowledge proofs, a client can prove its update was computed correctly on valid data without revealing the data itself. Homomorphic encryption allows the server to perform robust aggregation math directly on ciphertexts, preventing data leakage during the filtering process.

The system maintains a persistent trust score for each client across training rounds. Updates that consistently deviate from the consensus are penalized, while historically reliable clients gain influence. This long-term memory prevents attackers from oscillating between honest and malicious behavior to evade detection. Techniques like federated data valuation using Shapley values quantify each client's marginal contribution.

Before aggregation, the server inspects incoming updates for statistical anomalies. Spectral analysis of the gradient matrix can reveal coordinated attacks. Clustering algorithms like DBSCAN separate honest updates from outliers in high-dimensional gradient space. Cosine similarity checks detect gradient direction reversal attacks designed to maximize model damage with minimal perturbation.

A targeted attack where a malicious client uploads a deliberately corrupted model update to sabotage the global model's performance or implant a backdoor. Unlike random failures, poisoning attacks are strategically crafted to maximize damage while evading detection.

• Targeted Poisoning: Introduces a backdoor that causes misclassification only for specific inputs
• Untargeted Poisoning: Degrades overall model accuracy to render it unusable
• Stealth Techniques: Attackers may constrain perturbation magnitude to mimic benign updates

A cryptographic protocol that allows a central server to compute the sum of encrypted model updates from multiple clients without being able to inspect any individual client's contribution in plaintext. This prevents the server from identifying and excluding malicious updates based on content inspection.

• Relies on secret sharing and pairwise masking
• Server learns only the aggregate, never individual vectors
• Must be paired with Byzantine-resilient aggregation to handle poisoned inputs within the encrypted sum

The fundamental challenge in federated learning arising from non-identical data distributions across clients. This natural divergence complicates Byzantine detection because it becomes difficult to distinguish between a malicious update and a benign but statistically unusual one from a client with rare data.

• Label skew: Different clients have different class distributions
• Feature skew: Same labels but different input distributions
• Quantity skew: Highly variable dataset sizes across clients

A foundational Byzantine-resilient aggregation rule that selects the single local model update that is closest to its nearest neighbors in parameter space. Krum is provably resilient when the fraction of Byzantine clients is below a theoretical threshold.

• Computes pairwise Euclidean distances between all submitted updates
• Selects the update minimizing the sum of distances to its n − b − 2 closest neighbors
• Resilient to up to b Byzantine clients in a system of n total participants

A mathematical framework that provides a provable guarantee of privacy by injecting calibrated statistical noise into model updates. While primarily a privacy mechanism, the noise addition can complicate Byzantine detection by masking the distinguishing characteristics of malicious updates.

• Controlled by the privacy budget (ε, δ) parameters
• Local DP: Noise added on each client before transmission
• Central DP: Noise added by the aggregator after secure summation
• Trade-off: Stronger privacy guarantees reduce the signal available for anomaly detection

A communication efficiency technique that reduces the size of model updates through sparsification or quantization. Compression introduces artifacts that can mimic or mask Byzantine behavior, requiring resilience mechanisms to be robust to both intentional attacks and unintentional compression distortion.

• Top-k sparsification: Transmit only the largest gradient components
• Stochastic quantization: Reduce bit precision of each parameter
• Error feedback: Accumulate compression residuals to preserve convergence