Federated Learning

The foundational principle of federated learning is that raw training data never leaves the client device. Each participating node downloads the current global model, trains it locally on its own private dataset, and sends only the resulting model updates (gradients or weights) back to the server. This architecture directly addresses data sovereignty requirements and regulatory constraints like GDPR by keeping sensitive information within its origin environment. The server never sees individual records, only aggregated mathematical transformations.

To prevent the central server from inspecting individual client updates—which could leak information through gradient leakage—federated learning systems employ secure aggregation. This cryptographic protocol uses multi-party computation techniques to compute the sum or average of client updates in such a way that the server can only decrypt the final aggregated result. No single client's contribution is ever exposed in plaintext to the coordinator, providing a second layer of defense against inference attacks on model updates.

Federated learning alone does not provide formal privacy guarantees—model updates can still leak information through membership inference or model inversion attacks. To achieve provable privacy, federated learning is combined with differentially private stochastic gradient descent (DP-SGD). Each client clips per-sample gradients and adds calibrated Gaussian noise before transmission. The central server tracks a privacy budget (epsilon) that quantifies the total privacy loss, ensuring that the final model's parameters do not memorize individual training examples.

Real-world federated deployments face significant statistical heterogeneity (non-IID data distributions across clients) and system heterogeneity (varying compute, network, and battery constraints). A hospital in one region may have radically different patient demographics than another, violating the IID assumption of standard optimization. Advanced algorithms like FedProx add proximal terms to stabilize convergence under heterogeneous conditions, while Agnostic Federated Learning optimizes for worst-case client distributions to ensure fairness across the entire population.

Federated learning deployments fall into two distinct categories:

• Cross-silo: A small number of reliable, stateful clients—typically organizations like hospitals or banks—each holding large datasets. Clients are almost always available and have substantial compute resources.
• Cross-device: Massive populations of unreliable, stateless edge devices—smartphones, IoT sensors, wearables—each holding small, ephemeral datasets. Clients are intermittently available, and training must tolerate dropout and limited bandwidth. These scenarios demand fundamentally different selection, aggregation, and communication strategies.

A mathematical framework providing provable privacy guarantees by adding calibrated noise to model updates before aggregation. In federated learning, DP ensures that the central server cannot distinguish whether a single client's data was included in the training round.

• Epsilon (ε): Quantifies the privacy loss budget; lower values mean stronger guarantees
• DP-SGD: Clips per-sample gradients and injects Gaussian noise during local training
• Prevents gradient leakage attacks that reconstruct private inputs from shared updates

A cryptographic protocol that computes the sum or average of client model updates without revealing any individual client's contribution to the coordinating server. The server only sees the aggregated result, never the raw gradients.

• Uses Shamir secret sharing or masking vectors to blind individual updates
• Protects against honest-but-curious servers
• Essential for cross-silo federated learning in healthcare and finance

An attack that reconstructs private training data from publicly shared model gradients. In federated learning, an adversary intercepting gradient updates can iteratively optimize dummy inputs to match the observed gradients, recovering images, text, or tabular records.

• Deep Leakage from Gradients (DLG): Matches dummy gradients to real gradients via L-BFGS optimization
• Effective against high-dimensional inputs like images
• Mitigated by gradient clipping, compression, and secure aggregation

An attack that exploits access to a trained model's parameters and prediction API to reconstruct representative samples of the private training data. In federated settings, the shared global model becomes the attack surface.

• Reconstructs class representatives (e.g., a face matching a specific identity)
• Exploits confidence scores and softmax outputs
• Defended by limiting prediction API granularity and applying DP during training

An encryption scheme enabling computation directly on ciphertexts. In federated learning, clients encrypt their model updates before transmission; the server aggregates encrypted values and returns an encrypted global model that only clients can decrypt.

• Partially Homomorphic Encryption (PHE): Supports only addition or multiplication
• Fully Homomorphic Encryption (FHE): Supports arbitrary computation but incurs high computational overhead
• Eliminates the need to trust the aggregation server entirely

An integrity attack where malicious clients submit crafted model updates designed to corrupt the global model. Unlike centralized poisoning, federated attackers exploit the server's inability to inspect raw training data.

• Model replacement: Attacker scales their update to dominate aggregation
• Backdoor injection: Implants triggers that cause targeted misclassification
• Defended by robust aggregation (e.g., Krum, trimmed mean) and update anomaly detection