Federated Learning

The foundational principle of federated learning is that raw training data never leaves the client device. The model travels to the data, not the other way around. Each participating node downloads the current global model, performs training locally on its private dataset, and generates a focused model update. This update, typically a set of weight gradients, is the only information transmitted back to the server. This architecture directly addresses data residency requirements under regulations like GDPR and HIPAA by ensuring the central server never has direct access to sensitive user photos, keystrokes, or financial records.

Unlike traditional distributed training in a data center, federated learning must contend with data that is non-Independently and Identically Distributed (non-IID). Each client's local dataset reflects its unique usage patterns, creating a statistical mismatch between local data distributions and the global distribution. A user in Japan will have a different photo distribution than a user in Brazil. This heterogeneity can cause local models to diverge significantly, slowing convergence. Advanced techniques like FedProx add a proximal term to the local objective function to constrain local updates from straying too far from the global model, stabilizing training on heterogeneous data.

A cryptographic protocol that ensures the central server can only see the summed aggregate of model updates, not any individual client's contribution. Using Secure Multi-Party Computation (SMPC) techniques, clients mask their updates with pairwise secrets before transmission. The masks cancel out in the aggregate sum, revealing the total update while mathematically hiding individual vectors. This protects against an honest-but-curious server that might attempt to reconstruct private data from a single user's gradient update, a vulnerability known as gradient leakage.

To provide a formal mathematical guarantee against membership inference, federated learning systems often integrate differential privacy at the client level. Before transmitting a model update, each client clips the gradient norms to bound sensitivity and adds calibrated Gaussian noise. This ensures that the transmitted update is statistically indistinguishable whether or not any single data point was included in the local training. The central server can then track the total privacy budget spent over multiple rounds using a moments accountant, allowing the system to stop training before the privacy guarantee degrades beyond a specified epsilon threshold.

Federated learning architectures are categorized by the scale and trust model of the participating nodes:

• Cross-Device FL: Involves millions of unreliable, low-power edge devices like smartphones or IoT sensors. Clients are available unpredictably and have limited bandwidth. The primary challenge is communication efficiency and straggler mitigation.
• Cross-Silo FL: Involves a small number of reliable institutional participants, such as hospitals or banks. Each silo holds a large, curated dataset. The primary challenge is secure computation and legal compliance, often solved with Trusted Execution Environments (TEEs).

A mathematical framework that provides a provable privacy guarantee by injecting calibrated statistical noise into model updates. In federated learning, this ensures that an adversary cannot determine whether a specific individual's data was included in the training set, even by analyzing the shared gradients.

• Epsilon (ε): The privacy budget parameter controlling the trade-off between privacy and utility.
• Mechanism: Typically implemented via the Gaussian or Laplacian mechanism during gradient clipping.

A cryptographic protocol that allows the central server to compute the sum of model updates from all participating clients without being able to inspect any individual client's contribution in plaintext. This protects against an honest-but-curious server.

• Secret Sharing: Clients mask updates with pairwise secrets that cancel out upon summation.
• Dropout Robustness: The protocol is designed to recover the aggregate even if a subset of clients disconnects mid-round.

A critical privacy vulnerability where an attacker reconstructs a client's private local training data from the shared model gradients. By optimizing dummy inputs to match the transmitted gradients, pixel-accurate images or token-perfect text can be recovered.

• Deep Leakage from Gradients (DLG): An iterative optimization attack that minimizes the distance between dummy and real gradients.
• Mitigation: Gradient compression, large batch sizes, and adding differential privacy noise disrupt reconstruction fidelity.

A fundamental challenge in federated learning where local datasets across clients are not identically distributed. Client data often varies wildly in size and statistical distribution, violating the standard IID assumption of centralized training.

• Label Distribution Skew: One client may have mostly 'cats' while another has mostly 'dogs'.
• Feature Distribution Skew: Handwriting styles vary significantly across different users.
• Solution: Algorithms like FedProx add a proximal term to stabilize convergence on heterogeneous data.

A privacy breach where an adversary exploits access to model predictions or updates to reconstruct representative features of a target class. In a federated context, this can reveal sensitive statistical properties of a client's local dataset without directly accessing the raw data.

• Face Reconstruction: Attackers can generate high-fidelity images of a target identity from a facial recognition model.
• Defense: Limiting prediction confidence scores and applying mutual information constraints during training.

The foundational optimization algorithm for federated learning. Instead of sending raw data to a server, clients compute local model updates on their own devices, and the server aggregates these updates by taking a weighted average to produce a new global model.

• Communication Rounds: The global model is updated over multiple rounds of local training and aggregation.
• Fraction (C): A hyperparameter controlling what fraction of clients are sampled for computation in each round to manage stragglers.