Federated Learning

The core principle of federated learning is that raw data never leaves its source device or server. A global model is distributed to local nodes, where training occurs on private data silos. Only model updates—such as gradients or weights—are transmitted back to a central aggregation server. This architecture is essential for compliance with data residency laws like GDPR and for protecting sensitive financial transaction records in fraud detection consortia.

The foundational algorithm for federated learning, FedAvg, works through iterative rounds:

• Client Selection: A subset of nodes is selected for the training round.
• Local Training: Each client performs multiple epochs of stochastic gradient descent (SGD) on its local data.
• Model Aggregation: The server collects the updated model weights and computes a weighted average, typically proportional to the size of each client's local dataset. This process repeats until the global model converges, effectively learning from the entire distributed dataset without ever seeing it.

A critical security layer that prevents the coordinating server from inspecting individual client updates. Secure aggregation uses cryptographic techniques to ensure the server can only compute the sum of all client model updates, not any single client's contribution. This protects against gradient leakage attacks, where an adversary could reconstruct private training data from raw model gradients. In financial fraud detection, this prevents a competitor or malicious actor from inferring a bank's specific transaction patterns or customer behaviors from its model updates.

Federated learning is often combined with differential privacy to provide a formal mathematical guarantee against membership inference. Before a client transmits its model update, it clips the gradients and adds calibrated noise—typically from a Gaussian or Laplace distribution. This ensures that the global model's parameters do not reveal whether any single transaction was included in a local training dataset. The privacy budget, denoted by epsilon (ε), quantifies the trade-off between model accuracy and the strength of the privacy guarantee.

Unlike centralized training, data in federated learning is almost never independently and identically distributed (non-IID). Different banks or devices have skewed class distributions and feature representations. For example, one bank's fraud patterns may be predominantly credit card chargebacks, while another's are wire transfer anomalies. Advanced algorithms like FedProx and SCAFFOLD introduce proximal terms or control variates to correct for this statistical heterogeneity, preventing client drift and ensuring stable global model convergence across diverse financial environments.

In a consortium of financial institutions, a malicious or faulty participant could attempt to poison the global fraud detection model by sending arbitrary, destructive updates. Byzantine-robust aggregation rules replace simple averaging with statistical defenses:

• Krum: Selects the update that is closest to a majority of other updates.
• Trimmed Mean: Discards extreme values for each parameter before averaging.
• Median: Uses the coordinate-wise median of all updates. These techniques ensure the collaborative model remains accurate and trustworthy even when some participants are compromised or malfunctioning.

A mathematical framework that provides a quantifiable privacy guarantee by injecting calibrated noise into model updates before they leave a client device. Governed by the privacy budget (epsilon), it ensures that an adversary cannot determine whether any single data point was included in training. Common mechanisms include:

• Gaussian Mechanism: Adds noise drawn from a normal distribution
• Laplace Mechanism: Adds noise from a Laplace distribution In federated learning, user-level differential privacy protects entire local datasets, not just individual records.

A class of aggregation rules designed to tolerate malicious clients or system failures that send arbitrary, incorrect updates to derail the training process. Standard FedAvg is vulnerable to a single Byzantine update poisoning the global model. Robust alternatives include:

• Krum: Selects the update most similar to its neighbors
• Trimmed Mean: Discards extreme values per coordinate
• Median: Uses coordinate-wise median instead of mean These techniques are critical when federated learning spans untrusted or heterogeneous devices.

The foundational algorithm that makes federated learning practical. Instead of sending raw data to a central server, each client performs multiple steps of stochastic gradient descent (SGD) on its local data and sends only the resulting model weights. The server computes a weighted average of these updates, typically proportional to the size of each client's local dataset. This reduces communication rounds by 10-100x compared to naive distributed SGD, making it feasible for mobile devices and edge deployments with constrained bandwidth.

A distributed deep learning paradigm where the neural network is physically partitioned between client and server. The client processes initial layers and transmits only intermediate activations (smashed data) rather than raw inputs. The server completes the forward pass and backpropagates gradients to the cut layer. Key advantages:

• Client never shares raw data or model gradients
• Server never sees the input layer architecture
• Suitable for vertically partitioned data where features are split across organizations Often combined with homomorphic encryption on activations for defense-in-depth.

A critical privacy vulnerability where an honest-but-curious server reconstructs private training data from raw model gradients shared during federated training. Attack methods like Deep Leakage from Gradients (DLG) iteratively optimize dummy inputs to match observed gradients, often recovering pixel-accurate images or verbatim text within dozens of iterations. Mitigations include:

• Secure aggregation to hide individual gradients
• Differential privacy noise injection
• Gradient compression to reduce information content This attack vector is why raw gradient sharing is considered unsafe in production systems.