Federated Learning Architecture

The orchestrator node responsible for coordinating the training process without ever accessing raw data.

• Distributes the initial global model to participating clients
• Receives encrypted model updates (gradients or weights) from edge devices
• Applies a fusion algorithm such as Federated Averaging (FedAvg) to combine updates into a new global model
• Manages client selection, dropout handling, and communication rounds
• Often implements Secure Aggregation protocols to ensure individual updates remain cryptographically private even from the server itself

A lightweight training executor deployed on edge hardware such as smartphones, IoT sensors, or hospital servers.

• Downloads the current global model checkpoint from the aggregation server
• Performs local training on private, on-device data for a configured number of epochs
• Computes a model delta representing the learned weight adjustments
• Applies differential privacy noise injection before transmitting updates
• Must operate within strict compute, memory, and battery budgets typical of edge silicon

A cryptographic mechanism ensuring the aggregation server can only compute the sum of client updates without inspecting individual contributions.

• Uses pairwise masking and secret sharing between clients
• Relies on a trusted third party or Shamir's Secret Sharing for dropout robustness
• Guarantees that even a compromised server cannot reconstruct a single client's model update
• Critical for compliance with GDPR and HIPAA in cross-silo medical or financial deployments

The foundational fusion algorithm that combines locally trained model weights into a single improved global model.

• Each client trains on its local data for E epochs and returns updated weights
• The server computes a weighted average of client models, typically proportional to local dataset size
• Strikes a balance between communication efficiency and model convergence
• Variants like FedProx add a proximal term to handle statistical heterogeneity across non-IID client data distributions

A mathematically rigorous privacy guarantee applied to client updates before transmission.

• Adds calibrated Gaussian or Laplacian noise to model gradients
• Bounded by a privacy budget (ε, δ) that quantifies the maximum information leakage
• Implements clipping to bound the influence of any single data point
• Enables formal guarantees against membership inference attacks and model inversion
• Often combined with secure aggregation for a defense-in-depth privacy posture

Techniques to reduce the bandwidth bottleneck of transmitting high-dimensional model updates over constrained networks.

• Gradient quantization reduces 32-bit floats to 8-bit or even 1-bit representations
• Gradient sparsification transmits only the top-k largest gradient values per layer
• Federated distillation exchanges model outputs on a public dataset instead of weight matrices
• Critical for deployments with intermittent connectivity, such as autonomous vehicles or remote industrial sensors

The foundational aggregation algorithm that combines locally trained model updates from multiple clients into a single improved global model.

• Each client trains on local data and sends only model weight updates back to the server
• The server computes a weighted average of all received updates
• Weights are typically proportional to the size of each client's local dataset
• Communication rounds repeat until the global model converges

FedAvg dramatically reduces bandwidth compared to centralized training while preserving data locality.

A mathematical framework that injects calibrated noise into model updates to prevent reconstruction of individual training samples.

• The privacy budget (ε) quantifies the maximum information leakage allowed
• Gaussian or Laplacian noise is added to gradients before transmission
• A smaller ε value provides stronger privacy but may degrade model utility
• Local differential privacy applies noise on-device; global differential privacy applies it at the aggregation server

This technique defends against membership inference attacks and gradient leakage attacks.

A cryptographic multi-party computation technique ensuring the central server can only decrypt the sum of all client updates, never individual contributions.

• Clients encrypt their model updates using secret sharing or pairwise masking
• The server aggregates ciphertexts without ever seeing plaintext gradients
• Individual updates are only revealed if a threshold of clients colludes
• Protects against honest-but-curious servers and external eavesdroppers

Secure aggregation is critical when the model update itself could leak sensitive data patterns.

The realistic scenario where local datasets across clients are not independent and identically distributed, posing the central challenge in federated learning.

• Label distribution skew: some clients have only a subset of classes
• Feature distribution skew: the same label looks different across clients (e.g., handwriting styles)
• Quantity skew: clients have vastly different amounts of training data
• Non-IID data causes client drift, where local models diverge from the global optimum

Mitigation strategies include FedProx, SCAFFOLD, and personalized federated learning approaches.

The orchestration logic that determines which devices participate in each training round, balancing resource constraints with model convergence.

• Random subset sampling is the baseline approach for scalability
• Power-aware scheduling excludes devices with low battery or metered connections
• Staleness-aware selection prioritizes clients that haven't participated recently
• Heterogeneous hardware requires adaptive timeout and dropout handling

Effective scheduling minimizes straggler effects where slow devices bottleneck the entire round.

Two fundamental partitioning strategies based on how data is distributed across participants.

• Horizontal FL (HFL): clients share the same feature space but hold different sample populations—common in mobile keyboard prediction
• Vertical FL (VFL): clients hold different features for the same set of entities—common in banking and healthcare consortia
• VFL requires entity alignment via private set intersection before training
• HFL uses standard FedAvg; VFL requires split neural networks and encrypted intermediate activations

Choosing the correct topology determines the entire cryptographic and aggregation architecture.