Cross-Device Federated Learning

Orchestrates training across millions to billions of heterogeneous edge devices—smartphones, IoT sensors, wearables—rather than a handful of institutional servers. This scale introduces unique challenges in client selection, where only a statistically representative subset of available devices participates in each training round. The coordinator must handle ephemeral availability, as devices join and leave based on user behavior, charging status, and network conditions. Unlike cross-silo settings with reliable, always-on participants, cross-device systems assume the majority of clients are unavailable at any given moment.

Local datasets on each device reflect individual user behavior, creating extreme statistical heterogeneity. Data is not independently and identically distributed; a single user's photo library or typing patterns form a highly personalized, skewed distribution. This causes client drift, where local model updates optimize for idiosyncratic local minima rather than the global objective. Mitigation strategies include FedProx (proximal regularization), SCAFFOLD (variance reduction via control variates), and personalized federated learning that explicitly models user-specific deviations from the global model.

The uplink from device to server is the primary constraint, with bandwidth often limited to kilobits per second on cellular or LPWAN connections. This mandates aggressive gradient compression techniques:

• Sparsification: transmitting only the top-k gradient elements by magnitude
• Quantization: reducing weight updates from 32-bit floats to 2-8 bit integers
• Federated distillation: exchanging compact model outputs (logits) on a public dataset instead of full weight matrices Over-the-Air Computation (AirComp) exploits waveform superposition in wireless multiple-access channels to compute sums directly over the air, dramatically reducing latency.

Synchronous rounds are gated by the slowest participating device. Stragglers—devices with weak compute, poor connectivity, or interrupted training—can delay the entire federation by orders of magnitude. Solutions include:

• Asynchronous federated learning: the server updates the global model immediately upon receiving any single client's update, eliminating idle waiting
• Timeout-based client dropping: discarding updates from clients that exceed a deadline
• Coded computation: injecting redundant computation so the server can reconstruct the aggregate from the fastest subset of responses Each approach trades off convergence guarantees for wall-clock efficiency.

Raw model updates can leak sensitive information through model inversion attacks and gradient leakage. Cross-device systems deploy layered defenses:

• Secure Aggregation: a multi-party computation protocol where the server learns only the sum of encrypted updates, never individual contributions
• Differential Privacy: calibrated Gaussian noise is added to clipped updates, providing a provable (ε, δ)-privacy guarantee that bounds the influence of any single user
• Trusted Execution Environments (TEEs) on device hardware ensure local training occurs in an isolated enclave, protecting the model and data even from the device owner These techniques are often combined to balance utility against privacy budgets.

The underlying data distribution across the client population evolves over time—new slang enters keyboard dictionaries, seasonal photos change visual features, sensor calibrations drift. This temporal non-stationarity requires continuous adaptation. Detection mechanisms monitor population-level loss and gradient norm divergence to trigger model retraining. Federated continual learning techniques, including elastic weight consolidation and experience replay buffers on the server side, prevent catastrophic forgetting of previously learned patterns while accommodating new distributions.

The defining challenge of cross-device FL where local datasets are statistically heterogeneous—each device's data reflects its user's unique behavior, not the global population. This causes client drift where local models diverge from the global optimum.

• Label skew: Some clients have only a subset of classes
• Feature skew: Same labels appear with different feature distributions across devices
• Quantity skew: Highly variable dataset sizes, from dozens to thousands of samples

FedProx and SCAFFOLD are algorithms specifically designed to correct for this heterogeneity.

The scheduling mechanism that determines which subset of available devices participate in each training round. In cross-device FL with millions of clients, only a small fraction (typically 0.1-1%) are selected per round.

• Availability-based: Select devices that are charging, on Wi-Fi, and idle
• Data-quality-based: Prioritize devices with diverse or high-quality local data
• Multi-armed bandit: Learn which clients contribute most to model improvement

Poor selection strategies exacerbate non-IID issues and slow convergence.

Techniques to handle slow or unresponsive devices that delay synchronous training rounds. In cross-device settings, stragglers are inevitable due to heterogeneous hardware, network variability, and user interruptions.

• Timeout-based: Drop updates from devices exceeding a deadline
• Asynchronous aggregation: Update global model immediately upon receiving any update
• Coded computation: Use error-correcting codes to reconstruct missing updates

Without mitigation, a single slow device can stall an entire round involving thousands of participants.

The foundational cross-device FL algorithm where each selected client performs multiple local SGD steps on its own data before sending only the updated model weights to the server. The server aggregates these via a weighted average.

• Communication efficiency: Multiple local epochs reduce rounds by 10-100x vs. FedSGD
• Partial participation: Only a random subset of clients train each round
• Weighted aggregation: Updates weighted by local dataset size

FedAvg remains the baseline against which all cross-device algorithms are measured, though it struggles with severe non-IID distributions.

Communication efficiency techniques that reduce the size of model updates transmitted from resource-constrained devices. In cross-device FL, upload bandwidth is often the primary bottleneck.

• Sparsification: Transmit only the top-k gradient elements by magnitude
• Quantization: Reduce gradient precision from 32-bit float to 8-bit or even 1-bit
• Error feedback: Track compression residuals to prevent accuracy degradation

Combined with FedAvg, these techniques can reduce communication by 100-1000x with minimal accuracy loss.

A cryptographic protocol ensuring the central server can only compute the sum of encrypted model updates without inspecting any individual contribution. Critical for cross-device FL where devices contain deeply personal data.

• Secret sharing: Each client splits its update into shares distributed to other clients
• Pairwise masking: Clients agree on random masks that cancel out during summation
• Dropout robustness: Protocol tolerates clients dropping out mid-round

Google's production cross-device FL system uses secure aggregation to protect Gboard typing data from millions of users.