Cross-Device Federated Learning

Cross-device FL operates across millions to billions of heterogeneous edge devices, dwarfing the scale of cross-silo deployments. The server selects only a small, random fraction of available clients for each training round due to bandwidth constraints.

• Typical round participation: 50–5,000 clients sampled from a population of millions
• Each client contributes to only a tiny fraction of total rounds over its lifetime
• The server never maintains persistent state for individual clients
• Example: Google's Gboard keyboard trains across 1B+ Android devices

Edge devices participate only when meeting strict eligibility criteria: connected to unmetered Wi-Fi, plugged into power, and idle. This ensures training does not degrade user experience or consume cellular data.

• Clients are unreachable by the server; they must poll to check in
• Availability windows are unpredictable and typically short (minutes)
• A significant percentage of selected clients drop out mid-computation
• The system must tolerate 90%+ client dropout without stalling convergence

Each device holds data reflecting a single user's behavior, creating extreme statistical heterogeneity. Unlike cross-silo settings where institutions hold broad population samples, cross-device data is pathologically non-IID.

• Label distribution skew: One user may type only technical jargon; another only casual slang
• Feature distribution skew: Different sensor calibrations, screen sizes, usage patterns
• Quantity skew: Some users generate orders of magnitude more data than others
• This non-IIDness is the primary cause of client drift and convergence instability

Training must execute within the tight memory, battery, and thermal envelopes of consumer hardware. Models are compressed and training algorithms adapted for resource-constrained execution.

• Typical on-device model size: < 10 MB after quantization
• Training runs as a low-priority background process to avoid UI jank
• Local epochs are few (often just 1–5) due to time and power limits
• Techniques like federated distillation and partial model training reduce client-side compute burden
• Example: Federated training of a next-word prediction model with a ~2MB weight footprint

Uploading model updates over cellular or constrained Wi-Fi dominates the end-to-end wall-clock time and user data budgets. Cross-device FL employs aggressive compression and reduced communication frequency.

• Gradient compression: Quantization to 8-bit or fewer, sparsification, random masking
• Federated averaging with many local SGD steps reduces round frequency
• Secure aggregation protocols add communication overhead that must be minimized
• Typical per-client upload size target: < 100 KB per round
• Techniques like gradient sketching and one-shot federated learning push this further

Cross-device FL processes raw personal data on-device, but model updates can still leak sensitive information through gradient leakage or membership inference. Defense-in-depth privacy protections are mandatory.

• Differential privacy (DP) is applied at the server via a trusted aggregator, clipping and noising updates
• Secure aggregation ensures the server sees only the summed update, never individual contributions
• On-device DP adds noise locally before upload for stronger guarantees
• The DP budget (ε) is carefully tracked across rounds to bound total privacy loss
• Example: Apple's federated learning uses a per-round ε of ~4–8 for emoji suggestion models

The antithesis of cross-device FL. This topology involves a small, static set of reliable institutional clients (e.g., 2-50 hospitals or banks) that hold large, curated datasets. Unlike the massive scale of smartphones, cross-silo clients are stateful, participate in almost every training round, and have high computational power. The primary bottleneck is network bandwidth, not device dropout. This is the dominant paradigm for healthcare biomarker discovery and financial fraud detection.

The defining statistical challenge of cross-device FL. Local client datasets are heterogeneous and unbalanced, meaning a user's personal data does not represent the global population. In biomarker contexts, this manifests as different patient demographics per hospital. Non-IID data causes client drift, where local models diverge from the global optimum, slowing convergence. FedProx and SCAFFOLD are algorithms specifically designed to correct for this statistical heterogeneity.

The foundational algorithm powering most cross-device deployments. The process is iterative:

• A central server initializes a global model.
• A random subset of available edge devices downloads the model.
• Each device trains locally on its private data.
• Only encrypted model weight updates are sent back to the server.
• The server aggregates these updates via a weighted average to produce the next global model. This communication-efficient loop is the backbone of Gboard and Apple's QuickType keyboard.

A cryptographic protocol essential for cross-device privacy. It ensures the central server can only compute the sum of model updates from a cohort of devices, but cannot inspect any individual user's contribution in plaintext. The protocol uses pairwise masking between devices and secret sharing. If a device drops out mid-round (a common cross-device failure), the protocol recovers the aggregate without exposing the dropout's data, maintaining the integrity of the global model update.

A paradigm shift from the 'one-size-fits-all' global model. In cross-device settings, a single global model often performs poorly for outliers. pFL strategies include:

• Local fine-tuning: Start from the global model, then adapt to the user's device.
• Model interpolation: Blend a global model with a purely local model.
• Multi-task learning: Treat each device as a separate task. For biomarker systems, this allows a base diagnostic model to adapt to the specific imaging equipment or patient demographics of a remote clinic.

The primary security vulnerability in naive federated learning. A malicious server can analytically reconstruct private training data from raw model gradients. In deep gradient leakage, the server optimizes random noise to match the shared gradients, revealing the original input. Cross-device FL mitigates this through secure aggregation and differential privacy, which clips and noises gradients before transmission. This is non-negotiable for clinical biomarker data subject to HIPAA.