Partial Participation

Edge devices are not perpetually online or available for computation. Participation is gated by availability windows—periods when a device is powered, idle, and connected to a network (e.g., a smartphone charging overnight). Intermittent connectivity due to mobility or poor signal strength further restricts which clients can respond to a server's training request within a synchronous round's timeout period.

The server actively selects a subset of clients each round, a process known as client sampling. This is done to:

• Manage System Scale: Coordinating millions of devices simultaneously is infeasible.
• Control Communication Cost: Limits upstream bandwidth per round.
• Improve Efficiency: Avoids waiting for slow stragglers.
• Enhance Statistical Diversity: Strategies like random uniform sampling or importance sampling can improve model convergence by ensuring diverse data contributions over time.

Clients have severe and varied resource constraints that dictate their ability to participate:

• Compute Constraint: Limited CPU/GPU power may make local training too slow for a given round deadline.
• Memory Footprint: Training requires RAM for model parameters, optimizer states, and activations, which may exceed available memory.
• Energy Budget: Battery drain from training and communication is a primary concern. Devices may opt-out to preserve battery life.
• Thermal Throttling: Sustained computation can cause overheating, forcing the device to reduce performance or shut down. This client heterogeneity means the eligible participant pool changes dynamically based on device state.

A device must have sufficient and relevant local data to perform a useful training step. Drivers include:

• On-Device Dataset Size: A device with no new data since the last round provides no learning signal.
• Cold-Start Problem: New devices or those in new environments lack sufficient data for effective training.
• Statistical Skew: Devices with highly non-IID data that is unrepresentative of the target global distribution may be deliberately under-sampled to reduce bias. Servers may implement data-aware sampling to prioritize clients with more informative local datasets.

Infrastructure-level decisions and policies enforced by the federated learning orchestrator directly control participation:

• Synchronization Deadlines: The server sets a hard deadline for receiving updates; clients that don't finish in time are excluded from that round's aggregation.
• Staggered Participation: To manage server load and network congestion, clients may be assigned to different training cohorts on a schedule.
• Compliance & Opt-Out: User privacy settings, enterprise IT policies, or regulatory requirements may prevent a device from participating in certain training rounds involving sensitive data or models.

The cost of transmitting model updates drives partial participation as a fundamental optimization.

• Bandwidth Limitations: Uploading multi-megabyte model updates from a cellular or satellite connection is expensive and slow.
• Sparse Updates: Techniques like federated dropout or sending only sparse updates (a small fraction of non-zero gradients) reduce communication payload, but also mean only a subset of model parameters are 'participating' from each client in a given round.
• Compression: Applying compression to updates can enable participation from bandwidth-constrained clients that would otherwise be excluded.

The algorithms and heuristics used by a central server to choose which subset of available clients will participate in a given federated learning round. Effective strategies must balance multiple objectives:

• Statistical Efficiency: Selecting clients whose data is most representative or informative for the global model.
• System Efficiency: Prioritizing clients with strong connectivity, high compute resources, and ample energy to avoid stragglers.
• Fairness & Coverage: Ensuring all clients eventually participate to prevent bias and improve model personalization. Common approaches include random uniform sampling, probability-based sampling weighted by client dataset size, and adaptive strategies that learn client capabilities over time.

The limited, often intermittent, period when an edge device is able to participate in federated training. This window is constrained by factors intrinsic to edge and TinyML deployments:

• Power State: Devices may be asleep, powered off, or in a low-power mode to conserve battery.
• Connectivity: Ephemeral network links (e.g., only when connected to Wi-Fi).
• Resource Contention: The device's primary function (e.g., sensing, actuation) takes priority, leaving only idle cycles for training.
• User Activity: On personal devices, training may only be permitted when the device is charging and idle. Algorithms for partial participation must be designed to work asynchronously or with flexible deadlines to accommodate these unpredictable windows.

The significant variation in hardware and data characteristics across the federated population of edge devices. This heterogeneity directly drives and complicates partial participation. Key Dimensions of Heterogeneity:

• Hardware: Vast differences in CPU/MPU power, memory (RAM/Flash), and battery capacity.
• Data Distribution: Non-IID (Non-Independent and Identically Distributed) data, where local datasets vary in size, label distribution, and feature correlation.
• Network: Bandwidth and latency can differ by orders of magnitude.
• Availability: Some devices may be available for training frequently, others rarely. This variability means a uniform client selection or training protocol is inefficient; systems must adapt to the capabilities of the participating subset in each round.

The systemic slowdown caused when the progress of a synchronous federated learning round is blocked by one or more slow or unresponsive clients. In partial participation with heterogeneous clients, stragglers are a major bottleneck. Causes in Edge Context:

• Weak Hardware: A client with a slow microcontroller will take much longer to complete its local epoch.
• Poor Connectivity: Uploading a model update over a low-bandwidth link can dominate the round time.
• Thermal Throttling: Sustained compute can cause a device to overheat and reduce its clock speed. Mitigations:
• Deadlines: The server proceeds with updates from clients that respond within a time window.
• Asynchronous Aggregation: The server updates the global model as soon as any client update arrives, decoupling progress from the slowest participant.

The canonical federated learning algorithm where partial participation is a core component. In each round:

1. The server selects a subset of clients (partial participation).
2. It sends the current global model to each selected client.
3. Each client performs local stochastic gradient descent on its own data.
4. Clients send their updated model weights back to the server.
5. The server computes a weighted average of these models to produce a new global model. The algorithm's convergence guarantees explicitly account for the fact that only a fraction of clients are sampled each round. Its performance is highly sensitive to the client selection strategy and the degree of client heterogeneity.

The process of updating a model's parameters directly on the edge device using local data. This is the core computational task a client performs during its participation in a federated round. For TinyML, this is exceptionally challenging due to:

• Memory Footprint: Training requires storing the model, optimizer state, activations, and a batch of data simultaneously in severely limited RAM.
• Compute Constraint: Microcontrollers have limited FLOPS, making backpropagation slow and energy-intensive.
• Energy Budget: A single training round can significantly impact device battery life. Techniques like sparse updates, quantization-aware training, and tiny training frameworks are essential to make on-device training feasible, thereby enabling effective partial participation on resource-constrained devices.