Capability-Based Pruning

The core mechanism involves generating a unique sparsity mask for each client. This binary mask, derived from the client's profiled capabilities, zeroes out a specific subset of the global model's weights. Crucially, the pruning criteria (e.g., magnitude-based, gradient-based) is applied globally, but the resulting sparsity level is calibrated per device. A high-end smartphone might retain 70% of weights, while a microcontroller retains only 20%, creating client-specific sub-networks from the same architecture.

Pruning decisions rely on a capability profile. This profile can be:

• Static: Based on hardware specifications (CPU cores, RAM) reported at registration. Simple but may not reflect runtime state.
• Dynamic: Incorporates real-time metrics from an on-device resource monitor, such as available memory, CPU load, battery level, and thermal status. Dynamic profiles enable more adaptive and efficient pruning but add monitoring overhead. The profile dictates the target model footprint (size in MB) and FLOPs budget for the local training task.

The pruning logic is typically executed on the federation server before distributing the model. This centralizes the complex pruning algorithms and prevents computational overhead on constrained clients. The server:

1. Receives or retrieves a client's capability profile.
2. Applies the pruning algorithm to the global model to meet the client's constraints.
3. Transmits the pruned model architecture and remaining weights to the client. This approach ensures consistency in the pruning methodology across the federation.

To maximize on-device efficiency, capability-based pruning often employs structured pruning methods. Instead of removing individual weights (unstructured pruning), it removes entire filters, channels, or neurons. This results in genuinely smaller matrices that leverage optimized linear algebra libraries and reduce memory bandwidth usage. The chosen structure (e.g., filter pruning for CNNs) aligns with common hardware acceleration patterns on edge NPUs and GPUs.

A fundamental challenge is aggregating weight updates from differently pruned models. Standard Federated Averaging (FedAvg) cannot directly average sparse gradients from non-overlapping parameter sets. Solutions include:

• Mask-Aware Aggregation: The server maintains the sparsity mask for each client and only aggregates updates for the weights that were active for that client.
• Gradient Masking: Clients only compute and send gradients for their active weights, reducing communication cost. The server reconstructs a sparse update tensor using the known mask before aggregation.

Capability-based pruning is rarely used in isolation. It is part of a broader heterogeneity management stack:

• Dynamic Batching: A pruned model may support a larger local batch size within the same memory budget.
• Resource-Aware Scheduling: Clients with heavily pruned models may be scheduled more frequently, as their training rounds are shorter.
• Federated Dropout: Can be used in tandem, where pruning is capability-driven and dropout is random per round.
• Tiered Aggregation: Clients with similar sparsity profiles can be grouped for intermediate aggregation steps.

The foundational process of systematically measuring and cataloging the computational resources, memory, network connectivity, and power availability of edge devices enrolled in a federated learning system. This profile is stored in a Federated Device Registry and is essential for all subsequent heterogeneity management decisions.

• Key Metrics: CPU/GPU type, available RAM, battery level, network bandwidth/latency, thermal status.
• Use Case: Informs Resource-Aware Scheduling and Capability-Based Pruning by providing the data needed to match training tasks to suitable devices.

A federated learning orchestration strategy that dynamically assigns training tasks to clients based on their real-time available compute, memory, and energy constraints. It uses data from Client Capability Profiling to maximize system efficiency and completion rates.

• Mechanism: The server evaluates device profiles and current load to select clients likely to complete a Variable-Length Training Round within a target latency window.
• Objective: Prevents stragglers, reduces round failure rates, and is a core component of Elastic Federated Learning systems.

A variant of the standard Federated Averaging algorithm designed to aggregate model updates from clients with vastly different computational capabilities. It addresses the bias that can arise when devices perform different amounts of local work.

• Common Techniques: Weighting client updates by the number of local steps completed or the amount of data processed.
• Relation: Often works in tandem with Variable-Length Training Rounds and Tiered Aggregation to handle hardware heterogeneity effectively.

A method that splits a neural network model into segments, offloading computationally intensive layers to a server or nearby edge node while keeping simpler layers on a resource-constrained device. This enables federated training of models that would otherwise exceed a device's memory or compute limits.

• Process: An On-Device Resource Monitor can trigger repartitioning if local resources become constrained.
• Contrast with Pruning: Unlike Capability-Based Pruning, which creates a smaller full-model, partitioning executes a subset of layers locally.

A technique where a random subset of neurons or entire layers are deactivated for individual clients during a training round. This creates smaller, unique sub-models for each client, reducing their computational and memory load.

• Benefits: Lowers per-device resource requirements, can act as a regularizer to improve model generalization, and is a form of Partial Model Participation.
• System Impact: The server must intelligently aggregate these heterogeneous sub-model updates, often relating to strategies in Heterogeneous Federated Averaging.

A communication protocol where the server aggregates client model updates as soon as they are received, without waiting for a synchronized round to finish. This is critical for systems with high heterogeneity to accommodate clients with highly variable training times.

• Advantage: Eliminates the straggler effect, as slow devices do not block the aggregation of updates from faster ones.
• Challenge: Requires careful handling of stale updates from very slow clients, often managed by Adaptive Federated Optimization (FedOpt) algorithms like FedAsync.