Resource Monitor

The Resource Monitor continuously gathers metrics from all participating clients and server nodes. This forms the foundational dataset for all orchestration logic.

Key metrics include:

• Compute Utilization: CPU/GPU usage and thermal throttling status.
• Memory Pressure: RAM and swap usage, critical for determining if a model can be loaded.
• Network Conditions: Bandwidth, latency, and packet loss for the connection to the aggregator.
• Power State: Battery level and charging status on mobile/IoT devices.
• Storage I/O: Read/write speeds and available disk space for model checkpoints.

The monitor's data is the primary input for the Client Selection Module. Selecting devices with sufficient resources is essential for round completion and model quality.

Selection criteria powered by telemetry:

• Prioritizing devices with stable, high-bandwidth connections to minimize communication latency.
• Filtering out devices with critically low battery to avoid mid-round dropouts.
• Choosing clients with sufficient free memory to load the current global model.
• Implementing availability-aware policies that consider historical uptime patterns.

The monitor enables the Task Scheduler to assign work intelligently, preventing system overload and maximizing throughput.

Scheduling decisions include:

• Adaptive Batch Sizing: Assigning smaller local batch sizes to devices with constrained memory.
• Staggered Start Times: For cross-silo settings, initiating training on well-resourced nodes first.
• Workload Offloading: Directing compute-intensive validation tasks to the most powerful available servers.
• Priority Queue Management: Ensuring high-value clients (e.g., with rare data) are processed when they come online.

By establishing baselines and monitoring trends, the Resource Monitor acts as an early warning system for the Fault Tolerance Manager.

It detects:

• Client Dropout Signals: A sudden loss of network connectivity or a device powering off.
• Performance Degradation: A client's training speed slowing due to thermal throttling.
• Resource Exhaustion: Memory leaks in the local training process.
• Anomalous Behavior: Unexpectedly high CPU usage, which could indicate a malicious process or software bug.

Resource data is correlated with training metrics by the Convergence Monitor to diagnose issues and optimize the federated job.

Analysis enables:

• Identifying if slow convergence is due to systemic resource constraints (e.g., poor network) versus a poor model architecture.
• Tuning the frequency of communication rounds based on aggregate client readiness.
• Implementing resource-aware aggregation strategies, such as weighting updates from more stable clients slightly higher.
• Providing data for cost-benefit analysis of the federation (e.g., energy consumed per unit of accuracy gain).

The Resource Monitor is not a siloed component; it provides APIs and event streams that feed data into the broader orchestrator.

Key integrations:

• Heterogeneity Handler: Uses device profiles to customize client-side training code.
• Secure Aggregation Orchestrator: May delay cryptographic protocols if network telemetry indicates congestion.
• Audit Logger: Records resource snapshots alongside training events for post-hoc analysis of failures.
• Compliance Checker: Can enforce policies like "do not train on devices below 20% battery."

Example Tools: Prometheus for metric collection, with custom exporters on clients; OpenTelemetry for standardized telemetry data.

Monitors CPU and GPU usage percentages and thermal states across clients and the central server. This is critical for:

• Load Balancing: Preventing stragglers by avoiding overloaded devices.
• Task Scheduling: Assigning training tasks to devices with sufficient available compute.
• Energy Efficiency: Identifying devices with high thermal throttling that may be unsuitable for intensive rounds. Example: A client reporting 95% CPU utilization for 60 seconds would be flagged as a potential straggler and may be excluded from the next round.

Tracks RAM usage, swap activity, and available storage on edge devices. Key functions include:

• Model Feasibility Check: Ensuring a device has enough memory to load the current global model and perform training.
• Out-of-Memory Prevention: Proactively dropping clients that are nearing memory limits to avoid training crashes.
• Checkpoint Management: Monitoring storage space for saving model checkpoints and training logs locally. For TinyML deployments, this includes tracking SRAM and flash memory usage on microcontrollers.

Collects data on bandwidth, latency, packet loss, and data transfer costs. This informs:

• Communication-Efficient Scheduling: Selecting clients with high-bandwidth, low-cost connections for large model updates.
• Adaptive Compression: Dynamically adjusting the compression level of model updates based on current bandwidth.
• Fault Prediction: Identifying clients with unstable connections likely to drop out mid-round. In mobile networks, this also monitors cellular data quotas to respect user data plans.

Observes battery level, charging status, and power draw for mobile and IoT clients. This enables:

• Participant Selection: Prioritizing devices plugged into power for compute-intensive rounds.
• User Experience Preservation: Avoiding training on devices with critically low battery.
• Energy-Aware Algorithms: Informing algorithms like FedProx that add a proximal term to the loss, which can reduce the number of local epochs needed for convergence, saving power.

Pings for client liveness, tracks uptime, and monitors system process health. This supports:

• Fault Tolerance: Quickly identifying dropped clients to trigger retries or adjust aggregation weights.
• Resource Profiling: Building a historical profile of each client's reliability for smarter future selection.
• Security Posture: Detecting anomalous system states that could indicate a compromised device. Metrics are often aggregated into a client health score used by the Client Selection Module.

While respecting privacy, monitors local dataset size, label distribution (via secure statistics), and local training progress. This helps manage:

• Statistical Heterogeneity (Non-IID): Detecting clients with highly skewed data distributions that may harm global model convergence.
• Personalized Federated Learning: Identifying clients that would benefit from local personalization layers.
• Convergence Monitoring: Correlating slow client training progress with underlying resource constraints reported by other monitors.

The component that determines the order, timing, and resource allocation for federated learning tasks. It uses data from the Resource Monitor to make intelligent decisions.

• Key Functions: Implements policies for fairness, efficiency, and job priority.
• Integration Point: Consults real-time resource telemetry to avoid overloading clients with low battery or high CPU usage.
• Example: A scheduler might delay a training task for a mobile device until it is connected to Wi-Fi and charging, based on monitor data.

The algorithmic component that chooses which devices participate in a training round. Resource Monitor data is a primary input for its selection criteria.

• Selection Criteria: Includes device resource availability (CPU, memory), network bandwidth, battery level, and historical reliability.
• Objective: To select a cohort of clients that can complete the training task efficiently without dropping out.
• Impact: Poor selection, ignoring resource constraints, leads to stragglers and failed rounds.

A system module designed to manage variability across clients. The Resource Monitor provides the raw data on this heterogeneity.

• Managed Variability: Compute capability (CPU/GPU), memory, network connectivity, power profile, and data distribution.
• System Role: Uses monitor data to adapt training tasks (e.g., batch size, local epochs) to each client's capabilities.
• Goal: To ensure stable and efficient training despite a diverse device fleet.

A component that ensures job completion despite partial failures. It relies on the Resource Monitor for early fault detection.

• Strategies: Implements checkpointing, client dropout handling, and task retries.
• Detection: Uses monitor alerts on client disconnects, crashing processes, or sustained 100% CPU (potential hang) to trigger recovery actions.
• Example: If a monitor reports a client's process has terminated, the fault manager can reassign its task to another device.

Tracks global model performance across rounds to determine when training should stop. While focused on accuracy, it may use resource signals for context.

• Primary Metrics: Model accuracy/loss, update magnitudes, client contribution variance.
• Resource Context: Correlates periods of poor convergence with systemic resource issues (e.g., widespread network latency reported by the Resource Monitor).
• Decision Point: Uses metrics to recommend stopping, continuing, or adjusting hyperparameters.

Coordinates the deployment and execution of trained models on devices. It consults the Resource Monitor before deploying inference workloads.

• Deployment Check: Verifies a target device has sufficient free memory and compute headroom to load and run the model.
• Runtime Management: Can throttle inference frequency based on real-time battery or thermal data from the monitor.
• Lifecycle: Manages model versioning, A/B testing, and rollback on the edge.