Thermal Throttling

The primary mechanism for thermal throttling. The processor's operating voltage and clock frequency are dynamically reduced. Lowering the frequency decreases switching activity, while lowering the voltage reduces dynamic power consumption (P = CV²f). This creates a cubic reduction in power dissipation and heat generation, allowing the silicon to cool. In TinyML, this can stall or drastically slow local SGD iterations.

Throttling is triggered by on-die thermal sensors exceeding predefined critical temperatures.

• Tjmax (Junction Temperature): The maximum allowable temperature for the silicon die itself (e.g., 100°C). Exceeding this risks immediate transistor degradation.
• Tskin (Skin/Case Temperature): A lower, user-safety limit for the device's exterior. Throttling may engage earlier to keep the device surface cool to the touch, especially relevant for wearables and phones in Federated Learning scenarios.

For multi-core processors (common in higher-end edge SoCs), the operating system or firmware can park (disable) cores or migrate computational loads away from the hottest cores. This spreads thermal load or reduces total active silicon area. In a federated learning context, this reduces the available parallel compute for matrix operations during local training, increasing the time-per-round.

Thermal throttling presents a fundamental constraint for Federated Learning on TinyML devices:

• Unpredictable Round Duration: A training round's completion time becomes variable and data-dependent, complicating synchronous aggregation.
• Reduced Effective FLOPs: The device's peak theoretical compute is never sustainable, requiring algorithms designed for throttle-aware training.
• Energy Inefficiency: More energy is spent per computation as runtime extends, depleting the energy budget faster for a fixed task.

System and algorithm design can mitigate throttling's impact:

• Workload Chunking: Breaking local epochs into smaller, cooler-running bursts separated by idle cooldown periods.
• Adaptive Batch Sizes: Dynamically reducing the mini-batch size during a training round as temperature rises to limit heat generation per step.
• Model & Optimizer Choice: Using sparse updates and lightweight optimizers that generate less heat per parameter update than dense, full-batch SGD.
• Compiler Optimizations: Using low-precision arithmetic (e.g., 8-bit) via QAT/PTQ reduces data movement and compute intensity, lowering power draw.

A key specification misunderstanding. TDP is the maximum heat a cooling system is required to dissipate for the processor to run at base frequency under a complex workload. The sustained power limit for on-device training is often far lower, especially in passively cooled edge devices. Federated learning orchestrators must model thermal mass and cooling profiles to predict a device's sustainable compute window before throttling engages.

A compute constraint is the limitation imposed by the available processing power (e.g., CPU/clock speed) of a resource-constrained device. It directly restricts the complexity and speed of on-device training algorithms.

• Primary Metric: Often measured in operations per second (OPS) or clock frequency (MHz).
• Interaction with Throttling: Thermal throttling dynamically reduces the effective compute constraint by lowering clock speed to manage heat, creating a variable performance ceiling.
• Design Impact: Forces the use of simplified model architectures (e.g., for TinyML) and efficient training loops to fit within available cycles.

The energy budget is the total amount of electrical energy allocated for a computational task, such as a federated learning round, which is a fundamental constraint for battery-powered devices.

• Direct Relationship to Heat: Power consumption (Watts) is converted into heat (Joules). Exceeding the energy budget for intensive compute tasks is a primary trigger for thermal throttling.
• System-Wide Trade-off: Engineers must balance the energy cost of computation (training), wireless communication (sending updates), and sensor operation.
• Key Objective: Maximize learning progress per joule of energy consumed to extend device operational lifetime.

On-device training is the process of updating a machine learning model's parameters directly on an edge device using locally generated data, enabling continual learning without raw data egress.

• Thermal Culprit: This is typically the most computationally intensive and heat-generating operation on a TinyML device, far surpassing inference.
• Throttling Impact: Unchecked, training can quickly hit thermal limits, causing thermal throttling that drastically slows convergence or halts progress.
• Mitigation Strategies: Requires techniques like sparse updates, low-precision arithmetic, and adaptive batch sizes to manage thermal load.

Heterogeneous clients in federated learning refer to the significant variation in hardware capabilities (compute, memory, cooling), data distributions, and availability among participating edge devices.

• Thermal Diversity: Devices in different physical environments (e.g., indoor vs. outdoor, sealed enclosures) will have vastly different thermal profiles and throttling behaviors.
• Algorithmic Challenge: This heterogeneity complicates synchronous federated aggregation, as thermally-throttled straggler devices may complete local training rounds orders of magnitude slower.
• System Design Implication: Federated orchestrators must account for dynamic client performance, potentially using partial participation and asynchronous protocols.

Low-precision arithmetic involves performing neural network computations using numerical formats with fewer bits (e.g., 8-bit integers, 16-bit float) than standard 32-bit float.

• Thermal & Energy Benefits: Reduces memory bandwidth, power consumption, and transistor switching activity, which directly lowers heat generation.
• Core TinyML Technique: Enables on-device training and inference on microcontroller units (MCUs). Methods include Quantization-Aware Training (QAT) and Post-Training Quantization (PTQ).
• Trade-off: May introduce numerical error, requiring careful calibration to maintain model accuracy while achieving thermal gains.

Model sparsification is the process of inducing sparsity in a neural network by zeroing out a significant fraction of its parameters (weights) or gradients.

• Efficiency Driver: Creates sparse models that can be stored and computed more efficiently, as operations involving zeros can be skipped.
• Direct Thermal Reduction: Fewer active computations during on-device training translate to lower power draw and reduced heat output, mitigating thermal throttling.
• Federated Advantage: Enables sparse updates, where clients communicate only changed weights, saving both communication energy (less radio-on time) and compute energy.