Energy-Proportional Computing

The core goal is for power draw to increase linearly from a near-zero baseline as computational load increases. In an ideal system, a 50% load consumes 50% of peak power. This contrasts with traditional systems where idle power can be 50-70% of peak. For TinyML, this means designing inference engines and microcontroller cores whose active power is directly proportional to the number of operations (MACs) being performed.

Achieving true proportionality requires minimizing static power (leakage) when the system is idle. This is critical for battery-powered IoT devices that spend most of their life in a sleep state. Techniques include:

• Power gating to shut off supply voltage to idle blocks.
• Using high-threshold voltage (HVT) transistors in non-critical paths.
• Designing Always-On (AON) domains that are极小化 and use specialized low-leakage cells to monitor sensors with microwatts of power.

Instead of managing power for the entire chip, the system is partitioned into many independent power domains. This allows unused subsystems—like a specific neural processing unit (NPU) core, a sensor interface, or a radio—to be completely powered down (power gated) while others remain active. Modern microcontroller units (MCUs) for TinyML feature dozens of these domains, enabling precise power control matched to the minimal required functionality for a given task.

The system must dynamically match its performance capability to the instantaneous workload to avoid wasting energy on excess speed. This is implemented via:

• Dynamic Voltage and Frequency Scaling (DVFS): Adjusting core voltage and clock frequency in tandem.
• Adaptive Body Biasing: Modifying transistor threshold voltage for optimal efficiency at a given performance point.
• For ML accelerators, this includes varying compute array utilization and precision modes (e.g., switching from 8-bit to 4-bit inference) based on the throughput requirement.

Energy proportionality requires the software stack to intelligently manage hardware resources. This involves:

• Power-aware scheduling: Batching sensor readings or inference tasks to maximize the time components spend in deep sleep.
• Duty cycling: Periodically waking only the necessary power domains for the shortest time required.
• Predictive wake-up: Using simple always-on circuitry to predict when the main processor will be needed, avoiding periodic polling. In TinyML, this might involve a tiny binary classifier in the AON domain to decide if the main ML model needs to run.

Energy consumption must scale across the entire data path, not just the processor. Key considerations include:

• Memory hierarchy access costs: Accessing on-chip SRAM is far more energy-proportional than accessing external DRAM. TinyML models must fit in SRAM.
• I/O power scaling: The energy for sensor sampling or radio transmission should scale with data rate. Techniques like adaptive sampling rates for sensors and low-power radio modes (e.g., Bluetooth Low Energy advertising intervals) are essential.
• In-Memory Computing (IMC): Emerging architectures that perform computations within the memory array itself, drastically reducing energy spent on data movement, which is a major bottleneck.

A hardware-level power management technique that dynamically adjusts a processor's operating voltage and clock frequency in response to real-time computational demand. This reduces dynamic power, which scales with the square of the voltage and linearly with frequency. In TinyML, DVFS allows a microcontroller to run at high performance for complex inference bursts and then scale down dramatically during idle periods, forming a core mechanism for energy proportionality.

• Key Mechanism: Closed-loop control based on CPU utilization or task queue depth.
• TinyML Impact: Enables fine-grained matching of compute capability to the instantaneous demands of a neural network inference pipeline.

A circuit design technique that uses header or footer switches (MOSFETs) to completely cut off the power supply (VDD or GND) to an idle logic block, core, or SRAM bank. This eliminates both dynamic power and static leakage power, which becomes dominant in advanced semiconductor nodes. For energy-proportional systems, power gating is applied to entire subsystems (e.g., a floating-point unit, a secondary accelerator) when not in use, achieving near-zero power draw for those components.

• Trade-off: Introduces a state retention and restoration overhead (latency, energy).
• Use Case: Shutting down a convolutional accelerator in a vision system between frame captures.

A fundamental power-saving technique that disables the clock signal to specific registers or logic modules when they are not performing useful computation. This prevents unnecessary transistor toggling, eliminating the dynamic power dissipation associated with charging and discharging capacitive loads. It is a finer-grained, lower-overhead complement to power gating. In microcontroller-based ML systems, clock gating is applied pervasively at the register-transfer level (RTL) to freeze inactive portions of a neural processing unit (NPU) or digital signal processor (DSP).

• Implementation: Typically automated by synthesis tools using enable-signal conditions.
• Effect: Reduces dynamic power to zero for the gated block without losing state.

A system architecture pattern critical for energy-proportional sensing. The main CPU and ML accelerator reside in a power-gated domain. A tiny, ultra-low-power Always-On (AON) domain remains active, containing simple logic, a real-time clock, and interfaces to critical sensors. This AON domain monitors for a predefined wake-up event (e.g., accelerometer threshold, audio keyword detection, timer expiry). Only when a relevant event occurs does it power up the main domain for full inference. This enables near-zero idle power while maintaining responsiveness.

• Components: AON domain typically uses deeply sub-threshold logic for minimal leakage.
• TinyML Example: A vibration-based predictive maintenance sensor sleeps (micro-watts) until an AON comparator detects anomalous frequency, triggering full spectrogram analysis.

The most basic scheduling technique for energy-constrained devices, defining a periodic cycle of active time (Ton) and sleep time (Toff). The duty cycle percentage is (Ton / (Ton + Toff)). Energy consumption scales linearly with this duty cycle. For true energy proportionality, the duty cycle (or the frequency of cycles) is adapted dynamically based on the required sampling rate or event frequency. This is applied to radios, sensors, and the entire system.

• Metric: Average power = Duty Cycle * Active Power + (1 - Duty Cycle) * Sleep Power.
• Adaptive Use: An environmental sensor increases its duty cycle from 0.1% to 1% during a detected storm for higher-fidelity monitoring.

Key metrics for evaluating the energy-performance trade-off in TinyML systems. Energy-Delay Product (EDP = Energy * Delay) penalizes designs that save energy by drastically slowing performance. Inference-Per-Watt (Inf/J) measures the throughput efficiency of an ML accelerator. An energy-proportional system aims for a flat Inf/J curve across a wide range of utilization levels, avoiding high fixed overheads. These metrics guide architectural choices, such as selecting between a high-performance core with fast sleep transitions versus a slow, ultra-efficient core.

• Benchmarking: Used to compare the Pareto efficiency of different MCUs, NPUs, and model architectures.
• Design Goal: Maximize Inference-Per-Watt at both peak and low utilization points.