Power Management Unit (PMU)

The PMU provides stable, clean supply voltages to various system components (CPU, NPU, memory, I/O) from a primary battery or input source. It uses switching regulators (for high efficiency) and low-dropout linear regulators (LDOs) (for low-noise, precise voltages) to convert input power to the required levels. This ensures reliable operation despite fluctuations in input voltage or load current.

A critical function is controlling the precise order and timing in which power rails are turned on and off. Proper sequencing prevents latch-up (a damaging high-current state) and ensures stable startup. For an SoC, this might involve:

• Powering analog blocks before digital cores.
• Enabling I/O voltages before core logic.
• Ramping voltages at controlled slew rates. Incorrect sequencing can cause boot failures or permanent hardware damage.

The PMU works in concert with the operating system and hardware monitors to implement advanced power-saving techniques in real-time. Key mechanisms include:

• Dynamic Voltage and Frequency Scaling (DVFS): Adjusting processor voltage and clock frequency based on workload.
• Power Gating: Completely shutting off power to idle circuit blocks to eliminate static (leakage) power.
• Clock Gating: Disabling the clock to inactive modules to reduce dynamic power. This active management is essential for maximizing performance-per-watt.

The PMU defines and controls hierarchical sleep states (e.g., C-states for CPUs, D-states for devices). Each state trades off power savings against wake-up latency. Deeper states (e.g., suspend-to-RAM) power down more components but take longer to resume. The PMU manages the transitions between these states, often triggered by timers, interrupts, or wake-on-inference events from a low-power coprocessor.

For battery-powered devices, the PMU incorporates a Battery Management System (BMS). This includes:

• Fuel Gauging: Accurately estimating state-of-charge (SoC) and remaining capacity.
• Charging Control: Managing safe, efficient charging cycles (constant current/constant voltage).
• Protection: Guarding against over-voltage, over-current, and over-temperature conditions. This enables battery-aware scheduling of compute tasks to extend device operational life.

The PMU continuously monitors system health to ensure safe operation. It integrates:

• Current and Voltage Monitors: For real-time power profiling.
• Temperature Sensors: To trigger thermal throttling if critical thresholds are exceeded.
• Fault Detectors: For conditions like undervoltage lockout (UVLO) or overcurrent. Upon detecting a fault, the PMU can initiate a controlled shutdown or reset, protecting the hardware from damage.

Dynamic Voltage and Frequency Scaling (DVFS) is a runtime power management technique that adjusts a processor's operating voltage and clock frequency in response to real-time computational demand. By lowering voltage and frequency during periods of low activity, DVFS provides a cubic reduction in dynamic power consumption, as power is proportional to the square of voltage and linearly to frequency. It is a primary method for balancing performance-per-watt in AI accelerators and mobile SoCs.

Power gating is a circuit-level technique that completely shuts off the power supply to inactive or idle functional blocks on a silicon chip. This is achieved using header or footer switches (high-Vt transistors) that isolate the block's power rail. Unlike clock gating, power gating eliminates both dynamic power and static power (leakage power), making it essential for managing leakage in advanced process nodes. It is commonly used to power down entire AI accelerator cores between inference batches.

Performance-per-watt is the fundamental efficiency metric for computing systems, defined as the amount of useful computational work delivered per unit of electrical power consumed. For AI systems, it is often measured in inferences per second per watt (inf/sec/W) or tera-operations per second per watt (TOPS/W). This metric drives hardware selection and model optimization, as it directly correlates to battery life in mobile devices and operational cost in data centers. It encapsulates the trade-offs managed by the PMU and compression techniques.

Thermal throttling is a protective, feedback-driven mechanism where a processor's PMU or thermal management unit (TMU) automatically reduces its operating frequency and voltage when the die temperature approaches a critical junction temperature (Tjmax). This prevents physical damage from overheating but causes a drop in inference throughput. For sustained AI workloads, effective thermal design and power profiling are required to avoid performance cliffs caused by throttling.

Wake-on-inference is an event-driven system architecture designed for ultra-low-power always-on applications. A tiny, low-power coprocessor (e.g., a microcontroller or microNPU) runs a simple, perpetually active detection model (e.g., for keyword spotting or motion detection). Only when this model detects a trigger event does it signal the PMU to power up the main, high-performance AI accelerator. This architecture dramatically reduces the duty cycle of the power-hungry main processor, extending battery life.

The Energy-Delay Product (EDP) is a combined metric used to evaluate the trade-off between performance (delay) and energy efficiency. It is calculated as Energy Consumed * Execution Time. A lower EDP indicates a more optimal balance. For latency-sensitive AI inference, minimizing EDP is often more critical than minimizing energy alone. PMU policies and DVFS governors are tuned to optimize for EDP, ensuring tasks complete quickly without excessive energy waste.