Power States (C-States)

The C0 state is the active operating mode where the processor core is executing instructions. All internal clocks are running, and the core consumes its full dynamic and static power. This is the only state in which useful computational work is performed.

• Power Consumption: Maximum (100% of TDP).
• Exit Latency: 0 nanoseconds (already active).
• Context: All architectural state (registers, caches) is fully powered and valid.

The C1 state (often called Halt) is the shallowest idle state. The core stops executing instructions and halts its internal clock, but remains fully powered and voltage is maintained. The core can return to C0 almost instantaneously.

• Key Mechanism: Core clock is gated.
• Power Saved: Primarily dynamic power from clock switching.
• Use Case: Extremely short idle periods between instruction bursts.

The C6 state is a deep sleep state where the core's power is completely removed (power gated). The core's architectural state is saved to a dedicated SRAM before power-down and restored on wake-up. This eliminates virtually all leakage power for that core.

• Power Saved: >95% of core leakage power.
• Exit Latency: High (tens to hundreds of microseconds).
• State Retention: Core voltage (Vcc) is reduced to zero; state saved externally.

Package C-States (PC-states) apply power management to the entire processor package, including shared resources like the last-level cache (LLC) and integrated memory controller. Deeper package states like PC6 or PC10 can power-gate these shared blocks when all cores are in a deep C-state.

• Scope: Manages power for shared, uncore components.
• Impact: Enables significant system-level power savings beyond core C-states.
• Trigger: Requires all cores to be in a sufficiently deep C-state (e.g., C6).

A fundamental characteristic of C-states is the trade-off between power savings and latency. Deeper C-states save more power but have longer entry and exit latencies due to the time required to power down/up circuits and save/restore state.

• C1 Exit Latency: <1 microsecond.
• C6 Exit Latency: ~100 microseconds.
• OS Governor: The operating system's idle driver must predict idle duration to select the deepest C-state where the energy saved outweighs the latency penalty of entering/exiting.

C-states are defined by the Advanced Configuration and Power Interface (ACPI) specification. The operating system's CPU idle driver requests state transitions via ACPI methods. Firmware (ACPI tables) reports the available states and their latencies to the OS.

• Standardization: Ensures OS compatibility across different CPU architectures.
• Control Flow: OS observes workload → predicts idle → requests C-state via MWAIT instruction or similar → hardware executes transition.
• Exposed Interface: /sys/devices/system/cpu/cpuX/cpuidle/ in Linux.

AI inference is characterized by short, intense computational bursts followed by idle periods while waiting for new data or I/O. C-States allow the NPU to enter deep idle states (e.g., C6) between these bursts, dramatically reducing static leakage power. This is essential for edge devices where the NPU may be active only 10-30% of the time, making idle power the dominant factor in total energy consumption.

C-States and Performance States (P-States) are used in tandem. An optimal power management policy first uses Dynamic Voltage and Frequency Scaling (DVFS) to lower voltage/frequency (P-States) for a moderate load. If the core becomes fully idle, it then initiates a transition into a C-State. The transition latency and energy cost of entering/exiting a deep C-State must be amortized over the idle period. Hardware-software co-design is required to predict idle durations accurately.

The primary trade-off for C-States is wake-up latency. Deeper states (C6) save more power but take longer (microseconds to milliseconds) to restore context and resume computation. For real-time AI applications (e.g., autonomous vehicle perception), uncontrolled deep sleep can cause jitter and deadline misses. Solutions involve:

• Shallow C-States (C1) for very short idle periods.
• Hardware-based autonomous entry/exit managed by the NPU's Power Management Unit (PMU).
• OS/Driver hints from the AI framework predicting workload gaps.

Modern NPUs have hierarchical power domains. C-States can be applied at different levels:

• Core-level: Individual processing cores or MAC arrays can be independently powered down.
• Tile/Cluster-level: Groups of cores sharing an L1 memory or NoC router can enter a state together.
• Full NPU-level: The entire accelerator can be powered off, requiring re-initialization by the host CPU. Fine-grained control allows parts of the NPU to remain active for low-priority tasks while others sleep, maximizing utilization of Dark Silicon.

The NPU's C-States are integrated into the platform-wide Advanced Configuration and Power Interface (ACPI) framework. The host OS, via drivers, issues commands to the NPU's firmware (often using standards like PSCI on ARM). This ensures the NPU's sleep states are coordinated with CPU, GPU, and I/O device states during system-wide sleep (S3) or hibernate (S4) events, preventing power rail conflicts and ensuring clean wake-up sequences.

Effective C-State usage is non-negotiable for staying within Thermal Design Power (TDP) and power delivery network (PDN) limits. By minimizing idle power, C-States reduce the baseline thermal load, allowing more thermal headroom for sustained peak performance during active bursts. This directly improves the Performance per Watt metric. Power budgeting algorithms use C-State residency metrics to dynamically allocate power between CPU, GPU, and NPU domains.

Performance States (P-States) are ACPI-defined operational modes for an active (C0 state) processor core, where the operating system dynamically selects voltage and frequency pairs to balance performance and power consumption. Unlike C-States which manage idle power, P-States manage active power.

• P0 represents the highest performance state (maximum voltage/frequency).
• Pn states (e.g., P1, P2) represent successively lower performance and power levels.
• Governed by the same Advanced Configuration and Power Interface (ACPI) specification as C-States, enabling OS-directed power and performance management.

Dynamic Voltage and Frequency Scaling (DVFS) is the underlying hardware technique that enables P-States by dynamically adjusting a core's supply voltage and clock frequency in response to workload demand. It is a primary method for reducing dynamic power, which scales with the square of the voltage (P ∝ V² * f).

• Voltage-Frequency Coupling: Lower frequencies allow a reduction in operating voltage, yielding quadratic power savings.
• Transition Latency: Switching between voltage/frequency pairs incurs a time penalty, which scheduling algorithms must account for.
• Hardware Control: Typically managed by an on-chip Power Management Unit (PMU) based on OS requests.

The Advanced Configuration and Power Interface (ACPI) is the fundamental open standard that defines the software-hardware interface for power management, configuration, and thermal monitoring. It provides the framework for OS-directed control of C-States, P-States, and system sleep states (S-States).

• Global System States (G-States): Define overall system power status (e.g., G0 working, G3 mechanical off).
• Sleep States (S-States): Define low-power system suspend modes (e.g., S3 suspend-to-RAM).
• Device States (D-States): Define power states for peripheral devices.
• The ACPI specification provides the tables and methods that allow an operating system to discover and control these power resources.

A Power Management Unit (PMU) is a dedicated hardware controller, often a microcontroller or finite state machine, embedded within a System-on-Chip (SoC) or processor. It executes the low-level sequences for power state transitions, voltage regulation, and clock control.

• Orchestrates Transitions: Handles the complex, timing-critical sequence of steps to enter/exit C-States and P-States.
• Interface for Firmware: The PMU is typically controlled by platform firmware (e.g., via the Power State Coordination Interface (PSCI) on ARM) which in turn services OS requests.
• Manages Power Domains: Controls power gating switches and isolation cells for different blocks within the chip.

Leakage Power (or static power) is the electrical power dissipated by a transistor due to unwanted subthreshold and gate oxide current when the transistor is in a nominally 'off' state. It is a primary target for C-State power savings.

• Temperature Dependent: Leakage current increases exponentially with junction temperature.
• Process Dependent: Becomes more significant with smaller semiconductor process nodes (e.g., sub-10nm).
• C-State Impact: Deeper C-States (e.g., C6) aggressively reduce leakage by power gating entire core domains, cutting voltage to near zero. Shallower states may only clock gate the core, stopping dynamic power but not leakage.

The Power State Coordination Interface (PSCI) is an ARM standard that defines a firmware interface for power management operations on ARMv8 and later systems. It provides a vendor-agnostic way for an operating system hypervisor to request power state changes from the platform's trusted firmware.

• Core Idle Management: The OS calls PSCI CPU_SUSPEND to request entry into a specific low-power state for a core.
• Core Power On/Off: Provides functions for turning cores on (CPU_ON) and off (CPU_OFF).
• System Suspend: Manages system-level suspend/resume operations.
• This interface sits between the OS and the hardware-specific Power Management Unit (PMU).