Sleep States

Modern processors define a hierarchy of sleep states, commonly called C-states (CPU states). Each deeper state trades off greater power savings for longer wake-up latency.

• C0 (Active): The core is fully active, executing instructions.
• C1 (Halt): The core halts execution but maintains cache coherency; a fast wake-up.
• C2/C3 (Stop-Clock/Deep Sleep): The core's clocks are gated, and caches may be flushed. This saves significant dynamic power.
• C4+ (Deeper Sleep): Core voltage is reduced or removed (power gating), saving both dynamic and static (leakage) power, but requiring a full context restore on wake.

Sleep states extend beyond individual cores to the entire chip package and system.

• S-States (System States): Global ACPI states like S3 (Suspend-to-RAM) where the system context is held in DRAM (a low-power state) and only essential wake logic is active.
• P-States (Performance States): While not a full sleep state, P-states involve Dynamic Voltage and Frequency Scaling (DVFS) to reduce power during lighter workloads, acting as a precursor to entering a sleep state.
• Device-Level States (D-States): Peripherals like GPUs or NPUs have their own sleep state hierarchies, which must be coordinated with core C-states.

The transition in and out of a sleep state is not instantaneous. Entry latency is the time to save context and power down components. Exit latency (wake-up time) is the time to restore power, clocks, and context. This latency is the primary trade-off for the power savings achieved.

• Design Implication: For an AI system, the inference interval must be significantly longer than the total (entry + exit) latency to realize net energy savings. Frequent, sporadic inferences may favor shallower sleep states.

Sleep states are categorized by what system context is preserved, which directly impacts wake-up energy and time.

• Context-Retentive States: The processor's architectural state (register files, caches) is kept powered. This allows a faster wake-up but consumes more static power to maintain memory.
• Context-Loss States: The architectural state is flushed to external DRAM or non-volatile memory, and the core is power-gated. Waking requires a cold boot of the core and restoration of context from slower memory, consuming more energy and time for the wake event but saving maximum power while asleep.

A system in a sleep state must be awakened by a predefined hardware or software event. For edge AI, these are critical for responsive, event-driven inference.

• External Interrupts: Timers (RTC alarms), GPIO pins (sensor triggers), or network interfaces.
• Internal Events: Faults, thermal alerts, or interrupts from an always-on sensing block.
• Software-Driven: A scheduled task from the OS or a wake-on-inference signal from a low-power coprocessor that has detected a trigger pattern.

The Power Management Unit (PMU) is the dedicated hardware orchestrator for sleep states. Its responsibilities include:

• Voltage Sequencing: Ramping up/down supply rails in the correct order to prevent latch-up.
• Clock Gating: Controlling the distribution of clock signals to various blocks.
• Isolation: Inserting isolation cells to prevent floating signals from powered-down blocks from affecting active ones.
• State Negotiation: Implementing the Advanced Configuration and Power Interface (ACPI) or similar firmware protocols that allow the OS to request and transition between states.

In mobile System-on-Chips (SoCs), sleep states are standardized as C-states (CPU idle states).

• C1 (Halt): The CPU core stops executing instructions but remains powered and can wake almost instantly.
• C2 (Stop-Clock): The core's internal clocks are gated, reducing dynamic power.
• C3/C4 (Deep Sleep): The core's voltage is reduced or removed (power gated), and its cache state may be flushed to shared memory, drastically cutting leakage power. Modern SoCs integrate these with Dynamic Voltage and Frequency Scaling (DVFS) and have separate power domains for the NPU, GPU, and ISP, allowing them to be independently powered down.

Neural Processing Units (NPUs) and AI accelerators implement their own granular sleep hierarchies.

• Tensor Core/Array Power Gating: Individual compute arrays can be power-gated when not in use.
• SRAM Retention Modes: On-chip memory (SRAM) can be placed in a low-power data retention state, preserving model weights and activations while reducing refresh power, rather than flushing to DRAM.
• Clock Domain Gating: Independent clock domains for control logic, data movement engines, and compute units allow unused sections to be fully clock-gated. This architecture enables the accelerator to wake from a deep state and resume inference with minimal latency, as critical context is retained on-chip.

Efficient sleep requires intelligent wake-up mechanisms. This is often managed by a hierarchy of processors.

• Always-On Domain (AON): A tiny, ultra-low-power microcontroller or microNPU (e.g., in milliwatt budgets) remains active to monitor sensors. It runs simple always-on sensing models (e.g., keyword spotting, motion detection).
• Wake-on-Inference: The AON subsystem triggers a wake-up interrupt to the main CPU or NPU only when its inference passes a confidence threshold. This event-driven inference prevents the high-power system from idling unnecessarily.
• Fast Wake-Up Sequencers: Dedicated hardware sequencers in the Power Management Unit (PMU) restore power rails and clocks in a specific, optimized order to minimize wake-up latency.

Hardware sleep states are managed by software stacks.

• OS Idle Scheduler: The operating system's idle thread places CPU cores into the deepest possible C-state based on predicted idle duration.
• AI Framework Runtime: Inference runtimes (e.g., TFLite, ONNX Runtime) provide APIs to signal workload completion, allowing the driver to put the NPU to sleep. They may also batch inferences to maximize active periods and allow for deeper sleep (duty cycling).
• Power-Aware Schedulers: Advanced schedulers use battery-aware scheduling and power profiling data to decide when to run inference tasks, aligning them with expected active periods to avoid frequent, inefficient wake-ups.

Deeper sleep states save more power but incur higher wake-up latency and energy cost to restore context.

• Shallow Sleep (C1): Wake-up in microseconds. Ideal for very short, predictable idle periods.
• Deep Sleep (C3/C4, NPU Power-Gated): Wake-up can take milliseconds as PLLs relock and caches are repopulated. The Energy-Delay Product (EDP) must be evaluated to determine if the energy saved during sleep outweighs the energy and performance penalty of waking. Hardware designers optimize this by providing multiple state options and fast transition paths for common use cases.

Consider a smart security camera performing person detection at 1 FPS.

1. Active Burst (10ms): NPU and CPU are fully active, consuming ~1.5W. A frame is processed.
2. Idle Period (990ms): The main CPU and NPU enter deep sleep (C4), power-gating logic and placing SRAM in retention. Power drops to ~5mW.
3. Always-On Subsystem: A low-power MCU runs a simple motion sensor check. If motion is detected, it triggers wake-on-inference. This duty cycling results in an average power of ~20mW, enabling long battery life, versus >1W for continuous operation.