Power Gating

The core component of a power gating structure is a sleep transistor (or header/footer switch). This is a large, high-threshold-voltage (high-Vt) transistor placed between the power supply rail and the logic block (header) or between the block and ground (footer).

• Function: Acts as a switch to completely disconnect the block from VDD or GND.
• Design Trade-off: Sized to minimize voltage drop (IR drop) when active, while also minimizing leakage and area overhead.
• Types: Header switches cut off VDD; footer switches cut off GND. Some designs use both for finer control.

A chip is partitioned into multiple power domains, each of which can be independently powered on or off.

• Fine-Grain Power Gating: Applied at the level of individual standard cells or small logic clusters. Managed automatically by synthesis tools.
• Coarse-Grain Power Gating: Applied at the level of major functional blocks (e.g., a GPU core, a DSP unit, an inactive NN layer). Managed by system firmware or a Power Management Unit (PMU).
• Isolation Cells: Required at the outputs of a powered-down domain to prevent floating signals from causing crowbar current in neighboring active domains. They clamp outputs to a known logic state (0 or 1).

A critical challenge is preserving the logical state (register values) of a block before it is powered off, so computation can resume instantly upon wake-up.

• Retention Registers: Special flip-flops with a separate, always-on power supply that preserves their state while the surrounding logic is powered down.
• Shadow Registers: State is saved to a separate memory (e.g., SRAM) before shutdown and restored after power-up, adding latency.
• Trade-off: Retention registers increase area and design complexity but enable ultra-fast wake-up, essential for frequently toggled blocks.

Turning a power domain on or off is not instantaneous and involves controlled sequencing to avoid large current surges (inrush current) and circuit damage.

• Shutdown Sequence: 1) Clock is stopped. 2) Isolation is enabled. 3) Retention registers save state. 4) Sleep transistor is turned off.
• Wake-up Sequence: 1) Sleep transistor is turned on. 2) Power rail stabilizes. 3) Retention registers restore state. 4) Isolation is disabled. 5) Clock is resumed.
• Impact: This latency (often microseconds) determines how frequently power gating can be profitably used. Fast wake-up is a key design goal.

Power gating requires sophisticated control logic, often centralized in a Power Management Unit (PMU).

• Control Signals: Sleep signals control the sleep transistors. Isolation enable signals control the isolation cells. Retention save/restore signals control the retention registers.
• Software vs. Hardware Control: Can be initiated by software (OS driver) or automatically by hardware power controllers based on activity timers or workload predictors.
• Power Gates: The distributed network of sleep transistors and their local control logic that physically implements the switch.

The principal advantage of power gating is the near-total elimination of static power (leakage power) in the gated domain.

• Leakage Sources: Subthreshold leakage, gate oxide tunneling, and junction leakage are all reduced to near zero when the block is unpowered.
• Scaling Importance: As transistor geometries shrink, leakage power becomes a dominant, sometimes majority, component of total power. Power gating is essential in modern finFET and sub-nanometer processes.
• Contrast with Clock Gating: Clock gating only eliminates dynamic power from switching; power gating eliminates both dynamic and static power.

In a System-on-Chip (SoC) for edge AI, the Neural Processing Unit (NPU) or Digital Signal Processor (DSP) dedicated to model inference is a prime target for power gating. When the device is idle (e.g., a smart speaker not processing speech), the entire NPU block can be shut off, eliminating both its dynamic power (from computation) and static power/leakage (from transistors merely being powered on). This is distinct from clock gating, which only stops switching activity. For example, during a video frame capture interval on a security camera, the vision processor can be gated off, waking only when a frame is ready for inference.

Modern AI chips implement power gating at multiple levels of granularity to maximize savings:

• Block-Level: Shutting down entire functional units (e.g., a matrix multiplication engine).
• Sub-Block/Module-Level: Gating individual Processing Elements (PEs) within an NPU array that are not in use for a specific layer.
• Register-Level: Using header/footer switches to cut power to memory banks or register files holding inactive data. This hierarchy allows for partial shutdown during computation. For a sparse model, only the PEs processing non-zero weights are powered, while others remain gated.

Power gating is not a static setting; it is dynamically controlled by system software or a Power Management Unit (PMU). Effective strategies include:

• Burst Inference: Accumulating sensor data (e.g., multiple audio frames) and then waking the AI block for a single, high-utilization compute burst before returning it to a gated state. This minimizes the overhead of frequent power state transitions.
• Duty Cycling: In periodic sensing applications, the AI block is powered on at a fixed interval (e.g., every 100ms for an activity classifier), processes data, and is immediately gated off.
• Event-Driven Wake-up: A tiny, always-on microcontroller (MCU) or sensor hub runs a minimal detection model (e.g., keyword spotting). Only upon detecting a trigger event does it signal the PMU to power-gate-on the main NPU for a more complex task.

Applying power gating introduces critical engineering trade-offs:

• Wake-up Latency: The time to restore power, re-initialize circuitry, and reload context (e.g., model weights from DRAM) can be significant (microseconds to milliseconds). This directly impacts real-time inference latency.
• Transition Energy: The energy cost of switching the power gates themselves and recharging power rails. If the idle period is too short, this overhead can negate the energy saved by gating.
• State Retention: Some designs use retention flip-flops or Always-On (AON) memory to preserve critical state during gating, adding area and complexity but reducing wake-up time. The optimal gating policy balances these factors against the milliwatt budget of the device.

Power gating synergizes with algorithmic techniques from on-device model compression:

• Sparsity-Induced Gating: Model pruning creates networks with many zero weights. Hardware can be designed to detect zero-valued activations and gate the corresponding computation paths, turning algorithmic sparsity into direct power savings.
• Quantization Effects: Low-bit quantization (e.g., INT4) reduces the switching activity and capacitance in arithmetic logic units (ALUs), but the underlying circuits still leak power. Power gating provides the final step of eliminating this leakage power entirely during idle times. This hardware-software co-optimization is essential for maximizing frames per joule (FPJ) or inferences per joule.

Power gating is a key enabler for two frontier application domains:

1. Always-On Sensing: Devices like smart earbuds or wearables must process sensor data continuously for hours or days. By aggressively gating the majority of the chip between inference bursts, average power consumption can be reduced to single-digit milliwatts, enabling multi-day battery life.
2. Energy-Harvesting & Intermittent Systems: For devices powered by energy harvesting (solar, RF, vibration), available power is minuscule and intermittent. Power gating allows the system to accumulate energy in a capacitor while all logic is shut off, then power-on, execute a critical inference (e.g., condition monitoring), save state to non-volatile memory, and shut down again before energy is depleted.

A complementary power-saving technique to power gating. While power gating removes supply voltage, clock gating disables the clock signal to idle circuit blocks. This prevents unnecessary transistor switching, eliminating dynamic power consumption but not static leakage power. It is less aggressive than power gating but has a faster wake-up time and is often used for finer-grained, more frequent control.

A runtime power management technique that dynamically adjusts a processor's operating voltage and clock frequency based on the instantaneous computational workload. It exploits the cubic relationship between voltage and dynamic power (P ~ CV²f). By lowering voltage and frequency during periods of low utilization, DVFS provides significant energy savings, often used in conjunction with power gating for coarse-grained power domain control.

The primary adversary that power gating is designed to defeat. Static power is the current that leaks through a transistor even when it is not switching, caused by subthreshold and gate oxide leakage. It becomes a dominant factor in power consumption as semiconductor process nodes shrink below 90nm. Power gating is the most effective circuit-level technique to reduce static power to near zero for inactive blocks.

Predefined low-power modes of a processor or System-on-Chip (SoC), often standardized (e.g., ACPI C-states). These states represent a hierarchy of power-down levels:

• Shallow Sleep: Only clocks are gated.
• Deep Sleep: Clocks gated and non-essential power domains are power-gated.
• Hibernation: Nearly all logic is power-gated, with state saved to non-volatile memory. Power gating is the enabling technology for implementing deep sleep and hibernation states.

The dedicated hardware controller that orchestrates power-saving techniques like power gating, DVFS, and clock gating. The PMU is responsible for:

• Generating and sequencing the multiple voltage rails required for power domains.
• Controlling the isolation cells and retention registers during power gating sequences.
• Responding to software requests or hardware triggers to transition power states. It is the central nervous system for implementing energy-efficient inference policies.

A system-level architecture that leverages power gating at a macroscopic scale. A tiny, always-on, low-power coprocessor (e.g., a microcontroller or microNPU) runs a simple trigger model (e.g., for keyword spotting). The main, high-performance AI accelerator remains power-gated until a relevant event is detected. This pattern dramatically reduces average system power by keeping the most energy-hungry component off until absolutely needed.