Dynamic Power

The activity factor (α) represents the probability that a logic gate will switch (0→1 or 1→0) per clock cycle. It is a direct multiplier in the dynamic power equation.

• Sparsity Exploitation: Pruned neural networks have low activity factors as many weights are zero, preventing multiplications and associated switching.
• Data-Dependent Power: Power consumption varies with input data; a blank image may cause less switching than a complex scene.
• Architectural Impact: Designs with clock gating and data gating dynamically reduce α by disabling unused circuit paths.

The capacitive load (C) is the total capacitance that must be charged/discharged during a logic transition. It is determined by the physical geometry of the silicon.

• Wire Length: Longer on-chip interconnects between computational units (e.g., multiply-accumulate cells) have higher capacitance. Systolic arrays minimize this.
• Transistor Size: Larger drive transistors can switch faster but have higher intrinsic capacitance.
• Technology Node: Smaller process nodes (e.g., 5nm vs. 28nm) generally reduce capacitance, but wire capacitance becomes relatively more dominant.

Supply voltage (V) has a quadratic relationship with dynamic power (P ∝ V²), making it the most powerful lever for energy reduction.

• Dynamic Voltage and Frequency Scaling (DVFS): The cornerstone technique. Voltage and frequency are scaled down during low-utilization periods.
• Near-Threshold Computing (NTC): Operating the processor at voltages near the transistor's threshold voltage offers optimal energy-delay product (EDP) for throughput-oriented edge AI.
• Voltage Domains: Modern SoCs use multiple, independent voltage domains to apply optimal V to different blocks (e.g., NPU, CPU, memory).

Clock frequency (f) is a linear multiplier in the power equation. Higher frequency increases throughput but directly increases power and heat.

• Frequency Scaling: Often paired with voltage scaling (DVFS). Reducing frequency by 50% can nearly halve dynamic power if voltage is also reduced.
• Thermal Constraints: Sustained high frequency leads to heat buildup, triggering thermal throttling, which forces frequency reduction to prevent damage.
• Burst Inference: Running the NPU at high frequency for a short burst to finish a task quickly, then idling, can be more efficient than running slower for longer.

The design of the compute unit itself critically influences all factors (α, C, V, f).

• Dataflow Architectures: Weight-stationary or output-stationary dataflows minimize data movement and associated switching.
• Precision-Specific Units: Dedicated 8-bit integer (INT8) units have simpler logic and lower capacitance than FP32 units.
• Sparse Compute Engines: Hardware that skips operations where weights or activations are zero, effectively setting α=0 for those paths.
• On-Chip Memory Hierarchy: Large SRAM caches near compute reduce energy-hungry off-chip DRAM accesses.

Software decisions directly determine the hardware's switching activity and required frequency.

• Model Compression: Quantization reduces bit-width, lowering the capacitance switched per operation. Pruning increases sparsity (reduces α).
• Kernel Fusion: Compiler-level fusion of operations (e.g., convolution + ReLU) reduces intermediate data writes to memory, a high-capacitance operation.
• Efficient Formats: Using sparse tensor formats (e.g., CSR) allows the hardware engine to skip zero computations.
• Batch Size Selection: Larger batches improve hardware utilization but increase memory capacitance; smaller batches reduce latency and peak memory demand.

A fine-grained technique that disables the clock signal to specific circuit blocks, registers, or arithmetic logic units (ALUs) when they are idle during a computation cycle. In an AI accelerator, this prevents unnecessary switching activity in:

• Unused multiplier-accumulator (MAC) units within a systolic array.
• Memory banks not being accessed in a given cycle.
• Pipeline stages waiting for data. By stopping the clock, dynamic power consumption in those gated regions drops to nearly zero.

Architectural strategies that avoid continuous computation. Instead of running models at a fixed frame rate, inference is triggered only by meaningful events, drastically reducing average dynamic power.

• Event-Driven Inference: A low-power wake-on-inference coprocessor (e.g., a microNPU) monitors sensors and activates the main AI accelerator only when a specific trigger (like a keyword or visual event) is detected.
• Duty Cycling: The system alternates between very short active inference windows and long sleep periods, limiting the total time the high-power compute fabric is actively switching.

Leverages model compression techniques like pruning to create sparse neural networks, which directly reduce dynamic power. In a pruned model, a significant percentage of weights are zero. Specialized hardware and kernels can:

• Skip computations involving zero weights entirely, preventing the associated switching activity in MAC units.
• Suppress memory reads for zero-valued weights, reducing power-hungry DRAM access. This turns algorithmic efficiency (fewer operations) directly into electrical efficiency (less switching capacitance charged/discharged).

Minimizes the most power-expensive operations in the inference pipeline: data movement. Dynamic power is consumed each time a bit toggles on a long, capacitive bus or in a large memory array. Techniques include:

• Fused Operations: Compiler-based graph optimizations fuse layers to keep intermediate tensors in fast, low-power SRAM cache, avoiding writes/reads to higher-power DRAM.
• Data Reuse: Architecting systolic arrays and weight-stationary dataflows to maximize reuse of weights and activations locally, minimizing global memory traffic.
• Precision Gating: Using lower numerical precision (e.g., INT8 vs. FP32) reduces the number of bits switched per operation and the bus width required for data transfer.

Orchestrates inference workloads to maximize hardware utilization and minimize idle switching. Dynamic power is consumed even during inefficient execution. Smart scheduling addresses this by:

• Continuous Batching: For server-edge scenarios, dynamically grouping inference requests of varying sizes to keep the accelerator's compute units consistently occupied, avoiding frequent ramp-up/down power cycles.
• Kernel Selection: Choosing the most energy-efficient implementation of an operator (e.g., a depthwise convolution kernel) for the specific input shape and target hardware.
• Thermal-Aware Scheduling: Monitoring chip temperature and intelligently queueing or throttling jobs to avoid performance cliffs from thermal throttling, which forces drastic voltage/frequency reductions.

Static power is the electrical power consumed by a CMOS integrated circuit when it is powered on but not actively switching. Unlike dynamic power, it is not caused by switching activity but by unwanted current leakage through transistors, even when they are in an 'off' state. It is a function of transistor design, temperature, and supply voltage.

• Primary Cause: Subthreshold leakage current.
• Key Relationship: Becomes a dominant factor as transistor sizes shrink (process node scaling).
• Management Techniques: Power gating, body biasing, and using high-threshold voltage (HVT) cells.

DVFS is a runtime power management technique that dynamically adjusts a processor's operating voltage and clock frequency based on real-time computational workload. Since dynamic power is proportional to the square of the voltage and linearly to frequency, reducing these parameters during low-demand periods yields quadratic energy savings.

• Core Principle: Trade performance for efficiency.
• Application in AI: Used in mobile SoCs and NPUs to throttle during low-priority inference tasks.
• Challenge: Requires fine-grained workload prediction to avoid latency spikes.

Power gating is a circuit-level technique that completely shuts off power to inactive or idle blocks of a silicon chip using a header or footer switch. This eliminates both dynamic power and static (leakage) power consumption in those regions, making it the most aggressive power-saving method.

• Implementation: Uses high-threshold voltage transistors as power switches.
• Trade-off: Introduces wake-up latency and requires state retention strategies.
• Use Case: Turning off entire NPU cores, large SRAM blocks, or peripheral controllers when not in use.

Clock gating prevents the clock signal from propagating to specific circuit blocks or registers when they are not performing useful work. This stops unnecessary switching activity in the clock network and the underlying logic, directly reducing dynamic power consumption.

• Granularity: Can be applied at the module, block, or register level.
• Overhead: Minimal, as it only involves adding a simple enable gate to the clock path.
• Ubiquity: Heavily used in synthesis and is a fundamental optimization in RTL-to-GDS flows.

Performance-per-watt is the primary efficiency metric for computing systems, defined as the amount of useful computational work delivered for each watt of power consumed. It directly links the business value of an AI system (e.g., inferences per second) to its operational cost and thermal design.

• Calculation: (Throughput e.g., FPS or IPS) / (Average Power in Watts).
• Industry Benchmark: Critical for evaluating edge AI accelerators and mobile SoCs.
• Holistic View: Encourages co-optimization of algorithms, software, and hardware.

Joule per inference is a direct, end-to-end energy efficiency metric that measures the total energy, in joules, required to perform a single forward pass (inference) of a machine learning model on a given hardware platform. It is derived by integrating power over the exact inference time.

• Precision: More accurate than average power for sporadic workloads.
• Utility: Allows direct comparison of different model architectures and hardware platforms for a specific task.
• Measurement: Requires precise energy tracing hardware (e.g., Monsoon power monitor, chip PMU counters).