Energy-Delay Product (EDP)

The Energy-Delay Product (EDP) is a figure of merit that quantifies the trade-off between a system's energy consumption and its performance (latency). It is calculated as:

EDP = Energy × Delay

Where:

• Energy is the total energy consumed by the computation, measured in Joules (J).
• Delay is the total execution time (latency), measured in seconds (s).

The resulting unit is Joule-seconds (J·s). A lower EDP indicates a more optimal balance of efficiency and speed for a given workload.

EDP is used to compare system designs where neither raw speed nor minimal energy alone is the sole objective. It penalizes designs that achieve low energy by being excessively slow, and designs that are fast but profligate with power.

• Low Energy, High Delay: Results in a high EDP (undesirable).
• High Energy, Low Delay: Also results in a high EDP (undesirable).
• Optimal Balance: The design point that minimizes EDP represents the best energy-performance Pareto efficiency for that specific task.

It is particularly crucial for battery-constrained edge devices and Internet of Things (IoT) sensors where both battery life and responsiveness matter.

EDP sits within a hierarchy of efficiency metrics, each emphasizing a different aspect of the energy-performance trade-off.

• Energy (Joules): Pure efficiency, ignores speed.
• Delay/Performance (Seconds, FPS): Pure speed, ignores cost.
• Energy-Delay Product (EDP): Balances the two equally.
• Energy-Delay-Squared Product (ED²P): A more aggressive metric that heavily penalizes latency. Calculated as Energy × Delay². Used when timing deadlines are critical.

Choosing EDP vs. ED²P depends on system priorities: EDP for general balance, ED²P for latency-sensitive applications.

In on-device machine learning, EDP is used to evaluate different model compression and hardware configuration choices for a target workload.

Example Analysis:

• Technique A (Pruning): Reduces model size, lowering energy per inference but slightly increasing latency due to irregular sparsity. EDP may increase or decrease based on the balance.
• Technique B (Quantization): Enables use of low-power integer units, reducing both energy and latency, almost certainly lowering EDP.
• Hardware Tuning: Using Dynamic Voltage and Frequency Scaling (DVFS) to run a processor at a lower frequency/voltage point increases delay but saves more energy. The EDP curve helps find the optimal operating point.

Accurately calculating EDP requires simultaneous measurement of power and precise timing.

Required Tools:

• Power Monitor: Hardware tool (e.g., Monsoon power meter, Joulescope) or on-chip Power Management Unit (PMU) to sample current and voltage at high frequency, integrating to get Energy.
• High-Resolution Timer: To measure Delay from inference start to completion.

Process:

1. Isolate the target computation (e.g., a single model inference).
2. Start power sampling and timer.
3. Execute the workload.
4. Stop timer and power sampling.
5. Calculate: Energy = ∫ Power(t) dt. Then EDP = Energy × Delay.

Profiling across many runs is necessary to account for system variability.

While powerful, EDP has limitations that must be understood:

• Task-Dependent: The optimal EDP point is specific to a particular workload. A different model or input size changes the optimum.
• Ignores Absolute Constraints: A design with a fantastic EDP might still miss a hard real-time deadline or exceed a total milliwatt budget.
• Static vs. Dynamic Workloads: EDP typically measures a single, fixed task. For systems with variable workloads, an average or worst-case EDP must be considered.
• Overhead Exclusion: Ensure measurements focus on the core computation. Initialization, data movement, and system overheads should be included if they are part of the critical path, or excluded for a pure computational comparison.

It is best used as a comparative tool alongside other metrics like Joule per Inference and Frames Per Second (FPS).

Performance-per-watt is a fundamental efficiency metric for computing systems, defined as the amount of useful computational work delivered for each watt of power consumed. It is a higher-level, more application-focused metric than EDP.

• Key Difference from EDP: While EDP combines energy and latency into a single product (J·s), performance-per-watt is a ratio (e.g., inferences/second per watt). A system optimized for performance-per-watt may not be optimal for EDP if it achieves high throughput at the cost of significantly increased latency.
• Use Case: This metric is critical for data center operators and mobile device architects who need to maximize computational output within a fixed thermal design power (TDP) or battery budget.

Dynamic Voltage and Frequency Scaling (DVFS) is a hardware power management technique that dynamically adjusts a processor's operating voltage and clock frequency based on real-time computational workload. It is a primary lever for directly optimizing the Energy-Delay Product.

• Mechanism: Lowering voltage and frequency reduces dynamic power (proportional to ~CV²f) at the cost of increased task latency. The optimal DVFS operating point for minimal EDP often lies below the maximum frequency.
• Application in AI: Inference engines use DVFS to run less critical model layers at lower frequencies, or to quickly complete a burst computation before returning to a low-power state.

Joule per inference is a direct, absolute energy efficiency metric that measures the total energy, in joules, required to perform a single forward pass of a machine learning model on a given hardware platform.

• Measurement: It is obtained by integrating the instantaneous power draw over the exact duration of one inference execution. This provides a clear, comparable cost for a unit of AI work.
• Relation to EDP: EDP = (Joule per Inference) × (Inference Latency in seconds). Therefore, optimizing for joules per inference is necessary but not sufficient for optimizing EDP; latency must also be considered. A low-energy inference that is very slow can still result in a poor EDP.

Power profiling is the process of measuring and analyzing the detailed power consumption of a hardware system over time. It is the essential empirical method for calculating metrics like EDP and Joule per Inference.

• Tools: Uses specialized hardware (e.g., Monsoon power monitors, Joulescopes) or on-chip sensors to capture high-fidelity current and voltage traces.
• Workflow for EDP: Engineers correlate power spikes in the trace with specific software operations (e.g., running a particular model layer). By isolating the energy consumed during the inference task and measuring its duration, they can compute the EDP. This reveals optimization targets, such as memory-bound layers causing high idle power.

A milliwatt budget is a strict power consumption constraint, typically in the range of single-digit to tens of milliwatts, imposed on an edge device or subsystem. This budget directly dictates the feasible design space for EDP optimization.

• Context: Common for devices powered by small batteries (e.g., hearing aids, sensors) or energy harvesters (e.g., solar, vibration).
• Design Implication: Under a tight milliwatt budget, engineers must often accept higher latency (increased delay) to keep energy per operation low enough to meet the average power constraint. This frequently pushes system design towards Near-Threshold Computing (NTC) or aggressive duty cycling, where EDP is a critical trade-off metric.

Near-Threshold Computing (NTC) is a circuit design paradigm where digital logic operates with a supply voltage close to the transistor's threshold voltage. It provides an extreme trade-off between energy efficiency and performance that is highly relevant to EDP minimization.

• Energy-Delay Trade-off: Operating at near-threshold voltages drastically reduces dynamic energy (~V²) but significantly increases gate delay. This non-linear relationship creates a "sweet spot" where the product of energy and delay (EDP) is often minimized, rather than at the maximum frequency (max performance) or minimum voltage (min energy).
• Use Case: NTC is explored for always-on, energy-scavenging inference coprocessors where ultra-low power is paramount and moderate latency is acceptable.