Energy-Delay Product (EDP)

The Energy-Delay Product is defined as the product of the total energy consumed by a computation and the time taken to complete it. Its standard formula is:

EDP = Energy × Delay

Where:

• Energy is measured in Joules (J), representing the total electrical energy used.
• Delay (or execution time) is measured in seconds (s).

A lower EDP indicates a more optimal balance between speed and power consumption. It penalizes designs that are fast but power-hungry, as well as those that are extremely low-power but unacceptably slow.

EDP provides a single scalar metric to guide hardware and software optimization. The core design goal is to minimize the EDP.

• Optimizing for Performance Alone (minimizing delay) often leads to higher voltage/frequency, causing energy to increase super-linearly, resulting in a poor (high) EDP.
• Optimizing for Energy Alone (minimizing joules) can lead to excessively slow operation, also worsening EDP.

EDP forces architects to find the sweet spot in the design space. For TinyML, this often means operating processors at moderate frequencies below their maximum, using techniques like Dynamic Voltage and Frequency Scaling (DVFS) to find the optimal voltage-frequency point for a given workload.

EDP sits within a hierarchy of performance-efficiency metrics:

• Energy (Joules): The fundamental resource cost.
• Delay/Time (Seconds): The performance metric.
• Power (Watts = Joules/Second): The instantaneous rate of energy use.
• Energy-Delay Product (Joule-seconds): Balances energy and delay.
• Energy-Delay-Squared Product (ED²P): A more aggressive metric that penalizes long delays even more heavily (EDP × Delay). Used when latency is critically important.

For machine learning, Inferences-per-Joule is a common domain-specific metric. EDP can be related to it if the 'delay' per inference is consistent.

EDP is a crucial evaluation metric for TinyML systems due to severe constraints:

• Battery Life: Directly impacted by total energy (Joules). EDP helps maximize useful computations per battery charge.
• Thermal Limits: Microcontrollers and edge chips often lack active cooling. A high-power (high EDP) design will throttle or fail.
• Real-time Requirements: Many sensor applications have deadlines. EDP evaluates if energy savings come at the cost of missing these deadlines.

System designers use EDP to choose between:

• Different neural network architectures (e.g., MobileNet vs. EfficientNet).
• Precision levels (8-bit vs. 4-bit quantization).
• Hardware acceleration modes on a system-on-chip (SoC).

Accurately measuring EDP requires simultaneous capture of energy and time.

Tools and Methods:

• External Precision Power Monitors: Devices like Joulescopes or National Instruments PXIs provide high-fidelity, time-synchronized current/voltage sampling.
• On-Chip Energy Proxies: Many microcontrollers (e.g., Arm Cortex-M with Embedded Trace Macrocell) offer performance counters (cycles) and internal voltage/current sensors for estimation.
• Software Profiling: Tools like the Arm Energy Probe or FreeRTOS+Trace can correlate function execution timelines with power draw.

Procedure: Isolate the workload (e.g., a single inference), measure the average power during execution and the execution time, then calculate: Energy = Avg Power × Time, then EDP = Energy × Time.

While powerful, EDP has limitations that require careful interpretation:

• Fixed Workload Assumption: EDP is meaningful for comparing implementations of the same computational task. It cannot compare a face detection model to a speech recognition model.
• Ignores Static/Leakage Power: For very short tasks, static power during long idle periods before/after the task can dominate total system energy, which EDP of the active period doesn't capture.
• Non-Linear Trade-offs: The relationship between voltage, frequency, energy, and delay is non-linear. The true optimum for ED²P may differ from the EDP optimum.
• System-Level vs. Task-Level: Minimizing EDP for a CPU core might increase energy elsewhere (e.g., in memory). A system-level EDP that includes all components is more representative but harder to measure.

Therefore, EDP is best used as a comparative guide alongside other metrics like peak memory usage and task completion deadlines.

EDP quantifies the fundamental design tension in battery-powered devices. A low-latency, high-performance inference mode consumes more energy per prediction, rapidly depleting the battery. Conversely, an ultra-low-power mode saves energy but introduces unacceptable inference latency. System architects use EDP to find the optimal operating point—for example, a wake-word detector might use a low-EDP, always-on acoustic front-end to trigger a higher-EDP, high-accuracy speech recognizer only when needed.

EDP guides the choice of microcontroller (MCU) or accelerator and its configuration. Designers compare:

• General-purpose MCUs (Cortex-M) vs. TinyML accelerators (e.g., Ethos-U55).
• The impact of Dynamic Voltage and Frequency Scaling (DVFS) settings on both energy and delay.
• Precision vs. EDP: Running inference in INT8 vs. FP32 dramatically reduces both energy and delay, yielding a superior EDP. The optimal choice is the hardware/configuration that delivers the required accuracy with the lowest EDP.

Neural Architecture Search (NAS) for TinyML often uses EDP (or a weighted variant) as the primary optimization target alongside accuracy. The search algorithm evaluates candidate architectures by:

• Profiling inference latency and energy per inference on the target hardware.
• Calculating the EDP for each candidate.
• Selecting architectures that minimize EDP under an accuracy constraint. This automatically discovers networks with favorable properties like layer sparsity, early exits, and efficient operators that jointly minimize energy and time.

For multi-sensor IoT nodes, EDP informs task scheduling. The system must decide when and how to execute sensing, processing, and transmission tasks. An EDP-aware scheduler:

• Batches sensor readings to amortize the wake-up energy cost over multiple inferences.
• Dynamically adjusts the duty cycle of radios and sensors based on event detection confidence.
• Prioritizes tasks to minimize the total EDP of the operational cycle, ensuring the node meets its latency SLA while maximizing its operational lifetime on a battery or harvested energy.

EDP serves as a standardized, holistic benchmark for comparing TinyML frameworks, compilers, and deployed models. A benchmark suite measures:

• Total energy (Joules) to complete a fixed number of inferences.
• Average latency (seconds) per inference.
• The product gives a single figure of merit. For example, a model compiled with TensorFlow Lite Micro might be compared against the same model compiled with Apache TVM or executed on a different MCU, with the lower EDP solution being more efficient for the target application profile.

EDP provides a quantitative basis for the classic offloading dilemma. The decision to process data locally on an edge device versus sending it to the cloud involves comparing:

• Local EDP: Energy & delay for on-device inference.
• Transmission EDP: Energy & delay for radio wake-up, data transmission, cloud processing, and receiving the result. In many scenarios, especially with low-bandwidth sensor data, the energy and latency cost of transmission far exceeds local computation, making low-EDP on-device inference the optimal choice for both responsiveness and total system energy.

A performance-per-watt metric specific to machine learning, measuring the number of neural network inferences a system can perform per joule of energy consumed. It is a direct benchmark for the energy efficiency of AI accelerators and TinyML deployments.

• Primary Use: Benchmarking hardware accelerators, comparing model architectures, and evaluating deployment configurations.
• Calculation: Total inferences completed / Total energy consumed (in Joules).
• Relation to EDP: While EDP (Energy × Delay) is a combined metric, Inference-Per-Watt focuses purely on throughput efficiency. A system optimized for high Inference-Per-Watt may still have a high EDP if its latency is poor.

A power management technique that dynamically adjusts a processor's operating voltage and clock frequency based on real-time computational workload demands to optimize energy efficiency.

• Mechanism: Reduces voltage and frequency during low-demand periods, as dynamic power scales with the square of the voltage and linearly with frequency (P_dynamic ∝ C * V² * f).
• TinyML Application: Critical for microcontrollers running intermittent inference tasks. The OS or runtime can lower voltage/frequency during sensor data collection or idle periods.
• Impact on EDP: Aggressive DVFS lowers energy but increases execution time (delay). The optimal DVFS policy seeks the voltage/frequency point that minimizes the overall Energy-Delay Product.

Describes the design space in machine learning systems where reducing computational precision, model size, or activation sparsity saves energy but may also decrease the model's prediction accuracy.

• Key Techniques: Post-training quantization (e.g., INT8 vs. FP32), pruning (removing redundant weights), and using early exit networks.
• Pareto Frontier: Engineers plot accuracy vs. energy consumption to identify optimal model configurations for a given application constraint.
• Relation to EDP: This trade-off is a primary lever for optimizing EDP. A slightly less accurate model that uses significantly less energy and runs faster can yield a superior EDP for a non-critical application.

An operating system or runtime technique that allocates computational tasks across processing cores and schedules their execution times to minimize total system energy consumption while meeting performance deadlines.

• Strategies: Consolidating tasks onto fewer active cores to allow others to power gate, scheduling compute-intensive tasks during periods of available harvested energy, and intelligently batching sensor data for inference.
• TinyML Context: On heterogeneous MCUs with a main Cortex-M core and a ultra-low-power coprocessor (e.g., an Ethos-U55 NPU), the scheduler decides where to execute inference for minimal system energy.
• Goal: To directly minimize the Energy-Delay Product for a given workload by making intelligent runtime decisions about when and where to compute.

An algorithmic approach that determines the order and timing of task execution on a device with a finite energy budget (e.g., a battery or capacitor), aiming to complete a set of tasks before the available energy is depleted.

• Problem Formulation: Given a battery capacity E_budget and a set of tasks with known energy costs E_task and deadlines, find a feasible schedule.
• Advanced Consideration: Incorporates non-linear battery discharge models and energy harvesting forecasts to make scheduling decisions.
• Difference from EDP: While EDP is a metric for evaluating a single task's efficiency, energy-constrained scheduling is a control policy for managing multiple tasks over time within a hard energy limit.

Software and hardware instruments used to measure and analyze the detailed power consumption of software routines, hardware blocks, or full systems during execution. Essential for calculating metrics like EDP.

• Hardware Tools: High-precision digital multimeters, source measurement units (SMUs), and specialized development boards with integrated current sensors (e.g., Nordic Power Profiler Kit II, Joulescope).
• Software Tools: Traces from on-chip current sensors, processor performance counters (for CPU cycles), and integrated development environment (IDE) plugins that correlate power draw with source code execution.
• Workflow: Engineers use these tools to identify power-hungry functions, validate DVFS policies, and empirically measure the EDP of different model configurations and system states.