Milliwatt Budget

A milliwatt budget defines a hard upper limit for average power consumption, often between 1 mW and 100 mW. This is not a target but a non-negotiable system requirement. Exceeding it directly reduces device operational lifetime or risks brownouts in energy-harvesting systems. For context, a standard AA battery holds ~10,000 Joules; a 10 mW budget allows for ~1 million seconds (~11.5 days) of continuous operation.

The budget encompasses total system power, not just the AI accelerator. This includes:

• Sensor power (e.g., microphone, camera)
• Memory access (DRAM refresh, SRAM leakage)
• I/O and radio for communication
• Main processor and always-on co-processor states
• Static (leakage) power of all silicon Effective budgeting requires profiling each component's contribution during the inference pipeline.

Design must manage both metrics:

• Average Power: Determines long-term energy draw from the source (battery, harvester). Must stay under the mW budget.
• Peak Power: The maximum instantaneous power during active computation. High peaks can cause voltage droops, triggering system resets, and may exceed the current delivery capability of small batteries or energy harvesters. Techniques like duty cycling and Dynamic Voltage and Frequency Scaling (DVFS) are used to flatten peaks.

Power is energy per unit time. A 10 mW budget could mean 10 mW continuous or 100 mW for 10% of the time (duty cycle). Most systems use aggressive duty cycling:

• Sleep States: The system spends >99% of time in a micro-watt sleep mode.
• Wake-on-Inference: A tiny co-processor triggers the main AI accelerator only when needed.
• Burst Computation: The model runs at full speed for milliseconds, then powers down completely. The average over the cycle must meet the budget.

The budget dictates the feasible model architecture and hardware platform:

• Model Complexity: Limits the number of parameters and operations per inference (e.g., <50 million multiply-accumulates).
• Quantization: Extreme quantization (INT4, binary) is often mandatory to reduce memory traffic and enable efficient integer compute.
• Hardware Choice: Forces selection of ultra-low-power microcontrollers (MCUs), microNPUs, or Near-Threshold Computing (NTC) designs over application processors.
• Sparsity: Model pruning is critical to eliminate wasted energy on zero-weight computations.

The ultimate system-level efficiency metric under a mW budget is Joule per Inference. It combines: Joule/Inference = (Average Power in Watts) × (Inference Latency in Seconds) Optimization focuses on minimizing this product. A lower Joule per Inference directly translates to more inferences per battery charge. This metric is more useful than pure latency (FPS) or peak Operations per Watt (OP/W), as it captures the full energy cost of a useful unit of work.

Smart earbuds and hearing aids must process audio continuously for features like "Hey Siri" detection or environmental sound classification. This requires an always-on sensing pipeline on an ultra-low-power microcontroller (MCU) or microNPU. The entire audio front-end and a tiny keyword spotting model must fit within a 1-5 mW budget to enable all-day battery life, leveraging techniques like extreme quantization and sparse model inference.

Industrial IoT sensors monitor vibration, temperature, or pressure on machinery, running anomaly detection models locally to predict failures. To operate for years on a coin-cell battery, these nodes use duty cycling, waking briefly to sample, infer, and transmit a result via low-power radio (e.g., LoRaWAN). The milliwatt budget governs the active inference window, requiring models optimized via hardware-aware compression and event-driven inference to minimize active time.

Continuous health monitors (e.g., ECG patches, glucose sensors) process biometric signals to detect arrhythmias or trends. A sub-10 mW budget is critical for patient comfort and week-long wear. This involves dynamic voltage and frequency scaling (DVFS) on a dedicated low-power accelerator, power gating unused circuitry, and running tiny machine learning models that have undergone knowledge distillation from larger clinical models. Joule per inference is a key optimization metric.

Electronic shelf labels (ESLs) update prices wirelessly. Advanced versions include simple optical character recognition (OCR) to audit displayed prices or monitor stock levels via a tiny camera. The entire system, including the radio frequency communication and the ML inference, must operate on a coin cell for 5-10 years. This demands extreme optimization: the model is binarized, inference is scheduled during the brief RF wake-up period, and the MCU leverages deep sleep states for 99.9% of its life.

The primary efficiency metric for evaluating hardware and algorithms under power constraints. It is calculated as useful computational throughput (e.g., inferences per second) divided by average power draw (watts). For a system with a milliwatt budget, maximizing performance-per-watt is the core design objective, often requiring trade-offs between absolute speed, model accuracy, and energy consumption.

A direct, application-level metric of energy efficiency. It measures the total energy, in joules, required to perform a single forward pass (inference) of a machine learning model. This metric is crucial for product architects, as it allows for direct calculation of battery life based on expected usage patterns (e.g., "Device can perform 10,000 inferences on a single charge").