MCUNet

MCUNet enables always-listening voice user interfaces on battery-powered microcontrollers. Its optimized TinyNAS designs models small enough to reside in SRAM, allowing continuous audio stream analysis for detecting wake words (e.g., 'Hey Siri', 'OK Google') or simple commands without waking the main application processor.

• Key Feature: Ultra-low latency response by eliminating external memory access.
• Example: A smart home sensor that activates lights on a vocal command, operating for months on a coin-cell battery.

Deploys image classification and object detection for event-triggered sensing. A microcontroller with a low-resolution camera can run a MCUNet-optimized model to detect specific visual events.

• Industrial Monitoring: Detecting the absence of a part on an assembly line or unauthorized personnel entry.
• Smart Agriculture: Identifying pest presence or fruit ripeness from in-field camera traps.
• Efficiency: The system remains in a deep sleep state, with the MCUNet model waking the main processor only when a target visual event is confirmed, drastically saving energy.

Enables real-time biometric analysis and activity recognition directly on wearable devices. The TinyEngine inference runtime's efficient memory management allows complex sensor fusion models to run within the tight SRAM limits of wearable MCUs.

• Applications: Continuous heart rate variability (HRV) analysis, fall detection for the elderly, and complex gesture recognition for AR/VR controllers.
• Privacy Benefit: Sensitive health data is processed locally, never leaving the device, ensuring compliance with regulations like HIPAA.
• Performance: Achieves real-time inference on high-frequency sensor data (e.g., from accelerometers, gyroscopes, PPG sensors).

MCUNet allows time-series forecasting and vibration analysis models to run directly on vibration sensors and motor controllers. By analyzing patterns in sensor data on the edge, it can predict equipment failure.

• Process: A model trained on normal vs. abnormal vibration signatures runs continuously on an MCU attached to a motor.
• Advantage: Enables immediate, local alerts for shutdowns, avoiding costly cloud latency and maintaining operation during network outages.
• Data Reduction: Only summary alerts or aggregated health scores are transmitted, minimizing network bandwidth and cloud storage costs.

Facilitates intelligent sensing nodes for agriculture, smart cities, and building management. MCUNet models can classify complex sensor patterns that simple thresholding cannot detect.

• Examples:
  • Distinguishing between rain, wind, and hail from acoustic sensor data.
  • Detecting specific gas leaks from multi-sensor array patterns.
  • Classifying soil quality from spectral sensor data.
• System Impact: The co-design reduces peak SRAM usage, allowing larger, more accurate models to run. This enables longer intervals between battery changes or operation via energy harvesting (solar, RF).

While primarily an inference framework, MCUNet's memory-efficient design lays the groundwork for federated learning and personalization directly on microcontrollers. The TinyEngine's precise memory control is critical for managing the intermediate activations and gradients required for training.

• Use Case: A wearable device that adapts its activity recognition model to a user's unique gait over time, without sharing raw data.
• Challenge Addressed: MCUNet's architecture search can find networks that are not only small for inference but also feasible for the backward pass, balancing memory for both phases.

The neural architecture search (NAS) component of MCUNet. TinyNAS performs hardware-in-the-loop search to discover optimal network architectures under the severe SRAM and flash memory constraints of a target microcontroller.

• Key Innovation: It uses a two-stage NAS approach: first searching for the best model size (number of parameters), then searching for the best operators (e.g., kernel size, expansion ratios) within that size budget.
• Direct Constraint Optimization: The search reward is directly based on the measured peak memory usage and latency when the candidate model runs on the actual MCU with TinyEngine, not just theoretical FLOPs.

The memory-optimized inference engine co-designed with TinyNAS in MCUNet. It is a lightweight, CMSIS-NN compatible library that replaces frameworks like TensorFlow Lite Micro.

• In-Place Depthwise Convolution: A core optimization that performs depthwise convolution without allocating a large intermediate buffer, drastically reducing peak SRAM usage.
• Operator Fusion & Scheduling: Intelligently fuses layers (e.g., convolution, batch norm, activation) and schedules computation to minimize memory footprint throughout the inference graph.
• Static Memory Planning: Allocates all required memory at compile-time in a single contiguous block, eliminating runtime allocation overhead and fragmentation.

The broader category of automated design techniques to which TinyNAS belongs. HW-NAS optimizes neural network architectures by directly incorporating hardware performance feedback (latency, energy, memory) into the search loop.

• Beyond Accuracy: The search objective is a joint function of model accuracy and hardware cost (e.g., Accuracy / Latency).
• MCU-Specific Challenges: For microcontrollers, the critical constraint is often peak SRAM usage, not just FLOPs, making memory-aware search essential.
• Proxyless Methods: Modern approaches like ProxylessNAS directly evaluate candidates on the target hardware and task, avoiding inaccurate proxy measures.

A complementary approach to efficient model deployment. OFA trains a single large super-network that contains many smaller sub-networks within it via weight sharing.

• Zero-Shot Adaptation: After training the super-network once, many efficient sub-networks for different resource constraints (e.g., 200KB vs. 1MB models) can be extracted without any retraining.
• Contrast with MCUNet: While MCUNet searches a custom architecture for a specific MCU, OFA provides a portfolio of pre-optimized models for a range of devices. They can be combined—using OFA to provide the search space for a HW-NAS like TinyNAS.