Model Zoo

The core asset is a library of neural networks pre-trained on common edge tasks (e.g., keyword spotting, visual wake words, anomaly detection) and subsequently optimized for microcontrollers. Optimization involves:

• Post-training quantization to 8-bit or int8 precision.
• Weight pruning to remove redundant connections.
• Architecture modifications for lower peak memory usage. Each model is a ready-to-infer artifact, drastically reducing the engineering effort required to go from concept to deployed firmware.

Models are not generic; they are tailored and validated for specific microcontroller architectures and AI accelerators. A comprehensive zoo includes variants for:

• Arm Cortex-M cores (using CMSIS-NN kernels).
• Espressif ESP32 (using ESP-DL).
• STMicroelectronics STM32 (via STM32Cube.AI).
• Chips with microNPUs like the Arm Ethos-U55. This ensures the provided code or binaries leverage the target's unique instructions and memory hierarchy for maximum efficiency.

Every model entry is accompanied by verifiable performance metrics measured on real hardware, which is critical for system design. Standard benchmarks include:

• Latency (inference time in milliseconds).
• Peak RAM and Flash usage (in kilobytes).
• Accuracy (e.g., F1-score, precision/recall) on standard test datasets.
• Energy consumption (in millijoules per inference). These metrics allow developers to select models that fit their device's strict memory budget and power envelope.

To bridge the gap between the model and a production application, zoos provide reference firmware projects and drivers. This typically includes:

• End-to-end examples (e.g., full source code for a wake-word device).
• Sensor integration code for microphones, accelerometers, or cameras.
• Model invocation code demonstrating the framework's API (e.g., TFLM, MicroTVM runtime).
• Pre-processor and post-processor functions for sensor data and model outputs.

Models are stored in deployment-ready serialization formats accompanied by structured metadata. Key formats include:

• FlatBuffers (the .tflite format for TensorFlow Lite Micro).
• C byte arrays (.cpp/.h files) for direct compilation into firmware.
• ONNX for vendor toolchain input. Metadata describes the model's input/output tensor shapes, data types, normalization parameters, and the license under which it is distributed.

A professional model zoo includes automated pipelines to ensure quality and correctness. This encompasses:

• Unit tests for individual model operations on the target.
• Accuracy validation against a held-out test set on the device.
• Regression testing to catch performance degradation from framework updates.
• Robustness checks for edge-case sensor inputs. This suite guarantees that the model performs as advertised when integrated into a user's application.

A software library or toolchain specifically engineered to enable the deployment and execution of machine learning models on microcontroller-based embedded systems. These frameworks provide the core runtime and kernel libraries.

• Examples: TensorFlow Lite Micro (TFLM), CMSIS-NN, uTensor.
• Key Role: They execute the models provided by a Model Zoo, handling low-level memory management and invoking optimized arithmetic kernels.

The integrated set of software tools used to convert, optimize, and deploy machine learning models onto microcontroller hardware. It is the pipeline that prepares a Model Zoo's pre-trained models for a specific device.

• Components: Includes model converters (e.g., xxd, xxd), compilers (e.g., MicroTVM), optimizers (e.g., EON Compiler), and profiling utilities.
• Workflow: Takes a model from a zoo, applies hardware-specific optimizations like quantization, and outputs deployable code.

A neural network model represented as a constant C/C++ byte array within the source code, enabling direct compilation into a firmware binary. This is the most common deployment format for models from a TinyML Model Zoo.

• Mechanism: The model's weights and architecture are serialized into a header file (e.g., model_data.h).
• Advantage: Eliminates the need for a filesystem on the microcontroller, as the model is stored in read-only program memory (Flash).

A specialized compiler that translates high-level neural network models into highly optimized, low-level code targeted for microcontroller execution. It is a critical component of the toolchain that consumes Model Zoo assets.

• Function: Performs hardware-aware optimizations like operator fusion and scheduling for specific CPU cores or AI coprocessors.
• Examples: The compiler within Apache TVM's MicroTVM, or vendor-specific NPU compilers.

A vendor-specific software development kit that provides libraries, APIs, and tools to develop applications featuring local, on-device ML inference for a family of microcontrollers or processors.

• Purpose: Provides the glue layer between a Model Zoo's optimized model and the target hardware's peripherals and OS.
• Examples: STM32Cube.AI, ESP-DL, and vendor NPU SDKs. These often include proprietary optimizations for their silicon.

The end-to-end process of converting a trained model, optimizing it for target hardware, integrating it into embedded firmware, and validating its performance. A Model Zoo provides the starting point for this workflow.

• Key Stages: 1) Model selection from zoo, 2) Conversion & quantization, 3) Compilation for target, 4) Integration into firmware, 5) Profiling & validation on real hardware.
• Tools: Platforms like Edge Impulse automate much of this workflow, using models from their own curated zoos.