C Array Model

The model is stored as a constant byte array (e.g., const unsigned char g_model[] = {0x12, 0x34...};) within a .c or .h source file. This allows the model data to be compiled directly into the .text or .rodata section of the executable, eliminating the need for a file system. The entire model becomes a read-only memory (ROM) resident object, simplifying deployment to bare-metal microcontrollers.

The inference engine (e.g., a micro interpreter) accesses the model by reading the array directly from flash memory via a pointer. This approach:

• Minimizes RAM usage, as the model weights are not copied to volatile memory.
• Enables execute-in-place (XIP) capabilities on supported hardware.
• Creates a single, monolithic firmware binary that is inherently portable and easy to version control.

C array models are not handwritten. They are generated by a TinyML toolchain or converter tool. Common workflows include:

• Using xxd or similar utilities to convert a binary model file.
• Leveraging framework-specific tools like TensorFlow Lite Micro's xxd conversion or STM32Cube.AI.
• The output is a header file that can be #included directly into the application, abstracting the complex binary representation.

Because the model is constant data known at compile time, the toolchain can perform aggressive ahead-of-time (AOT) optimizations. This includes:

• Constant folding of fixed weights and biases.
• Memory planning to statically allocate buffers for activations (the tensor arena).
• Potential dead code elimination of unused model segments, further reducing the final binary size.

The immutable nature of a compiled-in model provides inherent security advantages:

• Tamper resistance: The model is part of the signed firmware image, protected by the same bootloader and update mechanisms.
• Deterministic behavior: The model cannot be altered at runtime, ensuring consistent inference.
• IP protection: The model weights are obfuscated within the machine code, making direct extraction more difficult than from a standalone file.

This model representation involves key trade-offs:

• No runtime updates: Updating the model requires a full firmware over-the-air (FOTA) update.
• Increased flash usage: The entire model occupies persistent flash memory.
• Toolchain dependency: Model iteration requires re-running the conversion and compilation steps.
• Limited dynamic flexibility: Techniques like weight pruning at runtime or dynamic model selection are not possible.

A memory-efficient inference framework born from the MCUNet co-design research. Instead of a general-purpose interpreter, TinyEngine performs ahead-of-time (AOT) compilation and kernel specialization. It generates ultra-lean, inlined C code where the model weights are hard-coded as constants, and the execution graph is unrolled, minimizing runtime overhead.

• Kernel Fusion: Aggressively fuses layers (e.g., convolution, batch norm, ReLU) into single, hand-optimized operators.
• Patch-based Inference: Processes large inputs (like images) in small, memory-friendly patches to reduce peak RAM usage.

A FlatBuffer model is a neural network serialized using the FlatBuffers cross-platform serialization library. It is the standard, memory-efficient format read by frameworks like TensorFlow Lite Micro (TFLM). The model is stored as a contiguous byte array, enabling direct memory mapping without parsing or copying, which is a prerequisite for conversion into a C array.

• Key Feature: Enables zero-copy deserialization, critical for memory-constrained devices.
• Relationship to C Array: The .tflite FlatBuffer file is the typical input to a conversion tool that outputs a C header file containing the model as a const unsigned char[].

A micro interpreter is the minimal runtime engine within a TinyML framework (e.g., in TFLM) that executes a model. It reads the FlatBuffer or C array model, plans the execution graph, and invokes optimized kernel functions. When using a C array model, the interpreter operates directly on the statically linked array in ROM/Flash.

• Runtime Role: Manages tensor memory (the tensor arena), schedules operators, and handles model I/O.
• Contrast with C Array Model: The C array is the static model data; the micro interpreter is the execution engine that runs it.

The tensor arena is a statically or dynamically allocated block of memory (typically SRAM) used by the micro interpreter to store all intermediate activation tensors and temporary data during inference. Its size is a critical design constraint, determined by the model's memory peak usage.

• Primary Function: Holds input, output, and intermediate layer results.
• Design Trade-off: A larger arena supports more complex models but consumes scarce RAM. The C array model resides in Flash/ROM, separating persistent model weights from volatile activations.

A micro-compiler (e.g., nncase, MicroTVM) is a specialized tool that translates a high-level neural network model into highly optimized, low-level code for a microcontroller. This often includes generating a C array representation as part of its ahead-of-time (AOT) compilation output.

• Process: Performs hardware-aware optimizations like operator fusion and quantization.
• Output: Produces the optimized C array model and, frequently, tailored inference kernel code, eliminating the need for a generic interpreter.

Operator fusion is a critical graph optimization technique where consecutive neural network operations (layers) are combined into a single, compound kernel. This reduces intermediate memory writes/reads and execution overhead, which is vital for microcontroller performance and memory efficiency.

• Example: Fusing a Convolution, Batch Normalization, and ReLU activation into one kernel.
• Impact on C Array: The fused operator graph is what gets serialized into the final C array, resulting in a more efficient executable model structure.

The TinyML deployment workflow is the end-to-end process for getting a model onto a device. The C array model is a key artifact in this pipeline.

Typical Steps:

1. Train & Export: Create a model (e.g., Keras .h5) and convert to a deployable format (e.g., .tflite FlatBuffer).
2. Optimize & Convert: Use a tool (e.g., xxd, xxd -i model.tflite > model.cc) or framework SDK to generate the C header file array.
3. Integrate: Include the .h file in the firmware project, link the model array, and call the inference API.
4. Validate: Test accuracy, latency, and memory usage on the target hardware.