Static Memory Allocation

All memory addresses and sizes are determined and fixed during the compilation and linking phases of the software build process. The linker generates a memory map that statically assigns sections like .data (initialized variables), .bss (uninitialized variables), and stack space. This eliminates the need for a runtime memory allocator (like malloc), which is often too large and non-deterministic for microcontrollers with only kilobytes of RAM.

Because memory layout is known in advance, memory access patterns are predictable. This allows for precise analysis of Worst-Case Execution Time (WCET), a non-negotiable requirement for real-time embedded and TinyML systems. There is no risk of runtime allocation failure or unpredictable delays from garbage collection or heap fragmentation, ensuring inference loops meet their timing deadlines.

Static allocation completely avoids heap fragmentation, a critical failure mode in long-running embedded systems. Fragmentation occurs when dynamic allocations and deallocations of varying sizes leave small, unusable gaps in memory. In static allocation, the memory footprint is constant for the lifetime of the program, guaranteeing all declared buffers remain available and contiguous.

• No allocator metadata overhead
• No risk of allocation failure after days/weeks of runtime
• Essential for mission-critical TinyML deployments

The runtime memory manager is removed from the system, saving precious Flash memory (code space) and CPU cycles. Memory is simply referenced by its fixed address. This overhead reduction is profound in TinyML, where the entire inference engine (e.g., TensorFlow Lite for Microcontrollers) and model must often fit in under 256KB of Flash. The CPU time saved from not managing a heap is dedicated to performing model inference.

The total RAM footprint is the sum of all statically declared buffers, global variables, and the pre-allocated call stack. This size is an absolute upper bound, visible to the developer at compile time. This allows engineers to definitively prove a model will run on a specific microcontroller (e.g., an ARM Cortex-M4 with 128KB RAM). Tools can analyze the map file to confirm no memory region overflows its hardware boundary.

The major trade-off is inflexibility. Memory cannot be reused for different purposes across application phases. The size of every buffer, including the largest intermediate tensor in a neural network, must be known and reserved upfront. This can lead to inefficient memory use if workloads are variable. Techniques to mitigate this include:

• Memory planning to share buffers between non-overlapping operators (operator fusion).
• Using arena-based allocators that statically reserve a pool but allow dynamic partitioning within it.
• Designing models with static, known tensor shapes.

Elliot is a research framework that takes a compiler-centric approach to static allocation. It treats the entire neural network inference as a single, static program. Using LLVM compiler passes, it allocates all tensors to global, statically addressed memory.

• Key Feature: Transforms a neural network into pure, allocation-free C code where every array is a global variable with a fixed address known at compile-time.
• Benefit: Eliminates all runtime allocation overhead and pointer indirection. This allows for precise Worst-Case Execution Time (WCET) analysis, as the memory layout is completely deterministic.
• Trade-off: Less flexible than arena-based approaches, as the model graph cannot change without recompilation.

Essential companion tools for static allocation are memory profilers and planners. These are not runtimes themselves but are used during development to size the memory arena.

• TF Lite Micro's RecordingMicroInterpreter: Runs the model and records the peak memory usage and a detailed allocation timeline, informing the required tensor_arena size.
• Offline Memory Planners: Tools that simulate the inference graph and apply different allocation algorithms (e.g., greedy, best-fit, dynamic programming) to find the minimal contiguous buffer size needed.
• Purpose: These tools turn the challenge of static allocation from guesswork into a measured engineering parameter, ensuring the system is provisioned for the worst-case scenario without wasting RAM.

In TinyML, memory footprint refers to the total amount of RAM (for runtime execution) and Flash/ROM (for model storage) a machine learning model and its inference engine require. Static allocation directly determines the RAM footprint, as all buffers are pre-allocated.

• RAM Footprint: The sum of all tensor buffers, activation memory, and stack space reserved at compile-time.
• Flash Footprint: The size of the model's parameters (weights), constant data, and executable code.

For microcontrollers with only tens to hundreds of kilobytes of total memory, precisely analyzing and minimizing this footprint is essential for feasibility.

Worst-Case Execution Time (WCET) is the maximum possible time a task, such as a complete model inference pass, could take to execute on deterministic hardware. Static memory allocation is a key enabler for predictable WCET analysis.

• Eliminates Allocation Overhead: No time spent on malloc/free or garbage collection.
• Prevents Fragmentation: Guarantees memory is available, avoiding unpredictable delays from compaction or out-of-memory errors.
• Critical for Real-Time Systems: Allows TinyML applications in safety-critical or real-time control loops (e.g., motor control, sensor triggering) to provide strict timing guarantees.

Fixed-point arithmetic is a numerical representation method that uses integers to approximate real numbers, defining a fixed number of fractional bits. It is a software-level optimization often paired with static allocation for ultra-efficient MCU execution.

• Hardware Efficiency: Executes using integer ALUs, avoiding the need for a Floating-Point Unit (FPU), which is often absent or power-intensive on microcontrollers.
• Deterministic Performance: Integer operations have consistent, predictable timing, complementing static allocation's memory determinism.
• Reduced Footprint: Storing weights as 8-bit or 16-bit integers (vs. 32-bit floats) directly shrinks the model's static Flash footprint.

Operator fusion is a compiler optimization that combines multiple sequential neural network layers (e.g., Convolution, BatchNorm, Activation) into a single, compound kernel. This technique synergizes with static memory allocation to minimize intermediate buffer usage.

• Reduces Intermediate Buffers: Fused operators compute results in-place or pass data via registers, eliminating the need to allocate large intermediate tensors in static memory.
• Improves Cache Locality: Reduces memory bandwidth by keeping data closer to the compute unit.
• Compiler-Driven: Performed by edge AI compilers (like TVM, Apache TVM Micro) during the model compilation phase, directly influencing the static memory layout plan.