Static Scheduling

The entire inference process is analyzed and planned before deployment. The compiler or framework (e.g., TensorFlow Lite Micro) performs a complete traversal of the compute graph, determining:

• The immutable sequence of all kernel executions.
• The precise lifetime of every activation tensor and intermediate buffer.
• The total, fixed memory requirement (RAM footprint). This eliminates all runtime graph interpretation and dynamic scheduling decisions, which are costly on MCUs.

All memory for model weights, activations, and scratch buffers is pre-allocated at compile-time or system startup. This is achieved through:

• Static Memory Allocation: A single, contiguous memory arena is defined.
• Memory Pooling: Fixed-size buffers within the arena are assigned to specific tensors.
• In-Place Computation: Where possible, layer outputs overwrite inputs to reuse memory. The result is zero dynamic memory allocation (malloc/free) during inference, preventing heap fragmentation and guaranteeing predictable memory usage.

By resolving all control flow and resource conflicts at compile-time, the inference loop is reduced to a simple, linear sequence of function calls to pre-optimized kernels. This eliminates:

• Runtime graph traversal and operator dispatch logic.
• Dynamic dependency checking between operations.
• Garbage collection or buffer search overhead. The runtime consists solely of the computation itself, minimizing CPU cycles spent on 'housekeeping' and maximizing efficiency for the actual INT8 inference or fixed-point arithmetic.

Static scheduling allows aggressive operator fusion, a critical optimization for MCUs. The compiler can analyze adjacent operations in the fixed schedule and combine them into a single, custom kernel. Common fusions include:

• Convolution + BatchNorm + Activation (e.g., ReLU): Fused into one operation.
• Depthwise Separable Convolution layers are inherently efficient and often implemented as fused kernels. Fusion reduces intermediate tensor writes to slow external RAM, decreases function call overhead, and enables further low-level optimizations like loop unrolling and SIMD instruction use within the fused kernel.

Because every memory access and instruction sequence is predetermined, the execution time for a single inference pass is highly consistent and bounded. This determinism is crucial for:

• Real-time systems with hard deadlines (e.g., sensor sampling loops).
• Power budgeting, as energy consumption per inference can be accurately modeled.
• System integration, where other tasks on the MCU can be scheduled around the known inference window. Variability is minimized, as there are no cache misses from unknown memory access patterns or delays from runtime allocation.

The primary constraint of static scheduling is its inability to adapt to dynamic runtime conditions. The schedule is optimized for a single, fixed model graph and typical input tensor shapes. This means:

• The model cannot have control flow (e.g., dynamic if statements) that changes the execution path.
• Input tensor dimensions (e.g., batch size, sequence length) are typically fixed.
• Dynamic neural networks or models with adaptive computation are not supported. This trade-off is acceptable for most TinyML applications where the model and task are well-defined and stable.

A memory management strategy where all buffers for model weights, activations, and intermediate tensors are pre-allocated at compile-time. This eliminates runtime allocation overhead and memory fragmentation, creating a completely deterministic memory profile that is essential for real-time systems and safety-critical applications.

• Enables Static Scheduling: The fixed, known memory addresses are a prerequisite for generating a static execution schedule.
• Zero Heap Usage: Avoids dynamic malloc/free, preventing heap fragmentation and ensuring predictable operation over indefinite runtimes.
• Guaranteed Fit: Engineers can definitively prove a model will fit within a device's RAM at compile time.

A directed acyclic graph (DAG) representation of a neural network where nodes represent operations (ops) and edges represent the multidimensional data arrays (tensors) flowing between them. This abstract representation is the input to the compiler that performs static scheduling.

• Optimization Foundation: The graph is analyzed for operator fusion, dead code elimination, and optimal execution ordering.
• Hardware-Agnostic: The same graph can be compiled and scheduled for different microcontroller targets (e.g., Cortex-M4 vs. RISC-V).
• Visualization: Provides a clear map of data dependencies, which is crucial for determining a valid static schedule.

The manual or automated low-level tuning of fundamental neural network operation implementations (kernels) for a specific microcontroller's CPU architecture. While static scheduling determines when and where ops run, kernel optimization defines how efficiently each op executes.

• Architecture-Specific Tuning: Involves hand-writing assembly or using SIMD instructions (like Arm's Helium technology) for ops like convolution and matrix multiplication.
• Libraries: Implemented in highly optimized libraries like CMSIS-NN for Arm Cortex-M or vendor-specific DSP libraries.
• Direct Performance Impact: Optimized kernels directly reduce the latency values used by the static scheduler's cost model.

A compiler optimization that combines multiple sequential neural network operations into a single, fused kernel. This is a critical pass that occurs before static scheduling, as it changes the compute graph.

• Reduces Intermediate Memory: Fusing a Convolution, Batch Normalization, and ReLU activation into one op eliminates the need to write and read the large intermediate tensors to RAM.
• Simplifies the Schedule: Fewer ops in the graph mean a simpler, more efficient static schedule with fewer context switches.
• Common Patterns: Convolution + ReLU, Batch Normalization + Scale, and Fully Connected + Bias are typical fusion candidates.

An optimization where the output of a neural network layer is written directly into the memory location of its (already consumed) input. This drastically reduces the peak RAM footprint, which is a primary constraint for static memory allocation and scheduling.

• Memory Reuse: Identifies tensors in the compute graph whose lifetimes do not overlap, allowing their memory buffers to be shared.
• Critical for Large Models: Enables the deployment of models whose total intermediate tensor size exceeds available RAM, as long as the peak concurrent usage fits.
• Scheduler Dependency: The static scheduler must be aware of these in-place policies to ensure correct data is not overwritten prematurely.