SRAM Footprint

The persistent storage required for the model's learned parameters. In microcontrollers with Harvard architecture, weights often reside in flash memory and are read directly, but many inference runtimes copy them to SRAM for faster access.

• Quantization Impact: INT8 quantization reduces weight footprint by 4× compared to FP32.
• Sparsity: Pruned models with structured sparsity can skip loading zero-valued weights entirely.
• In-Memory Compute: Analog in-memory computing (AIMC) eliminates SRAM weight buffers by performing matrix operations directly in non-volatile memory arrays.

The dominant SRAM consumer in most deep networks. Activations are the intermediate feature maps produced by each layer and must be retained until consumed by subsequent operations.

• Layer Fusion: Fusing convolution, batch normalization, and activation functions eliminates intermediate buffers.
• In-Place Operations: Reusing memory buffers for activations that are no longer needed reduces peak residency.
• Depthwise Separable Convolutions: These factorized ops generate smaller intermediate tensors compared to standard convolutions, directly lowering activation memory.

Scratchpad memory allocated during operator execution for tasks like im2col transformations, quantization scaling, or loop unrolling. These buffers are transient but contribute to the instantaneous peak SRAM requirement.

• Operator Scheduling: Memory planners analyze the full computation graph to allocate and deallocate temporary buffers with minimal fragmentation.
• Arena Allocation: A static memory arena pre-allocated at initialization avoids dynamic allocation overhead and fragmentation, common in TensorFlow Lite Micro runtimes.
• CMSIS-NN kernels are hand-optimized to minimize temporary buffer requirements for Arm Cortex-M processors.

The raw sensor data fed into the model and the final inference results. For RF applications, this often includes complex-valued IQ sample buffers that double the memory requirement compared to real-valued inputs.

• Streaming Inference: Processing data in smaller chunks rather than loading entire sequences reduces input buffer size.
• IQ Data Type Compression: Custom quantization schemes preserve phase and magnitude fidelity while reducing the bit-width of complex samples.
• Sensor Fusion: Combining multiple sensor streams before inference can inflate input memory if not carefully managed.

Memory consumed by the inference engine itself, including the interpreter state, operator resolver tables, and execution stack. This overhead is fixed per runtime but varies significantly between frameworks.

• TensorFlow Lite Micro requires a minimal runtime footprint of only a few kilobytes.
• Apache TVM generates ahead-of-time compiled code that eliminates interpreter overhead entirely.
• HLS4ML translates models directly to register-transfer level code for FPGA synthesis, bypassing traditional runtime memory models.

The algorithmic challenge of determining the optimal execution order to minimize the maximum concurrent memory usage across all tensors. This is a graph optimization problem solved by static memory planners.

• Topological Sorting: Operators are ordered to minimize the lifetime overlap of large tensors.
• Memory Reuse: Tensors with non-overlapping lifetimes share the same SRAM region.
• Roofline Model Analysis: Identifies whether the workload is memory-bound, guiding optimization efforts toward reducing SRAM traffic rather than compute.

The systematic removal of redundant or low-magnitude connections in a neural network to reduce model size and computational complexity. By eliminating near-zero weights, pruning directly reduces the number of parameters that must be stored in SRAM during inference.

• Unstructured pruning removes individual weights, creating sparse matrices
• Structured pruning removes entire channels or filters, enabling dense computation
• Magnitude-based pruning eliminates weights below a threshold
• Can achieve 50-90% parameter reduction with minimal accuracy loss when combined with fine-tuning

A precision reduction technique that maps 32-bit floating-point weights and activations to 8-bit integers, directly cutting SRAM storage requirements by a factor of 4. This is the most widely adopted compression method for microcontroller deployment.

• Per-tensor quantization applies a single scale factor across an entire tensor
• Per-channel quantization uses separate scales for each output channel, preserving accuracy
• Symmetric quantization maps values around zero for efficient integer arithmetic
• Supported natively by CMSIS-NN and TensorFlow Lite Micro runtimes

A compile-time optimization strategy that analyzes the neural network graph to determine the minimum SRAM buffer required for intermediate feature maps. By reusing memory buffers across non-overlapping layers, peak activation memory can be dramatically reduced.

• In-place computation overwrites input tensors with output tensors when dimensions match
• Memory pooling allocates a shared buffer pool for temporary tensors
• Layer fusion eliminates intermediate buffers between consecutive operations
• Critical for fitting models like MCUNet onto devices with under 256KB SRAM

A graph optimization technique that mathematically absorbs batch normalization parameters into the preceding convolutional layer's weights and biases. This eliminates the need to store running mean, variance, scale, and shift parameters in SRAM during inference.

• Reduces per-layer parameter count by four vectors
• Eliminates runtime normalization computations entirely
• Applied automatically by most inference compilers including Apache TVM
• Foldable parameters include gamma, beta, moving mean, and moving variance

A factorized convolution operation that splits standard spatial filtering into a depthwise step and a pointwise step, dramatically reducing both the weight storage footprint and intermediate activation memory. This is the foundational building block of MobileNet architectures.

• Depthwise convolution applies a single filter per input channel
• Pointwise convolution uses 1x1 kernels to combine channel outputs
• Reduces parameters by approximately 8-9x compared to standard 3x3 convolutions
• Enables complex RF signal processing models to fit within tight SRAM budgets

A family of mathematical techniques that approximate high-dimensional weight tensors with smaller constituent factors, directly reducing the SRAM footprint of stored model parameters. Low-rank factorization is particularly effective for fully-connected layers common in RF classification models.

• SVD-based decomposition factorizes weight matrices into low-rank components
• CP decomposition generalizes low-rank factorization to higher-order tensors
• Tucker decomposition separates a tensor into a core tensor and factor matrices
• Can reduce fully-connected layer parameters by 70% or more with controlled reconstruction error